2024-06-06T03:37:07.161 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-06T03:37:07.161 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/fsx}} --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743045 -- /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743045/orig.config.yaml 2024-06-06T03:37:07.212 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-06T03:37:07.305 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743045 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/fsx}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7743045' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 3d746ca5346f34eef7e826a1eb71e59e02b682ae owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5050 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 416/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 3d746ca5346f34eef7e826a1eb71e59e02b682ae syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLpKRlvQeNkC1HpnBiiHKPG+LtB/ZCpt8dVWhDBhXOlPpzltWMFs+Brk/FfjUImkPtdGpqkYYINUB1ftySvJdfs= smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD0n01SCwLFf7hoaHM/UUPWvgsQaNJ47uSfhJkCjdqCJoKHJnPHC52MFQTfnpp+7s+XkknPthqvqcy3W1wMjEDA= smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJyjvALN+r76GCB7xiaEObl15quhqpBcN6NqE0Em1SmhAQfsnY34jwIz+PY8mQFCMps6zX6FDGemv4NLnzSiNxY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-06-05_21:24:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-06T03:37:07.305 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa; will attempt to use it 2024-06-06T03:37:07.306 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks 2024-06-06T03:37:07.306 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-06T03:37:07.308 INFO:teuthology.task.internal:Checking packages... 2024-06-06T03:37:07.327 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-06T03:37:07.328 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-06T03:37:07.328 INFO:teuthology.packaging:ref: None 2024-06-06T03:37:07.328 INFO:teuthology.packaging:tag: None 2024-06-06T03:37:07.328 INFO:teuthology.packaging:branch: squid 2024-06-06T03:37:07.328 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T03:37:07.328 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-06T03:37:07.611 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2960.g3d746ca5 2024-06-06T03:37:07.613 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-06T03:37:07.619 INFO:teuthology.task.internal:no buildpackages task found 2024-06-06T03:37:07.620 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-06T03:37:07.636 INFO:teuthology.task.internal:Saving configuration 2024-06-06T03:37:07.656 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-06T03:37:07.664 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-06T03:37:07.688 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743045', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-06 03:31:44.879092', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLpKRlvQeNkC1HpnBiiHKPG+LtB/ZCpt8dVWhDBhXOlPpzltWMFs+Brk/FfjUImkPtdGpqkYYINUB1ftySvJdfs='} 2024-06-06T03:37:07.711 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743045', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-06 03:31:44.877119', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD0n01SCwLFf7hoaHM/UUPWvgsQaNJ47uSfhJkCjdqCJoKHJnPHC52MFQTfnpp+7s+XkknPthqvqcy3W1wMjEDA='} 2024-06-06T03:37:07.731 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743045', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-06 03:31:44.880841', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJyjvALN+r76GCB7xiaEObl15quhqpBcN6NqE0Em1SmhAQfsnY34jwIz+PY8mQFCMps6zX6FDGemv4NLnzSiNxY='} 2024-06-06T03:37:07.731 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-06T03:37:07.737 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-06-06T03:37:07.737 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-06-06T03:37:07.737 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-06-06T03:37:07.737 INFO:teuthology.run_tasks:Running task console_log... 2024-06-06T03:37:07.811 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f951be33be0>, signals=[15]) 2024-06-06T03:37:07.811 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-06T03:37:07.817 INFO:teuthology.task.internal:Opening connections... 2024-06-06T03:37:07.817 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-06-06T03:37:07.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:37:07.897 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-06-06T03:37:07.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:37:07.972 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2024-06-06T03:37:07.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:37:08.044 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-06T03:37:08.051 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-06-06T03:37:08.069 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-06-06T03:37:08.070 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-06-06T03:37:08.127 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-06-06T03:37:08.127 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-06-06T03:37:08.127 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-06-06T03:37:08.127 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-06-06T03:37:08.128 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-06-06T03:37:08.128 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-06-06T03:37:08.128 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-06T03:37:08.128 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-06-06T03:37:08.128 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-06-06T03:37:08.128 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-06T03:37:08.128 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-06-06T03:37:08.128 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-06T03:37:08.128 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-06T03:37:08.128 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-06T03:37:08.129 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-06-06T03:37:08.149 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-06-06T03:37:08.167 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-06-06T03:37:08.167 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-06-06T03:37:08.223 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="CentOS Stream" 2024-06-06T03:37:08.223 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="9" 2024-06-06T03:37:08.224 INFO:teuthology.orchestra.run.smithi086.stdout:ID="centos" 2024-06-06T03:37:08.224 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="rhel fedora" 2024-06-06T03:37:08.224 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="9" 2024-06-06T03:37:08.224 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el9" 2024-06-06T03:37:08.224 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-06T03:37:08.224 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-06-06T03:37:08.224 INFO:teuthology.orchestra.run.smithi086.stdout:LOGO="fedora-logo-icon" 2024-06-06T03:37:08.224 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-06T03:37:08.224 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://centos.org/" 2024-06-06T03:37:08.224 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-06T03:37:08.224 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-06T03:37:08.224 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-06T03:37:08.225 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-06-06T03:37:08.249 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2024-06-06T03:37:08.266 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2024-06-06T03:37:08.267 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2024-06-06T03:37:08.325 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream" 2024-06-06T03:37:08.325 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="9" 2024-06-06T03:37:08.325 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos" 2024-06-06T03:37:08.325 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora" 2024-06-06T03:37:08.326 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="9" 2024-06-06T03:37:08.326 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el9" 2024-06-06T03:37:08.326 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-06T03:37:08.326 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2024-06-06T03:37:08.326 INFO:teuthology.orchestra.run.smithi154.stdout:LOGO="fedora-logo-icon" 2024-06-06T03:37:08.326 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-06T03:37:08.326 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/" 2024-06-06T03:37:08.326 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-06T03:37:08.326 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-06T03:37:08.326 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-06T03:37:08.327 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2024-06-06T03:37:08.348 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-06T03:37:08.358 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-06T03:37:08.412 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-06T03:37:08.412 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-06-06T03:37:08.414 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-06-06T03:37:08.416 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2024-06-06T03:37:08.433 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-06T03:37:08.439 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-06T03:37:08.440 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-06-06T03:37:08.471 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-06-06T03:37:08.474 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2024-06-06T03:37:08.566 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-06T03:37:08.641 INFO:teuthology.run_tasks:Running task kernel... 2024-06-06T03:37:08.654 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-06-06T03:37:08.655 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-06-06T03:37:08.655 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-06T03:37:08.656 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-06T03:37:08.656 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-06T03:37:08.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:37:08.673 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-06-06T03:37:08.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:37:08.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:37:08.675 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-06-06T03:37:08.676 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-06-06T03:37:08.731 INFO:teuthology.orchestra.run.smithi086.stdout:5.14.0-437.el9.x86_64 2024-06-06T03:37:08.731 INFO:teuthology.task.kernel:Running kernel on smithi086: 5.14.0-437.el9.x86_64 2024-06-06T03:37:08.731 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum install -y kernel 2024-06-06T03:37:08.732 INFO:teuthology.orchestra.run.smithi154.stdout:5.14.0-437.el9.x86_64 2024-06-06T03:37:08.733 INFO:teuthology.task.kernel:Running kernel on smithi154: 5.14.0-437.el9.x86_64 2024-06-06T03:37:08.733 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum install -y kernel 2024-06-06T03:37:08.734 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-437.el9.x86_64 2024-06-06T03:37:08.734 INFO:teuthology.task.kernel:Running kernel on smithi045: 5.14.0-437.el9.x86_64 2024-06-06T03:37:08.734 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2024-06-06T03:37:11.115 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.1 MB 00:00 2024-06-06T03:38:09.679 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 140 kB/s | 8.1 MB 00:59 2024-06-06T03:38:11.546 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-06-06T03:38:30.004 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 104 kB/s | 8.1 MB 01:19 2024-06-06T03:39:25.814 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 90 kB/s | 6.2 MB 01:10 2024-06-06T03:39:27.287 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 17 kB 00:00 2024-06-06T03:39:33.523 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 3.5 MB/s | 21 MB 00:06 2024-06-06T03:39:38.787 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-06-06T03:39:40.885 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-06T03:39:40.885 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-06T03:39:40.977 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-06-06T03:39:40.991 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-06-06T03:39:40.991 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-06-06T03:39:40.992 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-06-06T03:39:40.992 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-06-06T03:39:40.992 INFO:teuthology.orchestra.run.smithi045.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-06T03:39:40.992 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-06-06T03:39:40.992 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-06T03:39:40.992 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-06T03:39:40.992 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-06T03:39:40.992 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T03:39:40.992 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-06-06T03:39:40.992 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-06-06T03:39:40.992 INFO:teuthology.orchestra.run.smithi045.stdout:Install 4 Packages 2024-06-06T03:39:40.992 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T03:39:40.993 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 80 M 2024-06-06T03:39:40.993 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 124 M 2024-06-06T03:39:40.993 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-06-06T03:39:41.206 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] kernel-core-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:39:41.208 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] kernel-modules-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:39:41.214 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] kernel-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:39:41.346 INFO:teuthology.orchestra.run.smithi045.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 3.9 MB/s | 893 kB 00:00 2024-06-06T03:39:41.605 INFO:teuthology.orchestra.run.smithi045.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-06-06T03:39:42.013 INFO:teuthology.orchestra.run.smithi045.stdout:(3/4): kernel-modules-core-5.14.0-452.el9.x86_6 43 MB/s | 29 MB 00:00 2024-06-06T03:39:42.338 INFO:teuthology.orchestra.run.smithi045.stdout:(4/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 28 MB/s | 35 MB 00:01 2024-06-06T03:39:42.341 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-06-06T03:39:42.341 INFO:teuthology.orchestra.run.smithi045.stdout:Total 59 MB/s | 80 MB 00:01 2024-06-06T03:39:43.066 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-06-06T03:39:43.463 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-06-06T03:39:43.463 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-06-06T03:39:44.442 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-06-06T03:39:44.442 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-06-06T03:39:47.023 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-06-06T03:39:47.681 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-06T03:39:47.771 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T03:39:50.602 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T03:39:50.753 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T03:39:55.844 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T03:39:55.893 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:40:00.998 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:40:05.865 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - AppStream 113 kB/s | 19 MB 02:53 2024-06-06T03:40:32.010 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:40:32.052 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:40:32.429 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:40:32.429 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-06T03:40:32.429 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T03:40:32.429 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T03:40:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:40:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T03:40:33.199 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-06-06T03:40:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-06T03:40:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-06T03:40:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-06T03:40:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-06T03:40:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T03:40:33.199 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-06-06T03:40:33.570 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T03:40:33.632 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T03:40:33.633 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T03:40:33.633 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-06T03:40:33.634 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-06T03:40:33.634 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-06T03:40:33.634 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-06-06T03:40:33.634 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=None, version=distro) 2024-06-06T03:40:33.634 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T03:40:33.687 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T03:40:33.687 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T03:40:33.688 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2024-06-06T03:40:33.727 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2024-06-06T03:40:33.728 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2024-06-06T03:40:33.728 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.06 2024-06-06T03:40:33.728 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 68.el9 2024-06-06T03:40:33.728 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2024-06-06T03:40:33.728 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-06T03:40:33.728 INFO:teuthology.orchestra.run.smithi045.stdout:Group : Unspecified 2024-06-06T03:40:33.728 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 8264254 2024-06-06T03:40:33.728 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2024-06-06T03:40:33.728 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-06T03:40:33.728 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-06T03:40:33.728 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-06T03:40:33.728 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-06T03:40:33.729 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : builder@centos.org 2024-06-06T03:40:33.729 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : CentOS 2024-06-06T03:40:33.729 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-06T03:40:33.729 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2024-06-06T03:40:33.729 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2024-06-06T03:40:33.729 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T03:40:33.729 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-06T03:40:33.729 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-06T03:40:33.729 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-06T03:40:33.729 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2024-06-06T03:40:33.729 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T03:40:33.729 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2024-06-06T03:40:33.730 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-06T03:40:33.730 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 5.14.0-452.el9.x86_64 2024-06-06T03:40:33.731 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-06T03:40:34.325 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2024-06-06T03:40:35.873 INFO:teuthology.orchestra.run.smithi045.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-06T03:40:35.911 INFO:teuthology.orchestra.run.smithi045.stderr:done 2024-06-06T03:40:35.913 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-06-06T03:40:35.969 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.4rfCKy7Z2e 2024-06-06T03:40:35.969 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4rfCKy7Z2e 2024-06-06T03:40:36.035 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.4rfCKy7Z2e 2024-06-06T03:40:36.175 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.4rfCKy7Z2e is 6KB 2024-06-06T03:40:36.225 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.4rfCKy7Z2e 2024-06-06T03:40:36.239 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2024-06-06T03:40:36.301 INFO:teuthology.orchestra.run.smithi045.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-06T03:40:36.301 INFO:teuthology.orchestra.run.smithi045.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-06T03:40:36.301 INFO:teuthology.orchestra.run.smithi045.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-06T03:40:36.301 INFO:teuthology.orchestra.run.smithi045.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-06T03:40:36.303 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-06T03:40:36.458 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2024-06-06T03:41:04.107 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - CRB 116 kB/s | 6.2 MB 00:54 2024-06-06T03:41:05.832 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 17 kB 00:00 2024-06-06T03:41:06.461 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-06-06T03:41:06.461 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-06-06T03:41:06.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:41:11.860 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 3.6 MB/s | 21 MB 00:05 2024-06-06T03:41:17.007 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-06-06T03:41:19.005 INFO:teuthology.orchestra.run.smithi154.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-06T03:41:19.005 INFO:teuthology.orchestra.run.smithi154.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-06T03:41:19.102 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-06-06T03:41:19.113 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-06-06T03:41:19.113 INFO:teuthology.orchestra.run.smithi154.stdout: Package Architecture Version Repository Size 2024-06-06T03:41:19.113 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-06-06T03:41:19.113 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-06-06T03:41:19.113 INFO:teuthology.orchestra.run.smithi154.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-06T03:41:19.113 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-06-06T03:41:19.114 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-06T03:41:19.114 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-06T03:41:19.114 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-06T03:41:19.114 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T03:41:19.114 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-06-06T03:41:19.114 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-06-06T03:41:19.114 INFO:teuthology.orchestra.run.smithi154.stdout:Install 4 Packages 2024-06-06T03:41:19.114 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T03:41:19.115 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 80 M 2024-06-06T03:41:19.115 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 124 M 2024-06-06T03:41:19.115 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-06-06T03:41:19.261 INFO:teuthology.orchestra.run.smithi154.stdout:[MIRROR] kernel-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:41:19.263 INFO:teuthology.orchestra.run.smithi154.stdout:[MIRROR] kernel-core-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:41:19.271 INFO:teuthology.orchestra.run.smithi154.stdout:[MIRROR] kernel-modules-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:41:19.875 INFO:teuthology.orchestra.run.smithi154.stdout:(1/4): kernel-core-5.14.0-452.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-06-06T03:41:19.908 INFO:teuthology.orchestra.run.smithi154.stdout:(2/4): kernel-5.14.0-452.el9.x86_64.rpm 1.2 MB/s | 893 kB 00:00 2024-06-06T03:41:20.417 INFO:teuthology.orchestra.run.smithi154.stdout:(3/4): kernel-modules-core-5.14.0-452.el9.x86_6 53 MB/s | 29 MB 00:00 2024-06-06T03:41:26.391 INFO:teuthology.orchestra.run.smithi154.stdout:(4/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 4.8 MB/s | 35 MB 00:07 2024-06-06T03:41:26.394 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-06-06T03:41:26.394 INFO:teuthology.orchestra.run.smithi154.stdout:Total 11 MB/s | 80 MB 00:07 2024-06-06T03:41:27.092 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-06-06T03:41:27.419 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-06-06T03:41:27.419 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-06-06T03:41:28.366 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-06-06T03:41:28.366 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-06-06T03:41:30.808 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-06-06T03:41:31.471 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-06T03:41:31.559 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T03:41:34.241 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T03:41:34.374 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T03:41:36.184 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 106 kB/s | 19 MB 03:05 2024-06-06T03:41:39.117 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T03:41:39.171 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:41:40.551 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 6.2 MB 00:00 2024-06-06T03:41:43.911 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:41:48.510 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 2.5 kB/s | 17 kB 00:06 2024-06-06T03:41:50.103 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-06-06T03:41:55.217 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-06-06T03:41:57.223 INFO:teuthology.orchestra.run.smithi086.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-06T03:41:57.223 INFO:teuthology.orchestra.run.smithi086.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-06T03:41:57.313 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-06-06T03:41:57.328 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-06-06T03:41:57.328 INFO:teuthology.orchestra.run.smithi086.stdout: Package Architecture Version Repository Size 2024-06-06T03:41:57.328 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-06-06T03:41:57.328 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-06-06T03:41:57.328 INFO:teuthology.orchestra.run.smithi086.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-06T03:41:57.328 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-06-06T03:41:57.328 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-06T03:41:57.328 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-06T03:41:57.328 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-06T03:41:57.329 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T03:41:57.329 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-06-06T03:41:57.329 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-06-06T03:41:57.329 INFO:teuthology.orchestra.run.smithi086.stdout:Install 4 Packages 2024-06-06T03:41:57.329 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T03:41:57.329 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 80 M 2024-06-06T03:41:57.330 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 124 M 2024-06-06T03:41:57.330 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-06-06T03:41:57.681 INFO:teuthology.orchestra.run.smithi086.stdout:[MIRROR] kernel-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:41:57.683 INFO:teuthology.orchestra.run.smithi086.stdout:[MIRROR] kernel-core-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:41:57.690 INFO:teuthology.orchestra.run.smithi086.stdout:[MIRROR] kernel-modules-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T03:41:57.818 INFO:teuthology.orchestra.run.smithi086.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 4.0 MB/s | 893 kB 00:00 2024-06-06T03:41:58.102 INFO:teuthology.orchestra.run.smithi086.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-06-06T03:41:58.461 INFO:teuthology.orchestra.run.smithi086.stdout:(3/4): kernel-modules-core-5.14.0-452.el9.x86_6 45 MB/s | 29 MB 00:00 2024-06-06T03:41:58.769 INFO:teuthology.orchestra.run.smithi086.stdout:(4/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 30 MB/s | 35 MB 00:01 2024-06-06T03:41:58.771 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-06-06T03:41:58.772 INFO:teuthology.orchestra.run.smithi086.stdout:Total 56 MB/s | 80 MB 00:01 2024-06-06T03:41:59.467 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-06-06T03:41:59.799 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-06-06T03:41:59.799 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-06-06T03:42:00.747 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-06-06T03:42:00.747 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-06-06T03:42:03.149 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-06-06T03:42:03.782 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-06T03:42:03.895 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T03:42:06.465 DEBUG:teuthology.orchestra.remote:timed out 2024-06-06T03:42:06.543 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T03:42:06.704 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T03:42:11.436 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T03:42:11.535 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:42:13.521 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:42:13.560 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:42:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:42:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-06T03:42:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T03:42:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T03:42:14.703 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:42:14.703 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T03:42:14.703 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-06-06T03:42:14.703 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-06T03:42:14.703 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-06T03:42:14.704 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-06T03:42:14.704 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-06T03:42:14.704 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T03:42:14.704 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-06-06T03:42:15.081 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T03:42:15.153 INFO:teuthology.orchestra.run.smithi154.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T03:42:15.154 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T03:42:15.154 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-06T03:42:15.154 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-06T03:42:15.154 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-06T03:42:15.154 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-06-06T03:42:15.154 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi154.front.sepia.ceph.com, path=None, version=distro) 2024-06-06T03:42:15.155 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T03:42:15.207 INFO:teuthology.orchestra.run.smithi154.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T03:42:15.207 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T03:42:15.207 DEBUG:teuthology.orchestra.run.smithi154:> sudo rpm -qi grub2-tools 2024-06-06T03:42:15.239 INFO:teuthology.orchestra.run.smithi154.stdout:Name : grub2-tools 2024-06-06T03:42:15.239 INFO:teuthology.orchestra.run.smithi154.stdout:Epoch : 1 2024-06-06T03:42:15.239 INFO:teuthology.orchestra.run.smithi154.stdout:Version : 2.06 2024-06-06T03:42:15.239 INFO:teuthology.orchestra.run.smithi154.stdout:Release : 68.el9 2024-06-06T03:42:15.239 INFO:teuthology.orchestra.run.smithi154.stdout:Architecture: x86_64 2024-06-06T03:42:15.239 INFO:teuthology.orchestra.run.smithi154.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-06T03:42:15.239 INFO:teuthology.orchestra.run.smithi154.stdout:Group : Unspecified 2024-06-06T03:42:15.239 INFO:teuthology.orchestra.run.smithi154.stdout:Size : 8264254 2024-06-06T03:42:15.239 INFO:teuthology.orchestra.run.smithi154.stdout:License : GPLv3+ 2024-06-06T03:42:15.239 INFO:teuthology.orchestra.run.smithi154.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-06T03:42:15.239 INFO:teuthology.orchestra.run.smithi154.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-06T03:42:15.239 INFO:teuthology.orchestra.run.smithi154.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-06T03:42:15.239 INFO:teuthology.orchestra.run.smithi154.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-06T03:42:15.240 INFO:teuthology.orchestra.run.smithi154.stdout:Packager : builder@centos.org 2024-06-06T03:42:15.240 INFO:teuthology.orchestra.run.smithi154.stdout:Vendor : CentOS 2024-06-06T03:42:15.240 INFO:teuthology.orchestra.run.smithi154.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-06T03:42:15.240 INFO:teuthology.orchestra.run.smithi154.stdout:Summary : Support tools for GRUB. 2024-06-06T03:42:15.240 INFO:teuthology.orchestra.run.smithi154.stdout:Description : 2024-06-06T03:42:15.240 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T03:42:15.240 INFO:teuthology.orchestra.run.smithi154.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-06T03:42:15.240 INFO:teuthology.orchestra.run.smithi154.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-06T03:42:15.240 INFO:teuthology.orchestra.run.smithi154.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-06T03:42:15.240 INFO:teuthology.orchestra.run.smithi154.stdout:hardware devices. 2024-06-06T03:42:15.240 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T03:42:15.240 INFO:teuthology.orchestra.run.smithi154.stdout:This subpackage provides tools for support of all platforms. 2024-06-06T03:42:15.241 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-06T03:42:15.241 INFO:teuthology.task.kernel:Updating grub on smithi154 to boot 5.14.0-452.el9.x86_64 2024-06-06T03:42:15.242 DEBUG:teuthology.orchestra.run.smithi154:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-06T03:42:15.465 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-06-06T03:42:15.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:42:15.832 INFO:teuthology.orchestra.run.smithi154.stderr:Generating grub configuration file ... 2024-06-06T03:42:15.902 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-06-06T03:42:16.296 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:42:16.316 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-06-06T03:42:16.317 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-452.el9.x86_64"... 2024-06-06T03:42:16.317 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-06-06T03:42:16.332 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-452.el9.x86_64 2024-06-06T03:42:16.332 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-06T03:42:16.332 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-06T03:42:16.332 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-06T03:42:17.160 INFO:teuthology.orchestra.run.smithi154.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-06T03:42:17.189 INFO:teuthology.orchestra.run.smithi154.stderr:done 2024-06-06T03:42:17.191 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-06-06T03:42:17.206 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.GVk1mBx6NQ 2024-06-06T03:42:17.206 DEBUG:teuthology.orchestra.run.smithi154:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.GVk1mBx6NQ 2024-06-06T03:42:17.281 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.GVk1mBx6NQ 2024-06-06T03:42:17.333 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-06-06T03:42:17.334 DEBUG:teuthology.orchestra.run.smithi045:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-06T03:42:17.415 INFO:teuthology.orchestra.run.smithi045.stdout:ttyS1 2024-06-06T03:42:17.425 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.GVk1mBx6NQ is 6KB 2024-06-06T03:42:17.447 DEBUG:teuthology.parallel:result is None 2024-06-06T03:42:17.473 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.GVk1mBx6NQ 2024-06-06T03:42:17.487 DEBUG:teuthology.orchestra.run.smithi154:> sudo /bin/ls /boot/loader/entries || true 2024-06-06T03:42:17.549 INFO:teuthology.orchestra.run.smithi154.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-06T03:42:17.549 INFO:teuthology.orchestra.run.smithi154.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-06T03:42:17.549 INFO:teuthology.orchestra.run.smithi154.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-06T03:42:17.549 INFO:teuthology.orchestra.run.smithi154.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-06T03:42:17.551 DEBUG:teuthology.orchestra.run.smithi154:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-06T03:42:17.706 DEBUG:teuthology.orchestra.run.smithi154:> sudo shutdown -r now 2024-06-06T03:42:45.588 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:42:45.627 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:42:46.010 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:42:46.010 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-06T03:42:46.010 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T03:42:46.011 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T03:42:46.812 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T03:42:46.812 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T03:42:46.812 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-06-06T03:42:46.812 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-06T03:42:46.812 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-06T03:42:46.812 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-06T03:42:46.813 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-06T03:42:46.813 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T03:42:46.813 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-06-06T03:42:47.166 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T03:42:47.259 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T03:42:47.259 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T03:42:47.260 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-06T03:42:47.260 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-06T03:42:47.260 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-06T03:42:47.260 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-06-06T03:42:47.260 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi086.front.sepia.ceph.com, path=None, version=distro) 2024-06-06T03:42:47.260 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T03:42:47.312 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T03:42:47.313 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T03:42:47.313 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -qi grub2-tools 2024-06-06T03:42:47.343 INFO:teuthology.orchestra.run.smithi086.stdout:Name : grub2-tools 2024-06-06T03:42:47.343 INFO:teuthology.orchestra.run.smithi086.stdout:Epoch : 1 2024-06-06T03:42:47.343 INFO:teuthology.orchestra.run.smithi086.stdout:Version : 2.06 2024-06-06T03:42:47.343 INFO:teuthology.orchestra.run.smithi086.stdout:Release : 68.el9 2024-06-06T03:42:47.344 INFO:teuthology.orchestra.run.smithi086.stdout:Architecture: x86_64 2024-06-06T03:42:47.344 INFO:teuthology.orchestra.run.smithi086.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-06T03:42:47.344 INFO:teuthology.orchestra.run.smithi086.stdout:Group : Unspecified 2024-06-06T03:42:47.344 INFO:teuthology.orchestra.run.smithi086.stdout:Size : 8264254 2024-06-06T03:42:47.344 INFO:teuthology.orchestra.run.smithi086.stdout:License : GPLv3+ 2024-06-06T03:42:47.344 INFO:teuthology.orchestra.run.smithi086.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-06T03:42:47.344 INFO:teuthology.orchestra.run.smithi086.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-06T03:42:47.344 INFO:teuthology.orchestra.run.smithi086.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-06T03:42:47.344 INFO:teuthology.orchestra.run.smithi086.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-06T03:42:47.344 INFO:teuthology.orchestra.run.smithi086.stdout:Packager : builder@centos.org 2024-06-06T03:42:47.344 INFO:teuthology.orchestra.run.smithi086.stdout:Vendor : CentOS 2024-06-06T03:42:47.344 INFO:teuthology.orchestra.run.smithi086.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-06T03:42:47.344 INFO:teuthology.orchestra.run.smithi086.stdout:Summary : Support tools for GRUB. 2024-06-06T03:42:47.345 INFO:teuthology.orchestra.run.smithi086.stdout:Description : 2024-06-06T03:42:47.345 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T03:42:47.345 INFO:teuthology.orchestra.run.smithi086.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-06T03:42:47.345 INFO:teuthology.orchestra.run.smithi086.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-06T03:42:47.345 INFO:teuthology.orchestra.run.smithi086.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-06T03:42:47.345 INFO:teuthology.orchestra.run.smithi086.stdout:hardware devices. 2024-06-06T03:42:47.345 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T03:42:47.345 INFO:teuthology.orchestra.run.smithi086.stdout:This subpackage provides tools for support of all platforms. 2024-06-06T03:42:47.346 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-06T03:42:47.346 INFO:teuthology.task.kernel:Updating grub on smithi086 to boot 5.14.0-452.el9.x86_64 2024-06-06T03:42:47.346 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-06T03:42:47.709 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-06-06T03:42:47.709 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-06-06T03:42:47.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:42:47.966 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-06-06T03:42:49.367 INFO:teuthology.orchestra.run.smithi086.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-06T03:42:49.396 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-06-06T03:42:49.398 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-06-06T03:42:49.413 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.ws4xcrg6Fa 2024-06-06T03:42:49.413 DEBUG:teuthology.orchestra.run.smithi086:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ws4xcrg6Fa 2024-06-06T03:42:49.489 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.ws4xcrg6Fa 2024-06-06T03:42:49.629 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.ws4xcrg6Fa is 6KB 2024-06-06T03:42:49.677 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.ws4xcrg6Fa 2024-06-06T03:42:49.691 DEBUG:teuthology.orchestra.run.smithi086:> sudo /bin/ls /boot/loader/entries || true 2024-06-06T03:42:49.752 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-06T03:42:49.752 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-06T03:42:49.753 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-06T03:42:49.753 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-06T03:42:49.754 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-06T03:42:49.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo shutdown -r now 2024-06-06T03:43:19.897 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-06-06T03:43:19.898 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-06-06T03:43:19.899 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:43:38.274 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-06-06T03:43:47.275 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-06-06T03:43:47.276 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:43:47.713 DEBUG:teuthology.orchestra.remote:timed out 2024-06-06T03:43:50.334 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-06-06T03:43:56.714 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-06-06T03:43:56.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:43:59.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2024-06-06T03:44:02.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-06-06T03:44:02.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:44:11.775 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-06-06T03:44:11.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:44:12.236 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-06-06T03:44:12.618 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-06-06T03:44:12.618 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-452.el9.x86_64"... 2024-06-06T03:44:12.619 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-06-06T03:44:12.634 INFO:teuthology.orchestra.run.smithi154.stdout:5.14.0-452.el9.x86_64 2024-06-06T03:44:12.634 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-06T03:44:12.634 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-06T03:44:12.634 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-06T03:44:13.635 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-06-06T03:44:13.635 DEBUG:teuthology.orchestra.run.smithi154:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-06T03:44:13.746 INFO:teuthology.orchestra.run.smithi154.stdout:ttyS1 2024-06-06T03:44:13.774 DEBUG:teuthology.parallel:result is None 2024-06-06T03:44:20.766 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-06-06T03:44:35.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-06-06T03:44:35.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:44:36.072 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-06-06T03:44:36.619 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-06-06T03:44:36.619 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-452.el9.x86_64"... 2024-06-06T03:44:36.619 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-06-06T03:44:36.662 INFO:teuthology.orchestra.run.smithi086.stdout:5.14.0-452.el9.x86_64 2024-06-06T03:44:36.662 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-06T03:44:36.662 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-06T03:44:36.662 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-06T03:44:37.664 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-06-06T03:44:37.664 DEBUG:teuthology.orchestra.run.smithi086:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-06T03:44:37.745 INFO:teuthology.orchestra.run.smithi086.stdout:ttyS1 2024-06-06T03:44:37.790 DEBUG:teuthology.parallel:result is None 2024-06-06T03:44:37.790 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-06T03:44:37.797 INFO:teuthology.task.internal:Creating test directory... 2024-06-06T03:44:37.798 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-06T03:44:37.801 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-06T03:44:37.804 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-06T03:44:37.823 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-06T03:44:37.854 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-06T03:44:37.865 INFO:teuthology.task.internal:Creating archive directory... 2024-06-06T03:44:37.865 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-06T03:44:37.867 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-06T03:44:37.870 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-06T03:44:37.978 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-06T03:44:37.984 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-06T03:44:37.985 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-06T03:44:37.987 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-06T03:44:37.990 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-06T03:44:38.025 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T03:44:38.027 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T03:44:38.030 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T03:44:38.037 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T03:44:38.042 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T03:44:38.043 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T03:44:38.043 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-06T03:44:38.049 INFO:teuthology.task.internal:Configuring sudo... 2024-06-06T03:44:38.050 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-06T03:44:38.085 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-06T03:44:38.089 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-06T03:44:38.118 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-06T03:44:38.127 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-06T03:44:38.127 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-06T03:44:38.155 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-06T03:44:38.157 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-06T03:44:38.177 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T03:44:38.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T03:44:38.302 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T03:44:38.385 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T03:44:38.428 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T03:44:38.428 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-06T03:44:38.495 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T03:44:38.535 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T03:44:38.587 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T03:44:38.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T03:44:38.715 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-06-06T03:44:38.715 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-06T03:44:38.782 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T03:44:38.827 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T03:44:38.884 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T03:44:38.966 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T03:44:39.010 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-06-06T03:44:39.010 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-06T03:44:39.076 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-06-06T03:44:39.078 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-06-06T03:44:39.081 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2024-06-06T03:44:39.119 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T03:44:39.134 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T03:44:39.162 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T03:44:39.515 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-06T03:44:39.523 INFO:teuthology.task.internal:Starting timer... 2024-06-06T03:44:39.523 INFO:teuthology.run_tasks:Running task pcp... 2024-06-06T03:44:39.533 INFO:teuthology.run_tasks:Running task selinux... 2024-06-06T03:44:39.542 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-06T03:44:39.542 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-06-06T03:44:39.635 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-06-06T03:44:39.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2024-06-06T03:44:39.686 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: 2024-06-06T03:44:39.688 DEBUG:teuthology.orchestra.run.smithi154:> sudo service auditd rotate 2024-06-06T03:44:39.746 INFO:teuthology.orchestra.run.smithi154.stdout:Rotating logs: 2024-06-06T03:44:39.748 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-06T03:44:39.748 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-06-06T03:44:39.842 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-06-06T03:44:39.842 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2024-06-06T03:44:39.866 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2024-06-06T03:44:39.867 DEBUG:teuthology.orchestra.run.smithi154:> /usr/sbin/getenforce 2024-06-06T03:44:39.931 INFO:teuthology.orchestra.run.smithi154.stdout:Permissive 2024-06-06T03:44:39.931 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi086.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi154.front.sepia.ceph.com': 'permissive'} 2024-06-06T03:44:39.931 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T03:44:39.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:44:39.963 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T03:44:39.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:44:39.988 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T03:44:40.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T03:44:40.016 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-06T03:44:40.016 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-06-06T03:44:40.045 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2024-06-06T03:44:40.077 DEBUG:teuthology.orchestra.run.smithi154:> sudo /usr/sbin/setenforce permissive 2024-06-06T03:44:40.103 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-06T03:44:40.113 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-06-06T03:44:40.116 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-06-06T03:44:40.242 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-06T03:44:40.252 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-06T03:44:40.253 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi086.front.sepia.ceph.com,smithi154.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-06T03:58:01.730 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi086.front.sepia.ceph.com'), Remote(name='ubuntu@smithi154.front.sepia.ceph.com')] 2024-06-06T03:58:01.765 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-06-06T03:58:01.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:58:01.848 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-06-06T03:58:01.927 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-06-06T03:58:01.927 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-06-06T03:58:01.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:58:02.001 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-06-06T03:58:02.077 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-06-06T03:58:02.077 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-06-06T03:58:02.078 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T03:58:02.150 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-06-06T03:58:02.186 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-06-06T03:58:02.187 INFO:teuthology.run_tasks:Running task clock... 2024-06-06T03:58:02.196 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-06T03:58:02.197 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-06T03:58:02.197 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T03:58:02.199 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-06T03:58:02.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T03:58:02.202 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-06T03:58:02.202 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T03:58:02.229 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-06T03:58:02.230 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-06T03:58:02.242 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-06T03:58:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-06T03:58:02.257 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-06T03:58:02.266 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2024-06-06T03:58:02.271 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-06-06T03:58:02.272 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-06T03:58:02.276 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2024-06-06T03:58:02.283 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-06-06T03:58:02.290 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-06T03:58:02.297 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: ntpd: command not found 2024-06-06T03:58:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-06T03:58:02.305 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-06T03:58:02.308 INFO:teuthology.orchestra.run.smithi154.stdout:506 Cannot talk to daemon 2024-06-06T03:58:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-06T03:58:02.322 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-06T03:58:02.336 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-06T03:58:02.367 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-06-06T03:58:02.370 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T03:58:02.370 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-06-06T03:58:02.370 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:58:02.371 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:58:02.371 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:58:02.371 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:58:02.373 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-06-06T03:58:02.377 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T03:58:02.377 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-06-06T03:58:02.377 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:58:02.377 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:58:02.377 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:58:02.377 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:58:02.401 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-06-06T03:58:02.404 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T03:58:02.404 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-06-06T03:58:02.404 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:58:02.404 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:58:02.405 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:58:02.405 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T03:58:02.405 INFO:teuthology.run_tasks:Running task install... 2024-06-06T03:58:02.412 DEBUG:teuthology.task.install:project ceph 2024-06-06T03:58:02.412 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'}} 2024-06-06T03:58:02.412 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-06T03:58:02.413 INFO:teuthology.task.install:Using flavor: default 2024-06-06T03:58:02.418 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-06T03:58:02.418 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-06-06T03:58:02.418 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-06T03:58:02.419 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T03:58:02.420 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-06T03:58:02.421 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T03:58:02.422 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-06T03:58:02.422 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T03:58:02.561 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-06T03:58:02.561 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-06T03:58:02.573 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-06T03:58:02.573 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-06T03:58:02.643 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-06T03:58:02.644 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-06T03:58:02.675 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-06T03:58:02.675 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T03:58:02.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-06T03:58:02.702 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-06T03:58:02.702 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-06T03:58:02.740 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-06T03:58:02.741 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-06-06T03:58:02.741 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-06T03:58:02.767 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-06T03:58:02.767 DEBUG:teuthology.orchestra.run.smithi154:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-06T03:58:02.769 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-06T03:58:02.813 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-06T03:58:02.814 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-06-06T03:58:02.814 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-06T03:58:02.832 DEBUG:teuthology.orchestra.run.smithi154:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-06T03:58:02.839 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-06T03:58:02.839 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-06T03:58:02.845 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-06T03:58:02.905 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-06T03:58:02.907 DEBUG:teuthology.orchestra.run.smithi154:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-06T03:58:02.910 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-06-06T03:58:02.911 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-06-06T03:58:02.971 INFO:teuthology.orchestra.run.smithi154.stdout:check_obsoletes = 1 2024-06-06T03:58:02.972 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum clean all 2024-06-06T03:58:02.979 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-06T03:58:03.042 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2024-06-06T03:58:03.044 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2024-06-06T03:58:03.150 INFO:teuthology.orchestra.run.smithi045.stdout:45 files removed 2024-06-06T03:58:03.177 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-06T03:58:03.218 INFO:teuthology.orchestra.run.smithi154.stdout:45 files removed 2024-06-06T03:58:03.249 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-06T03:58:03.268 INFO:teuthology.orchestra.run.smithi086.stdout:45 files removed 2024-06-06T03:58:03.294 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-06T03:58:03.870 INFO:teuthology.orchestra.run.smithi154.stdout:ceph packages for x86_64 243 kB/s | 83 kB 00:00 2024-06-06T03:58:03.886 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 199 kB/s | 83 kB 00:00 2024-06-06T03:58:03.969 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 207 kB/s | 83 kB 00:00 2024-06-06T03:58:04.237 INFO:teuthology.orchestra.run.smithi154.stdout:ceph noarch packages 37 kB/s | 11 kB 00:00 2024-06-06T03:58:04.328 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-06-06T03:58:04.344 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 36 kB/s | 11 kB 00:00 2024-06-06T03:58:04.662 INFO:teuthology.orchestra.run.smithi154.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-06-06T03:58:04.670 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 6.4 kB/s | 1.9 kB 00:00 2024-06-06T03:58:04.762 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-06-06T03:59:38.378 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 88 kB/s | 8.1 MB 01:33 2024-06-06T03:59:40.182 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-06-06T03:59:44.399 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-06-06T03:59:46.724 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 17 kB 00:01 2024-06-06T03:59:52.894 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 3.5 MB/s | 21 MB 00:06 2024-06-06T03:59:58.198 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - BaseOS 73 kB/s | 8.1 MB 01:53 2024-06-06T03:59:58.240 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-06-06T04:00:00.684 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T04:00:00.684 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T04:00:00.695 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-06T04:00:00.696 INFO:teuthology.orchestra.run.smithi045.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-06T04:00:00.697 INFO:teuthology.orchestra.run.smithi045.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-06T04:00:00.698 INFO:teuthology.orchestra.run.smithi045.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-06T04:00:00.699 INFO:teuthology.orchestra.run.smithi045.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-06T04:00:00.702 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-06T04:00:00.703 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-06T04:00:00.769 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-06-06T04:00:00.788 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-06-06T04:00:00.788 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-06-06T04:00:00.789 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-06-06T04:00:00.789 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-06-06T04:00:00.789 INFO:teuthology.orchestra.run.smithi045.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-06T04:00:00.789 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-06T04:00:00.789 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-06T04:00:00.789 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-06T04:00:00.789 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-06T04:00:00.789 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-06T04:00:00.789 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-06T04:00:00.789 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-06T04:00:00.789 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-06T04:00:00.789 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-06T04:00:00.790 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-06T04:00:00.790 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-06T04:00:00.790 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-06T04:00:00.790 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-06T04:00:00.790 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-06T04:00:00.790 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-06T04:00:00.790 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-06T04:00:00.790 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-06T04:00:00.790 INFO:teuthology.orchestra.run.smithi045.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-06T04:00:00.790 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-06T04:00:00.790 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-06T04:00:00.791 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-06T04:00:00.791 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-06T04:00:00.791 INFO:teuthology.orchestra.run.smithi045.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-06T04:00:00.791 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-06T04:00:00.791 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-06T04:00:00.791 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-06T04:00:00.791 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-06T04:00:00.791 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-06T04:00:00.791 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-06T04:00:00.791 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-06T04:00:00.791 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-06T04:00:00.791 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-06T04:00:00.792 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-06T04:00:00.792 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T04:00:00.792 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-06T04:00:00.792 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-06T04:00:00.792 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-06-06T04:00:00.792 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-06T04:00:00.792 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-06T04:00:00.792 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-06T04:00:00.792 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-06T04:00:00.792 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-06T04:00:00.792 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-06T04:00:00.792 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-06T04:00:00.792 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-06T04:00:00.793 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T04:00:00.793 INFO:teuthology.orchestra.run.smithi045.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-06T04:00:00.793 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-06T04:00:00.793 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-06-06T04:00:00.793 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-06T04:00:00.793 INFO:teuthology.orchestra.run.smithi045.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-06T04:00:00.793 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-06T04:00:00.793 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-06T04:00:00.793 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-06T04:00:00.793 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-06T04:00:00.793 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-06T04:00:00.793 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-06T04:00:00.794 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-06T04:00:00.794 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-06T04:00:00.794 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-06T04:00:00.794 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-06T04:00:00.794 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-06T04:00:00.794 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-06T04:00:00.794 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-06T04:00:00.794 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-06T04:00:00.794 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-06T04:00:00.794 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-06T04:00:00.794 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-06T04:00:00.795 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-06T04:00:00.795 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-06T04:00:00.795 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-06T04:00:00.795 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-06T04:00:00.795 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-06T04:00:00.795 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-06T04:00:00.795 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-06T04:00:00.795 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-06T04:00:00.795 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-06T04:00:00.795 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-06T04:00:00.796 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-06T04:00:00.796 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-06T04:00:00.796 INFO:teuthology.orchestra.run.smithi045.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-06T04:00:00.796 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-06T04:00:00.796 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-06T04:00:00.796 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-06T04:00:00.796 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-06T04:00:00.796 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-06T04:00:00.796 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-06T04:00:00.796 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-06T04:00:00.796 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-06T04:00:00.797 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-06T04:00:00.797 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-06T04:00:00.797 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-06T04:00:00.797 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-06T04:00:00.797 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-06T04:00:00.797 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-06T04:00:00.797 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-06T04:00:00.797 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-06T04:00:00.797 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-06T04:00:00.797 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-06T04:00:00.797 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-06T04:00:00.797 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-06T04:00:00.797 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-06T04:00:00.797 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-06T04:00:00.797 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-06T04:00:00.798 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-06T04:00:00.798 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-06T04:00:00.798 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-06T04:00:00.798 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-06T04:00:00.798 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-06T04:00:00.798 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-06T04:00:00.798 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-06T04:00:00.798 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-06T04:00:00.798 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-06T04:00:00.798 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-06T04:00:00.798 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-06T04:00:00.798 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-06T04:00:00.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-06T04:00:00.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-06T04:00:00.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-06T04:00:00.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-06T04:00:00.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-06T04:00:00.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-06T04:00:00.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-06T04:00:00.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-06T04:00:00.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-06T04:00:00.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-06T04:00:00.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-06T04:00:00.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-06T04:00:00.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-06T04:00:00.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-06T04:00:00.800 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-06T04:00:00.800 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-06T04:00:00.800 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-06T04:00:00.800 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-06T04:00:00.800 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-06T04:00:00.800 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-06T04:00:00.800 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-06T04:00:00.800 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-06T04:00:00.800 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-06T04:00:00.800 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-06T04:00:00.800 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-06T04:00:00.800 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-06T04:00:00.800 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-06T04:00:00.800 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-06T04:00:00.801 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-06T04:00:00.801 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-06T04:00:00.801 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-06T04:00:00.801 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-06T04:00:00.801 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-06T04:00:00.801 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-06T04:00:00.801 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-06T04:00:00.801 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-06T04:00:00.801 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-06T04:00:00.801 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-06T04:00:00.801 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-06T04:00:00.801 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-06T04:00:00.801 INFO:teuthology.orchestra.run.smithi045.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-06T04:00:00.801 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-06T04:00:00.801 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-06T04:00:00.802 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2024-06-06T04:00:00.802 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-06T04:00:00.802 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:00:00.802 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-06-06T04:00:00.802 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-06-06T04:00:00.802 INFO:teuthology.orchestra.run.smithi045.stdout:Install 148 Packages 2024-06-06T04:00:00.802 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 11 Packages 2024-06-06T04:00:00.802 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:00:00.802 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 225 M 2024-06-06T04:00:00.803 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-06-06T04:00:01.983 INFO:teuthology.orchestra.run.smithi045.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-06-06T04:00:02.375 INFO:teuthology.orchestra.run.smithi045.stdout:(2/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 9.1 MB/s | 5.4 MB 00:00 2024-06-06T04:00:02.401 INFO:teuthology.orchestra.run.smithi045.stdout:(3/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 2.1 MB/s | 890 kB 00:00 2024-06-06T04:00:02.434 INFO:teuthology.orchestra.run.smithi045.stdout:(4/159): ceph-immutable-object-cache-19.0.0-294 2.4 MB/s | 145 kB 00:00 2024-06-06T04:00:02.634 INFO:teuthology.orchestra.run.smithi045.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 7.5 MB/s | 1.5 MB 00:00 2024-06-06T04:00:03.076 INFO:teuthology.orchestra.run.smithi045.stdout:(6/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 11 MB/s | 4.7 MB 00:00 2024-06-06T04:00:03.135 INFO:teuthology.orchestra.run.smithi045.stdout:(7/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 3.2 MB/s | 2.4 MB 00:00 2024-06-06T04:00:03.785 INFO:teuthology.orchestra.run.smithi045.stdout:(8/159): ceph-common-19.0.0-2946.g3d89accf.el9. 10 MB/s | 21 MB 00:02 2024-06-06T04:00:03.844 INFO:teuthology.orchestra.run.smithi045.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 432 kB/s | 25 kB 00:00 2024-06-06T04:00:04.544 INFO:teuthology.orchestra.run.smithi045.stdout:(10/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 11 MB/s | 17 MB 00:01 2024-06-06T04:00:04.636 INFO:teuthology.orchestra.run.smithi045.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.4 MB/s | 221 kB 00:00 2024-06-06T04:00:04.728 INFO:teuthology.orchestra.run.smithi045.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 296 kB/s | 27 kB 00:00 2024-06-06T04:00:04.811 INFO:teuthology.orchestra.run.smithi045.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 380 kB/s | 32 kB 00:00 2024-06-06T04:00:04.920 INFO:teuthology.orchestra.run.smithi045.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 6.7 MB/s | 747 kB 00:00 2024-06-06T04:00:05.020 INFO:teuthology.orchestra.run.smithi045.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.6 MB/s | 163 kB 00:00 2024-06-06T04:00:05.104 INFO:teuthology.orchestra.run.smithi045.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.5 MB/s | 127 kB 00:00 2024-06-06T04:00:05.196 INFO:teuthology.orchestra.run.smithi045.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 5.3 MB/s | 501 kB 00:00 2024-06-06T04:00:05.597 INFO:teuthology.orchestra.run.smithi045.stdout:(18/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 4.2 MB/s | 10 MB 00:02 2024-06-06T04:00:05.730 INFO:teuthology.orchestra.run.smithi045.stdout:(19/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 10 MB/s | 5.3 MB 00:00 2024-06-06T04:00:05.746 INFO:teuthology.orchestra.run.smithi045.stdout:(20/159): python3-ceph-argparse-19.0.0-2946.g3d 301 kB/s | 45 kB 00:00 2024-06-06T04:00:05.805 INFO:teuthology.orchestra.run.smithi045.stdout:(21/159): python3-ceph-common-19.0.0-2946.g3d89 1.8 MB/s | 135 kB 00:00 2024-06-06T04:00:05.831 INFO:teuthology.orchestra.run.smithi045.stdout:(22/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.9 MB/s | 162 kB 00:00 2024-06-06T04:00:05.881 INFO:teuthology.orchestra.run.smithi045.stdout:(23/159): python3-rados-19.0.0-2946.g3d89accf.e 4.2 MB/s | 321 kB 00:00 2024-06-06T04:00:05.948 INFO:teuthology.orchestra.run.smithi045.stdout:(24/159): python3-rbd-19.0.0-2946.g3d89accf.el9 2.5 MB/s | 300 kB 00:00 2024-06-06T04:00:05.973 INFO:teuthology.orchestra.run.smithi045.stdout:(25/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.1 MB/s | 99 kB 00:00 2024-06-06T04:00:06.008 INFO:teuthology.orchestra.run.smithi045.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.4 MB/s | 85 kB 00:00 2024-06-06T04:00:06.099 INFO:teuthology.orchestra.run.smithi045.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 1.8 MB/s | 171 kB 00:00 2024-06-06T04:00:06.182 INFO:teuthology.orchestra.run.smithi045.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 380 kB/s | 32 kB 00:00 2024-06-06T04:00:06.274 INFO:teuthology.orchestra.run.smithi045.stdout:(29/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 10 MB/s | 3.1 MB 00:00 2024-06-06T04:00:06.416 INFO:teuthology.orchestra.run.smithi045.stdout:(30/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 608 kB/s | 142 kB 00:00 2024-06-06T04:00:06.642 INFO:teuthology.orchestra.run.smithi045.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 9.6 MB/s | 3.5 MB 00:00 2024-06-06T04:00:06.725 INFO:teuthology.orchestra.run.smithi045.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.9 MB/s | 249 kB 00:00 2024-06-06T04:00:06.792 INFO:teuthology.orchestra.run.smithi045.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 742 kB/s | 49 kB 00:00 2024-06-06T04:00:07.218 INFO:teuthology.orchestra.run.smithi045.stdout:(34/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 14 MB/s | 48 MB 00:03 2024-06-06T04:00:07.243 INFO:teuthology.orchestra.run.smithi045.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2946.g3 36 kB/s | 16 kB 00:00 2024-06-06T04:00:07.268 INFO:teuthology.orchestra.run.smithi045.stdout:(36/159): ceph-volume-19.0.0-2946.g3d89accf.el9 5.2 MB/s | 264 kB 00:00 2024-06-06T04:00:07.335 INFO:teuthology.orchestra.run.smithi045.stdout:(37/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 8.1 MB/s | 763 kB 00:00 2024-06-06T04:00:07.360 INFO:teuthology.orchestra.run.smithi045.stdout:(38/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 198 kB/s | 18 kB 00:00 2024-06-06T04:00:07.426 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:00:07.435 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:00:07.511 INFO:teuthology.orchestra.run.smithi045.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 459 kB/s | 80 kB 00:00 2024-06-06T04:00:07.514 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:00:07.569 INFO:teuthology.orchestra.run.smithi045.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-06-06T04:00:07.653 INFO:teuthology.orchestra.run.smithi045.stdout:(41/159): ceph-mgr-diskprediction-local-19.0.0- 6.0 MB/s | 7.4 MB 00:01 2024-06-06T04:00:07.686 INFO:teuthology.orchestra.run.smithi045.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-06-06T04:00:07.711 INFO:teuthology.orchestra.run.smithi045.stdout:(43/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-06-06T04:00:07.745 INFO:teuthology.orchestra.run.smithi045.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 21 MB/s | 1.2 MB 00:00 2024-06-06T04:00:07.770 INFO:teuthology.orchestra.run.smithi045.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-06-06T04:00:07.796 INFO:teuthology.orchestra.run.smithi045.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-06-06T04:00:07.821 INFO:teuthology.orchestra.run.smithi045.stdout:(47/159): python3-ply-3.11-14.el9.noarch.rpm 795 kB/s | 106 kB 00:00 2024-06-06T04:00:07.821 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-06T04:00:07.854 INFO:teuthology.orchestra.run.smithi045.stdout:(48/159): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-06-06T04:00:07.855 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:00:07.858 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:00:08.228 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 500 for http://mirror.servaxnet.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 162.255.226.75) 2024-06-06T04:00:09.422 INFO:teuthology.orchestra.run.smithi045.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 68 kB/s | 107 kB 00:01 2024-06-06T04:00:09.447 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 500 for http://mirror.servaxnet.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 162.255.226.75) 2024-06-06T04:00:09.450 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:00:14.563 INFO:teuthology.orchestra.run.smithi045.stdout:(50/159): bison-3.7.4-5.el9.x86_64.rpm 140 kB/s | 943 kB 00:06 2024-06-06T04:00:19.896 INFO:teuthology.orchestra.run.smithi045.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 59 kB/s | 313 kB 00:05 2024-06-06T04:00:19.954 INFO:teuthology.orchestra.run.smithi045.stdout:(52/159): automake-1.16.2-8.el9.noarch.rpm 57 kB/s | 689 kB 00:12 2024-06-06T04:00:20.422 INFO:teuthology.orchestra.run.smithi045.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 62 kB/s | 33 kB 00:00 2024-06-06T04:00:20.897 INFO:teuthology.orchestra.run.smithi045.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 38 kB/s | 18 kB 00:00 2024-06-06T04:00:21.289 INFO:teuthology.orchestra.run.smithi045.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 28 kB/s | 11 kB 00:00 2024-06-06T04:00:25.230 INFO:teuthology.orchestra.run.smithi045.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 41 kB/s | 160 kB 00:03 2024-06-06T04:00:25.472 INFO:teuthology.orchestra.run.smithi045.stdout:(57/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 555 kB/s | 3.0 MB 00:05 2024-06-06T04:00:29.055 INFO:teuthology.orchestra.run.smithi045.stdout:(58/159): librdkafka-1.6.1-102.el9.x86_64.rpm 185 kB/s | 662 kB 00:03 2024-06-06T04:00:30.597 INFO:teuthology.orchestra.run.smithi045.stdout:(59/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 8.4 kB/s | 45 kB 00:05 2024-06-06T04:00:31.580 INFO:teuthology.orchestra.run.smithi045.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 26 kB/s | 25 kB 00:00 2024-06-06T04:00:31.645 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libzstd-devel-1.5.1-2.el9.x86_64.rpm: Status code: 500 for http://mirror.servaxnet.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/libzstd-devel-1.5.1-2.el9.x86_64.rpm (IP: 162.255.226.75) 2024-06-06T04:00:31.957 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 56 kB/s | 8.1 MB 02:27 2024-06-06T04:00:33.755 INFO:teuthology.orchestra.run.smithi045.stdout:(61/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 22 kB/s | 47 kB 00:02 2024-06-06T04:00:35.789 INFO:teuthology.orchestra.run.smithi045.stdout:(62/159): libtool-2.4.6-45.el9.x86_64.rpm 86 kB/s | 582 kB 00:06 2024-06-06T04:00:35.837 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] lua-5.4.4-4.el9.x86_64.rpm: Status code: 500 for http://mirror.servaxnet.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/lua-5.4.4-4.el9.x86_64.rpm (IP: 162.255.226.75) 2024-06-06T04:00:38.339 INFO:teuthology.orchestra.run.smithi045.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 64 kB/s | 292 kB 00:04 2024-06-06T04:00:39.730 INFO:teuthology.orchestra.run.smithi045.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 48 kB/s | 188 kB 00:03 2024-06-06T04:00:39.981 INFO:teuthology.orchestra.run.smithi045.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 21 kB/s | 35 kB 00:01 2024-06-06T04:01:46.785 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - AppStream 182 kB/s | 19 MB 01:47 2024-06-06T04:01:50.876 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-06-06T04:02:08.284 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - Extras packages 1.0 kB/s | 17 kB 00:16 2024-06-06T04:02:15.823 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] openblas-openmp-0.3.21-2.el9.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for https://mirror.servaxnet.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.21-2.el9.x86_64.rpm [OpenSSL SSL_read: error:0A000126:SSL routines::unexpected eof while reading, errno 0] 2024-06-06T04:02:15.825 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] openssl-devel-3.2.1-1.el9.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for https://mirror.servaxnet.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openssl-devel-3.2.1-1.el9.x86_64.rpm [OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 0] 2024-06-06T04:02:19.046 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] openblas-openmp-0.3.21-2.el9.x86_64.rpm: Status code: 500 for http://mirror.servaxnet.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.21-2.el9.x86_64.rpm (IP: 162.255.226.75) 2024-06-06T04:02:19.259 INFO:teuthology.orchestra.run.smithi045.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 49 kB/s | 4.7 MB 01:39 2024-06-06T04:02:20.771 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Curl error (28): Timeout was reached for https://mirror.servaxnet.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm [Failed to connect to mirror.servaxnet.com port 443: Connection timed out] 2024-06-06T04:02:20.809 INFO:teuthology.orchestra.run.smithi045.stdout:(67/159): elfutils-libelf-devel-0.191-4.el9.x86 169 B/s | 22 kB 02:11 2024-06-06T04:02:23.142 INFO:teuthology.orchestra.run.smithi045.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 419 kB/s | 1.6 MB 00:03 2024-06-06T04:02:23.618 INFO:teuthology.orchestra.run.smithi045.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 280 kB/s | 133 kB 00:00 2024-06-06T04:02:24.293 INFO:teuthology.orchestra.run.smithi045.stdout:(70/159): postgresql-server-13.14-1.el9.x86_64. 8.6 MB/s | 5.8 MB 00:00 2024-06-06T04:02:24.417 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 1.3 MB/s | 21 MB 00:15 2024-06-06T04:02:24.693 INFO:teuthology.orchestra.run.smithi045.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 2.5 MB/s | 1.0 MB 00:00 2024-06-06T04:02:25.060 INFO:teuthology.orchestra.run.smithi045.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 238 kB/s | 87 kB 00:00 2024-06-06T04:02:29.630 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-06-06T04:02:32.024 INFO:teuthology.orchestra.run.smithi154.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T04:02:32.025 INFO:teuthology.orchestra.run.smithi154.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T04:02:32.036 INFO:teuthology.orchestra.run.smithi154.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-06T04:02:32.036 INFO:teuthology.orchestra.run.smithi154.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-06T04:02:32.037 INFO:teuthology.orchestra.run.smithi154.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-06T04:02:32.038 INFO:teuthology.orchestra.run.smithi154.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-06T04:02:32.040 INFO:teuthology.orchestra.run.smithi154.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-06T04:02:32.042 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-06T04:02:32.043 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-06T04:02:32.107 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-06-06T04:02:32.130 INFO:teuthology.orchestra.run.smithi154.stdout:======================================================================================= 2024-06-06T04:02:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-06-06T04:02:32.130 INFO:teuthology.orchestra.run.smithi154.stdout:======================================================================================= 2024-06-06T04:02:32.130 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-06-06T04:02:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-06T04:02:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-06T04:02:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-06T04:02:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-06T04:02:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-06T04:02:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-06T04:02:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-06T04:02:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-06T04:02:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-06T04:02:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-06T04:02:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-06T04:02:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-06T04:02:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-06T04:02:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-06T04:02:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-06T04:02:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-06T04:02:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-06T04:02:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-06T04:02:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-06T04:02:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-06T04:02:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-06T04:02:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-06T04:02:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-06T04:02:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-06T04:02:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-06T04:02:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-06T04:02:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-06T04:02:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-06T04:02:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-06T04:02:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-06T04:02:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-06T04:02:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-06T04:02:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-06T04:02:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-06T04:02:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T04:02:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-06T04:02:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-06T04:02:32.133 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrading: 2024-06-06T04:02:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-06T04:02:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-06T04:02:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-06T04:02:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-06T04:02:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-06T04:02:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-06T04:02:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-06T04:02:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-06T04:02:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T04:02:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-06T04:02:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-06T04:02:32.134 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-06-06T04:02:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-06T04:02:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-06T04:02:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-06T04:02:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-06T04:02:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-06T04:02:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-06T04:02:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-06T04:02:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-06T04:02:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-06T04:02:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-06T04:02:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-06T04:02:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-06T04:02:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-06T04:02:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-06T04:02:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-06T04:02:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-06T04:02:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-06T04:02:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-06T04:02:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-06T04:02:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-06T04:02:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-06T04:02:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-06T04:02:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-06T04:02:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-06T04:02:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-06T04:02:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-06T04:02:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-06T04:02:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-06T04:02:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-06T04:02:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-06T04:02:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-06T04:02:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-06T04:02:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-06T04:02:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-06T04:02:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-06T04:02:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-06T04:02:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-06T04:02:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-06T04:02:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-06T04:02:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-06T04:02:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-06T04:02:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-06T04:02:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-06T04:02:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-06T04:02:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-06T04:02:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-06T04:02:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-06T04:02:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-06T04:02:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-06T04:02:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-06T04:02:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-06T04:02:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-06T04:02:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-06T04:02:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-06T04:02:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-06T04:02:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-06T04:02:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-06T04:02:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-06T04:02:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-06T04:02:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-06T04:02:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-06T04:02:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-06T04:02:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-06T04:02:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-06T04:02:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-06T04:02:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-06T04:02:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-06T04:02:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-06T04:02:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-06T04:02:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-06T04:02:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-06T04:02:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-06T04:02:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-06T04:02:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-06T04:02:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-06T04:02:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-06T04:02:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-06T04:02:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-06T04:02:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-06T04:02:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-06T04:02:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-06T04:02:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-06T04:02:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-06T04:02:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-06T04:02:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-06T04:02:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-06T04:02:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-06T04:02:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-06T04:02:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-06T04:02:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-06T04:02:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-06T04:02:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-06T04:02:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-06T04:02:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-06T04:02:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-06T04:02:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-06T04:02:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-06T04:02:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-06T04:02:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-06T04:02:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-06T04:02:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-06T04:02:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-06T04:02:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-06T04:02:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-06T04:02:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-06T04:02:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-06T04:02:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-06T04:02:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-06T04:02:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-06T04:02:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-06T04:02:32.142 INFO:teuthology.orchestra.run.smithi154.stdout:Installing weak dependencies: 2024-06-06T04:02:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-06T04:02:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:02:32.143 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-06-06T04:02:32.143 INFO:teuthology.orchestra.run.smithi154.stdout:======================================================================================= 2024-06-06T04:02:32.143 INFO:teuthology.orchestra.run.smithi154.stdout:Install 148 Packages 2024-06-06T04:02:32.143 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrade 11 Packages 2024-06-06T04:02:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:02:32.143 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 225 M 2024-06-06T04:02:32.143 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-06-06T04:02:41.573 INFO:teuthology.orchestra.run.smithi154.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-06-06T04:02:42.306 INFO:teuthology.orchestra.run.smithi154.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.2 MB/s | 890 kB 00:00 2024-06-06T04:02:42.398 INFO:teuthology.orchestra.run.smithi154.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.5 MB/s | 145 kB 00:00 2024-06-06T04:02:43.457 INFO:teuthology.orchestra.run.smithi154.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 2.2 MB/s | 2.4 MB 00:01 2024-06-06T04:02:43.549 INFO:teuthology.orchestra.run.smithi154.stdout:(5/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 2.3 MB/s | 5.4 MB 00:02 2024-06-06T04:02:44.349 INFO:teuthology.orchestra.run.smithi154.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 1.7 MB/s | 1.5 MB 00:00 2024-06-06T04:02:45.224 INFO:teuthology.orchestra.run.smithi045.stdout:(73/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 36 kB/s | 4.4 MB 02:05 2024-06-06T04:02:45.757 INFO:teuthology.orchestra.run.smithi154.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 2.1 MB/s | 4.7 MB 00:02 2024-06-06T04:02:45.883 INFO:teuthology.orchestra.run.smithi045.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 93 kB/s | 61 kB 00:00 2024-06-06T04:02:46.849 INFO:teuthology.orchestra.run.smithi154.stdout:(8/159): ceph-common-19.0.0-2946.g3d89accf.el9. 3.7 MB/s | 21 MB 00:05 2024-06-06T04:02:46.908 INFO:teuthology.orchestra.run.smithi154.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 434 kB/s | 25 kB 00:00 2024-06-06T04:02:47.650 INFO:teuthology.orchestra.run.smithi154.stdout:(10/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 5.4 MB/s | 10 MB 00:01 2024-06-06T04:02:47.716 INFO:teuthology.orchestra.run.smithi154.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 3.3 MB/s | 221 kB 00:00 2024-06-06T04:02:47.758 INFO:teuthology.orchestra.run.smithi154.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 651 kB/s | 27 kB 00:00 2024-06-06T04:02:47.800 INFO:teuthology.orchestra.run.smithi154.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 765 kB/s | 32 kB 00:00 2024-06-06T04:02:47.950 INFO:teuthology.orchestra.run.smithi154.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 4.9 MB/s | 747 kB 00:00 2024-06-06T04:02:48.009 INFO:teuthology.orchestra.run.smithi154.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 2.7 MB/s | 163 kB 00:00 2024-06-06T04:02:48.068 INFO:teuthology.orchestra.run.smithi154.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 2.1 MB/s | 127 kB 00:00 2024-06-06T04:02:48.083 INFO:teuthology.orchestra.run.smithi045.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 111 kB/s | 244 kB 00:02 2024-06-06T04:02:48.176 INFO:teuthology.orchestra.run.smithi154.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 4.5 MB/s | 501 kB 00:00 2024-06-06T04:02:48.885 INFO:teuthology.orchestra.run.smithi154.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 7.5 MB/s | 5.3 MB 00:00 2024-06-06T04:02:48.935 INFO:teuthology.orchestra.run.smithi154.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 906 kB/s | 45 kB 00:00 2024-06-06T04:02:48.985 INFO:teuthology.orchestra.run.smithi154.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 2.6 MB/s | 135 kB 00:00 2024-06-06T04:02:49.044 INFO:teuthology.orchestra.run.smithi154.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 2.7 MB/s | 162 kB 00:00 2024-06-06T04:02:49.125 INFO:teuthology.orchestra.run.smithi045.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 238 kB/s | 248 kB 00:01 2024-06-06T04:02:49.136 INFO:teuthology.orchestra.run.smithi154.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 3.4 MB/s | 321 kB 00:00 2024-06-06T04:02:49.220 INFO:teuthology.orchestra.run.smithi154.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 3.5 MB/s | 300 kB 00:00 2024-06-06T04:02:49.267 INFO:teuthology.orchestra.run.smithi045.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 181 kB/s | 26 kB 00:00 2024-06-06T04:02:49.279 INFO:teuthology.orchestra.run.smithi154.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.7 MB/s | 99 kB 00:00 2024-06-06T04:02:49.329 INFO:teuthology.orchestra.run.smithi154.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.7 MB/s | 85 kB 00:00 2024-06-06T04:02:49.392 INFO:teuthology.orchestra.run.smithi045.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-06-06T04:02:49.559 INFO:teuthology.orchestra.run.smithi045.stdout:(79/159): postgresql-contrib-13.14-1.el9.x86_64 32 kB/s | 911 kB 00:28 2024-06-06T04:02:49.678 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] python3-mako-1.1.4-6.el9.noarch.rpm: Status code: 500 for http://mirror.servaxnet.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-mako-1.1.4-6.el9.noarch.rpm (IP: 162.255.226.75) 2024-06-06T04:02:49.687 INFO:teuthology.orchestra.run.smithi154.stdout:(26/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 3.1 MB/s | 17 MB 00:05 2024-06-06T04:02:49.763 INFO:teuthology.orchestra.run.smithi154.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.2 MB/s | 171 kB 00:00 2024-06-06T04:02:49.793 INFO:teuthology.orchestra.run.smithi045.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 149 kB/s | 35 kB 00:00 2024-06-06T04:02:49.830 INFO:teuthology.orchestra.run.smithi154.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 475 kB/s | 32 kB 00:00 2024-06-06T04:02:49.880 INFO:teuthology.orchestra.run.smithi154.stdout:(29/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 5.6 MB/s | 3.1 MB 00:00 2024-06-06T04:02:49.905 INFO:teuthology.orchestra.run.smithi154.stdout:(30/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.8 MB/s | 142 kB 00:00 2024-06-06T04:02:50.306 INFO:teuthology.orchestra.run.smithi154.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 8.3 MB/s | 3.5 MB 00:00 2024-06-06T04:02:50.373 INFO:teuthology.orchestra.run.smithi154.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 3.7 MB/s | 249 kB 00:00 2024-06-06T04:02:50.423 INFO:teuthology.orchestra.run.smithi154.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 987 kB/s | 49 kB 00:00 2024-06-06T04:02:50.482 INFO:teuthology.orchestra.run.smithi154.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 282 kB/s | 16 kB 00:00 2024-06-06T04:02:50.582 INFO:teuthology.orchestra.run.smithi154.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 2.6 MB/s | 264 kB 00:00 2024-06-06T04:02:50.707 INFO:teuthology.orchestra.run.smithi154.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 6.0 MB/s | 763 kB 00:00 2024-06-06T04:02:50.758 INFO:teuthology.orchestra.run.smithi154.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 365 kB/s | 18 kB 00:00 2024-06-06T04:02:50.806 INFO:teuthology.orchestra.run.smithi154.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:02:51.409 INFO:teuthology.orchestra.run.smithi045.stdout:(81/159): python3-mako-1.1.4-6.el9.noarch.rpm 85 kB/s | 172 kB 00:02 2024-06-06T04:02:51.458 INFO:teuthology.orchestra.run.smithi154.stdout:(38/159): ceph-mgr-diskprediction-local-19.0.0- 4.8 MB/s | 7.4 MB 00:01 2024-06-06T04:02:51.483 INFO:teuthology.orchestra.run.smithi154.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 111 kB/s | 80 kB 00:00 2024-06-06T04:02:51.484 INFO:teuthology.orchestra.run.smithi154.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:02:51.486 INFO:teuthology.orchestra.run.smithi154.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:02:52.225 INFO:teuthology.orchestra.run.smithi154.stdout:(40/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 9.1 MB/s | 48 MB 00:05 2024-06-06T04:02:52.660 INFO:teuthology.orchestra.run.smithi045.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 200 kB/s | 250 kB 00:01 2024-06-06T04:02:54.233 INFO:teuthology.orchestra.run.smithi154.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 71 kB/s | 195 kB 00:02 2024-06-06T04:02:55.301 INFO:teuthology.orchestra.run.smithi045.stdout:(83/159): python3-babel-2.9.1-2.el9.noarch.rpm 202 kB/s | 6.0 MB 00:30 2024-06-06T04:02:55.360 INFO:teuthology.orchestra.run.smithi045.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 82 kB/s | 222 kB 00:02 2024-06-06T04:02:55.560 INFO:teuthology.orchestra.run.smithi045.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 211 kB/s | 42 kB 00:00 2024-06-06T04:02:55.967 INFO:teuthology.orchestra.run.smithi154.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 178 kB/s | 803 kB 00:04 2024-06-06T04:02:56.127 INFO:teuthology.orchestra.run.smithi045.stdout:(86/159): python3-packaging-20.9-5.el9.noarch.r 93 kB/s | 77 kB 00:00 2024-06-06T04:02:56.442 INFO:teuthology.orchestra.run.smithi154.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 224 kB/s | 106 kB 00:00 2024-06-06T04:02:56.726 INFO:teuthology.orchestra.run.smithi154.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 477 kB/s | 135 kB 00:00 2024-06-06T04:02:56.818 INFO:teuthology.orchestra.run.smithi045.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 229 kB/s | 159 kB 00:00 2024-06-06T04:02:57.369 INFO:teuthology.orchestra.run.smithi045.stdout:(88/159): python3-pyasn1-modules-0.4.8-6.el9.no 507 kB/s | 279 kB 00:00 2024-06-06T04:02:57.576 INFO:teuthology.orchestra.run.smithi154.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 147 kB/s | 125 kB 00:00 2024-06-06T04:02:57.752 INFO:teuthology.orchestra.run.smithi045.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 133 kB/s | 51 kB 00:00 2024-06-06T04:02:58.386 INFO:teuthology.orchestra.run.smithi045.stdout:(90/159): python3-requests-oauthlib-1.3.0-12.el 85 kB/s | 54 kB 00:00 2024-06-06T04:02:58.393 INFO:teuthology.orchestra.run.smithi154.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 263 kB/s | 215 kB 00:00 2024-06-06T04:02:58.439 INFO:teuthology.orchestra.run.smithi154.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-06T04:02:58.568 INFO:teuthology.orchestra.run.smithi154.stdout:(47/159): automake-1.16.2-8.el9.noarch.rpm 3.9 MB/s | 689 kB 00:00 2024-06-06T04:02:58.571 INFO:teuthology.orchestra.run.smithi154.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:02:58.635 INFO:teuthology.orchestra.run.smithi154.stdout:(48/159): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-06-06T04:02:58.638 INFO:teuthology.orchestra.run.smithi154.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:02:58.677 INFO:teuthology.orchestra.run.smithi154.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-06-06T04:02:58.710 INFO:teuthology.orchestra.run.smithi154.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 651 kB/s | 22 kB 00:00 2024-06-06T04:02:58.753 INFO:teuthology.orchestra.run.smithi154.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-06-06T04:02:58.786 INFO:teuthology.orchestra.run.smithi154.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 979 kB/s | 33 kB 00:00 2024-06-06T04:02:58.870 INFO:teuthology.orchestra.run.smithi154.stdout:(53/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 36 MB/s | 3.0 MB 00:00 2024-06-06T04:02:58.903 INFO:teuthology.orchestra.run.smithi154.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 546 kB/s | 18 kB 00:00 2024-06-06T04:02:58.937 INFO:teuthology.orchestra.run.smithi154.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 334 kB/s | 11 kB 00:00 2024-06-06T04:02:58.979 INFO:teuthology.orchestra.run.smithi154.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-06-06T04:02:59.013 INFO:teuthology.orchestra.run.smithi154.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-06-06T04:02:59.054 INFO:teuthology.orchestra.run.smithi154.stdout:(58/159): librdkafka-1.6.1-102.el9.x86_64.rpm 16 MB/s | 662 kB 00:00 2024-06-06T04:02:59.096 INFO:teuthology.orchestra.run.smithi154.stdout:(59/159): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 582 kB 00:00 2024-06-06T04:02:59.130 INFO:teuthology.orchestra.run.smithi154.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 770 kB/s | 25 kB 00:00 2024-06-06T04:02:59.163 INFO:teuthology.orchestra.run.smithi154.stdout:(61/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-06-06T04:02:59.206 INFO:teuthology.orchestra.run.smithi154.stdout:(62/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 292 kB 00:00 2024-06-06T04:02:59.248 INFO:teuthology.orchestra.run.smithi154.stdout:(63/159): lua-5.4.4-4.el9.x86_64.rpm 4.5 MB/s | 188 kB 00:00 2024-06-06T04:02:59.281 INFO:teuthology.orchestra.run.smithi154.stdout:(64/159): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 35 kB 00:00 2024-06-06T04:02:59.398 INFO:teuthology.orchestra.run.smithi154.stdout:(65/159): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-06-06T04:02:59.507 INFO:teuthology.orchestra.run.smithi154.stdout:(66/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-06-06T04:02:59.574 INFO:teuthology.orchestra.run.smithi154.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-06-06T04:02:59.624 INFO:teuthology.orchestra.run.smithi154.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 18 MB/s | 911 kB 00:00 2024-06-06T04:02:59.666 INFO:teuthology.orchestra.run.smithi154.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 3.1 MB/s | 133 kB 00:00 2024-06-06T04:02:59.800 INFO:teuthology.orchestra.run.smithi154.stdout:(70/159): postgresql-server-13.14-1.el9.x86_64. 43 MB/s | 5.8 MB 00:00 2024-06-06T04:02:59.858 INFO:teuthology.orchestra.run.smithi154.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-06-06T04:02:59.900 INFO:teuthology.orchestra.run.smithi154.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-06-06T04:03:00.034 INFO:teuthology.orchestra.run.smithi154.stdout:(73/159): python3-babel-2.9.1-2.el9.noarch.rpm 45 MB/s | 6.0 MB 00:00 2024-06-06T04:03:00.067 INFO:teuthology.orchestra.run.smithi154.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-06-06T04:03:00.109 INFO:teuthology.orchestra.run.smithi154.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-06-06T04:03:00.151 INFO:teuthology.orchestra.run.smithi154.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-06-06T04:03:00.185 INFO:teuthology.orchestra.run.smithi154.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 762 kB/s | 26 kB 00:00 2024-06-06T04:03:00.219 INFO:teuthology.orchestra.run.smithi154.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 576 kB/s | 19 kB 00:00 2024-06-06T04:03:00.261 INFO:teuthology.orchestra.run.smithi154.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-06-06T04:03:00.294 INFO:teuthology.orchestra.run.smithi154.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 1.0 MB/s | 35 kB 00:00 2024-06-06T04:03:00.419 INFO:teuthology.orchestra.run.smithi154.stdout:(81/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 5.1 MB 00:00 2024-06-06T04:03:00.461 INFO:teuthology.orchestra.run.smithi154.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-06-06T04:03:00.503 INFO:teuthology.orchestra.run.smithi154.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 5.2 MB/s | 222 kB 00:00 2024-06-06T04:03:00.537 INFO:teuthology.orchestra.run.smithi154.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 2.3 MB/s | 77 kB 00:00 2024-06-06T04:03:00.570 INFO:teuthology.orchestra.run.smithi154.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 42 kB 00:00 2024-06-06T04:03:00.613 INFO:teuthology.orchestra.run.smithi154.stdout:(86/159): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-06-06T04:03:00.654 INFO:teuthology.orchestra.run.smithi154.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-06-06T04:03:00.696 INFO:teuthology.orchestra.run.smithi154.stdout:(88/159): python3-pyasn1-modules-0.4.8-6.el9.no 6.5 MB/s | 279 kB 00:00 2024-06-06T04:03:00.730 INFO:teuthology.orchestra.run.smithi154.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.5 MB/s | 51 kB 00:00 2024-06-06T04:03:00.763 INFO:teuthology.orchestra.run.smithi154.stdout:(90/159): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-06-06T04:03:01.064 INFO:teuthology.orchestra.run.smithi154.stdout:(91/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 54 MB/s | 16 MB 00:00 2024-06-06T04:03:01.097 INFO:teuthology.orchestra.run.smithi154.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-06-06T04:03:01.131 INFO:teuthology.orchestra.run.smithi154.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-06-06T04:03:01.173 INFO:teuthology.orchestra.run.smithi154.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-06-06T04:03:01.198 INFO:teuthology.orchestra.run.smithi154.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 1.4 MB/s | 36 kB 00:00 2024-06-06T04:03:01.232 INFO:teuthology.orchestra.run.smithi154.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-06-06T04:03:01.274 INFO:teuthology.orchestra.run.smithi154.stdout:(97/159): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 305 kB 00:00 2024-06-06T04:03:01.307 INFO:teuthology.orchestra.run.smithi154.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2024-06-06T04:03:01.350 INFO:teuthology.orchestra.run.smithi154.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-06-06T04:03:01.383 INFO:teuthology.orchestra.run.smithi154.stdout:(100/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2024-06-06T04:03:01.417 INFO:teuthology.orchestra.run.smithi154.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-06-06T04:03:01.420 INFO:teuthology.orchestra.run.smithi154.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:03:01.450 INFO:teuthology.orchestra.run.smithi154.stdout:(102/159): lua-devel-5.4.4-4.el9.x86_64.rpm 671 kB/s | 22 kB 00:00 2024-06-06T04:03:01.454 INFO:teuthology.orchestra.run.smithi154.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:03:01.509 INFO:teuthology.orchestra.run.smithi154.stdout:(103/159): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-06-06T04:03:02.059 INFO:teuthology.orchestra.run.smithi154.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 997 kB/s | 548 kB 00:00 2024-06-06T04:03:02.401 INFO:teuthology.orchestra.run.smithi154.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 904 kB/s | 309 kB 00:00 2024-06-06T04:03:02.435 INFO:teuthology.orchestra.run.smithi154.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 578 kB/s | 19 kB 00:00 2024-06-06T04:03:06.251 INFO:teuthology.orchestra.run.smithi154.stdout:(107/159): libarrow-9.0.0-11.el9.x86_64.rpm 1.2 MB/s | 4.4 MB 00:03 2024-06-06T04:03:06.293 INFO:teuthology.orchestra.run.smithi154.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 625 kB/s | 26 kB 00:00 2024-06-06T04:03:06.352 INFO:teuthology.orchestra.run.smithi154.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 832 kB/s | 49 kB 00:00 2024-06-06T04:03:06.402 INFO:teuthology.orchestra.run.smithi154.stdout:(110/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 18 kB/s | 253 kB 00:14 2024-06-06T04:03:06.452 INFO:teuthology.orchestra.run.smithi154.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 672 kB/s | 67 kB 00:00 2024-06-06T04:03:06.678 INFO:teuthology.orchestra.run.smithi154.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 550 kB/s | 151 kB 00:00 2024-06-06T04:03:07.053 INFO:teuthology.orchestra.run.smithi154.stdout:(113/159): python3-cryptography-36.0.1-4.el9.x8 98 kB/s | 1.2 MB 00:12 2024-06-06T04:03:07.162 INFO:teuthology.orchestra.run.smithi154.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 401 kB/s | 43 kB 00:00 2024-06-06T04:03:07.212 INFO:teuthology.orchestra.run.smithi154.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 644 kB/s | 32 kB 00:00 2024-06-06T04:03:07.254 INFO:teuthology.orchestra.run.smithi154.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 360 kB/s | 15 kB 00:00 2024-06-06T04:03:07.279 INFO:teuthology.orchestra.run.smithi154.stdout:(117/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.0 MB/s | 839 kB 00:00 2024-06-06T04:03:07.313 INFO:teuthology.orchestra.run.smithi154.stdout:(118/159): python3-asyncssh-2.13.2-5.el9.noarch 863 kB/s | 548 kB 00:00 2024-06-06T04:03:07.421 INFO:teuthology.orchestra.run.smithi154.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-06-06T04:03:07.613 INFO:teuthology.orchestra.run.smithi154.stdout:(120/159): python3-google-auth-2.29.0-1.el9.noa 1.0 MB/s | 206 kB 00:00 2024-06-06T04:03:07.639 INFO:teuthology.orchestra.run.smithi154.stdout:(121/159): python3-cherrypy-18.6.1-2.el9.noarch 998 kB/s | 358 kB 00:00 2024-06-06T04:03:07.739 INFO:teuthology.orchestra.run.smithi154.stdout:(122/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 712 kB/s | 303 kB 00:00 2024-06-06T04:03:07.764 INFO:teuthology.orchestra.run.smithi154.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-06-06T04:03:07.790 INFO:teuthology.orchestra.run.smithi154.stdout:(124/159): python3-grpcio-tools-1.46.7-10.el9.x 959 kB/s | 144 kB 00:00 2024-06-06T04:03:07.806 INFO:teuthology.orchestra.run.smithi154.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 420 kB/s | 18 kB 00:00 2024-06-06T04:03:07.823 INFO:teuthology.orchestra.run.smithi154.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 695 kB/s | 23 kB 00:00 2024-06-06T04:03:07.849 INFO:teuthology.orchestra.run.smithi154.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 465 kB/s | 19 kB 00:00 2024-06-06T04:03:07.866 INFO:teuthology.orchestra.run.smithi154.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 468 kB/s | 20 kB 00:00 2024-06-06T04:03:07.916 INFO:teuthology.orchestra.run.smithi154.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 930 kB/s | 46 kB 00:00 2024-06-06T04:03:08.000 INFO:teuthology.orchestra.run.smithi154.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 947 kB/s | 79 kB 00:00 2024-06-06T04:03:08.067 INFO:teuthology.orchestra.run.smithi154.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 869 kB/s | 58 kB 00:00 2024-06-06T04:03:08.317 INFO:teuthology.orchestra.run.smithi154.stdout:(132/159): python3-pecan-1.4.2-3.el9.noarch.rpm 1.1 MB/s | 272 kB 00:00 2024-06-06T04:03:08.351 INFO:teuthology.orchestra.run.smithi154.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 494 kB/s | 16 kB 00:00 2024-06-06T04:03:08.451 INFO:teuthology.orchestra.run.smithi154.stdout:(134/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 900 kB/s | 90 kB 00:00 2024-06-06T04:03:08.493 INFO:teuthology.orchestra.run.smithi154.stdout:(135/159): python3-pyperclip-1.8.2-2.el9.noarch 678 kB/s | 28 kB 00:00 2024-06-06T04:03:08.535 INFO:teuthology.orchestra.run.smithi154.stdout:(136/159): python3-repoze-lru-0.7-16.el9.noarch 739 kB/s | 31 kB 00:00 2024-06-06T04:03:08.694 INFO:teuthology.orchestra.run.smithi154.stdout:(137/159): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-06-06T04:03:08.769 INFO:teuthology.orchestra.run.smithi154.stdout:(138/159): python3-rsa-4.9-2.el9.noarch.rpm 789 kB/s | 59 kB 00:00 2024-06-06T04:03:08.811 INFO:teuthology.orchestra.run.smithi154.stdout:(139/159): python3-tempora-5.0.0-2.el9.noarch.r 860 kB/s | 36 kB 00:00 2024-06-06T04:03:08.870 INFO:teuthology.orchestra.run.smithi154.stdout:(140/159): python3-kubernetes-26.1.0-2.el9.noar 1.0 MB/s | 1.0 MB 00:01 2024-06-06T04:03:08.887 INFO:teuthology.orchestra.run.smithi154.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-06-06T04:03:08.979 INFO:teuthology.orchestra.run.smithi154.stdout:(142/159): python3-websocket-client-1.2.3-2.el9 981 kB/s | 90 kB 00:00 2024-06-06T04:03:09.096 INFO:teuthology.orchestra.run.smithi154.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.0 MB/s | 230 kB 00:00 2024-06-06T04:03:09.154 INFO:teuthology.orchestra.run.smithi154.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-06-06T04:03:09.346 INFO:teuthology.orchestra.run.smithi154.stdout:(145/159): re2-20211101-3.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2024-06-06T04:03:09.388 INFO:teuthology.orchestra.run.smithi154.stdout:(146/159): python3-grpcio-1.46.7-10.el9.x86_64. 1.1 MB/s | 2.0 MB 00:01 2024-06-06T04:03:09.413 INFO:teuthology.orchestra.run.smithi154.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 983 kB/s | 427 kB 00:00 2024-06-06T04:03:10.514 INFO:teuthology.orchestra.run.smithi154.stdout:(148/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 2.9 MB/s | 3.1 MB 00:01 2024-06-06T04:03:10.722 INFO:teuthology.orchestra.run.smithi154.stdout:(149/159): thrift-0.15.0-2.el9.x86_64.rpm 1.1 MB/s | 1.6 MB 00:01 2024-06-06T04:03:11.422 INFO:teuthology.orchestra.run.smithi154.stdout:(150/159): librados2-19.0.0-2946.g3d89accf.el9. 1.7 MB/s | 3.4 MB 00:02 2024-06-06T04:03:11.466 INFO:teuthology.orchestra.run.smithi154.stdout:[MIRROR] NetworkManager-team-1.47.91-1.el9.x86_64.rpm: Status code: 500 for http://mirror.servaxnet.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-team-1.47.91-1.el9.x86_64.rpm (IP: 162.255.226.75) 2024-06-06T04:03:11.469 INFO:teuthology.orchestra.run.smithi154.stdout:[MIRROR] NetworkManager-team-1.47.91-1.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-team-1.47.91-1.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:03:16.489 INFO:teuthology.orchestra.run.smithi154.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 8.2 kB/s | 41 kB 00:05 2024-06-06T04:03:17.633 INFO:teuthology.orchestra.run.smithi154.stdout:[MIRROR] NetworkManager-tui-1.47.91-1.el9.x86_64.rpm: Status code: 500 for http://mirror.servaxnet.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/NetworkManager-tui-1.47.91-1.el9.x86_64.rpm (IP: 162.255.226.75) 2024-06-06T04:03:17.684 INFO:teuthology.orchestra.run.smithi045.stdout:(91/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 187 kB/s | 5.1 MB 00:27 2024-06-06T04:03:18.101 INFO:teuthology.orchestra.run.smithi045.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 100 kB/s | 42 kB 00:00 2024-06-06T04:03:18.601 INFO:teuthology.orchestra.run.smithi045.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 88 kB/s | 44 kB 00:00 2024-06-06T04:03:19.543 INFO:teuthology.orchestra.run.smithi045.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 234 kB/s | 221 kB 00:00 2024-06-06T04:03:19.768 INFO:teuthology.orchestra.run.smithi045.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 160 kB/s | 36 kB 00:00 2024-06-06T04:03:20.168 INFO:teuthology.orchestra.run.smithi045.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 118 kB/s | 47 kB 00:00 2024-06-06T04:03:20.734 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] socat-1.7.4.1-5.el9.x86_64.rpm: Status code: 500 for http://mirror.servaxnet.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/socat-1.7.4.1-5.el9.x86_64.rpm (IP: 162.255.226.75) 2024-06-06T04:03:22.718 INFO:teuthology.orchestra.run.smithi045.stdout:(97/159): socat-1.7.4.1-5.el9.x86_64.rpm 120 kB/s | 305 kB 00:02 2024-06-06T04:03:23.943 INFO:teuthology.orchestra.run.smithi045.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 47 kB/s | 58 kB 00:01 2024-06-06T04:03:24.902 INFO:teuthology.orchestra.run.smithi045.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 98 kB/s | 94 kB 00:00 2024-06-06T04:03:25.336 INFO:teuthology.orchestra.run.smithi045.stdout:(100/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 147 kB/s | 64 kB 00:00 2024-06-06T04:03:26.077 INFO:teuthology.orchestra.run.smithi045.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 60 kB/s | 45 kB 00:00 2024-06-06T04:03:26.127 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:03:26.173 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 500 for http://mirror.servaxnet.com/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 162.255.226.75) 2024-06-06T04:03:26.736 INFO:teuthology.orchestra.run.smithi045.stdout:(102/159): lua-devel-5.4.4-4.el9.x86_64.rpm 34 kB/s | 22 kB 00:00 2024-06-06T04:03:26.743 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:03:27.101 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 500 for http://mirror.servaxnet.com/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 162.255.226.75) 2024-06-06T04:03:31.136 INFO:teuthology.orchestra.run.smithi154.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 17 kB/s | 248 kB 00:14 2024-06-06T04:03:31.378 INFO:teuthology.orchestra.run.smithi154.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 152 kB/s | 37 kB 00:00 2024-06-06T04:03:31.970 INFO:teuthology.orchestra.run.smithi154.stdout:(154/159): NetworkManager-1.47.91-1.el9.x86_64. 110 kB/s | 2.3 MB 00:21 2024-06-06T04:03:34.852 INFO:teuthology.orchestra.run.smithi045.stdout:(103/159): protobuf-compiler-3.14.0-13.el9.x86_ 106 kB/s | 863 kB 00:08 2024-06-06T04:03:35.294 INFO:teuthology.orchestra.run.smithi045.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-06-06T04:03:35.502 INFO:teuthology.orchestra.run.smithi045.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.4 MB/s | 309 kB 00:00 2024-06-06T04:03:35.528 INFO:teuthology.orchestra.run.smithi045.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 777 kB/s | 19 kB 00:00 2024-06-06T04:03:35.578 INFO:teuthology.orchestra.run.smithi154.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 49 kB/s | 207 kB 00:04 2024-06-06T04:03:36.312 INFO:teuthology.orchestra.run.smithi154.stdout:(156/159): NetworkManager-libnm-1.47.91-1.el9.x 72 kB/s | 1.8 MB 00:25 2024-06-06T04:03:37.570 INFO:teuthology.orchestra.run.smithi045.stdout:(107/159): libarrow-9.0.0-11.el9.x86_64.rpm 2.2 MB/s | 4.4 MB 00:02 2024-06-06T04:03:37.594 INFO:teuthology.orchestra.run.smithi045.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-06-06T04:03:37.612 INFO:teuthology.orchestra.run.smithi045.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 2.8 MB/s | 49 kB 00:00 2024-06-06T04:03:37.645 INFO:teuthology.orchestra.run.smithi045.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-06-06T04:03:37.728 INFO:teuthology.orchestra.run.smithi045.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-06-06T04:03:38.270 INFO:teuthology.orchestra.run.smithi045.stdout:(112/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.5 MB/s | 839 kB 00:00 2024-06-06T04:03:38.562 INFO:teuthology.orchestra.run.smithi045.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-06-06T04:03:38.613 INFO:teuthology.orchestra.run.smithi045.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 869 kB/s | 43 kB 00:00 2024-06-06T04:03:38.654 INFO:teuthology.orchestra.run.smithi045.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 774 kB/s | 32 kB 00:00 2024-06-06T04:03:38.688 INFO:teuthology.orchestra.run.smithi045.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 450 kB/s | 15 kB 00:00 2024-06-06T04:03:38.738 INFO:teuthology.orchestra.run.smithi045.stdout:(117/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-06-06T04:03:38.779 INFO:teuthology.orchestra.run.smithi154.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 38 kB/s | 260 kB 00:06 2024-06-06T04:03:38.947 INFO:teuthology.orchestra.run.smithi045.stdout:(118/159): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2024-06-06T04:03:39.180 INFO:teuthology.orchestra.run.smithi045.stdout:(119/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.3 MB/s | 303 kB 00:00 2024-06-06T04:03:39.355 INFO:teuthology.orchestra.run.smithi045.stdout:(120/159): python3-google-auth-2.29.0-1.el9.noa 1.1 MB/s | 206 kB 00:00 2024-06-06T04:03:40.364 INFO:teuthology.orchestra.run.smithi045.stdout:(121/159): python3-grpcio-1.46.7-10.el9.x86_64. 2.0 MB/s | 2.0 MB 00:01 2024-06-06T04:03:40.472 INFO:teuthology.orchestra.run.smithi045.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-06-06T04:03:40.506 INFO:teuthology.orchestra.run.smithi045.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 319 kB/s | 11 kB 00:00 2024-06-06T04:03:40.531 INFO:teuthology.orchestra.run.smithi045.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-06-06T04:03:40.573 INFO:teuthology.orchestra.run.smithi045.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 558 kB/s | 23 kB 00:00 2024-06-06T04:03:40.615 INFO:teuthology.orchestra.run.smithi045.stdout:(126/159): python3-jaraco-functools-3.5.0-2.el9 467 kB/s | 19 kB 00:00 2024-06-06T04:03:40.649 INFO:teuthology.orchestra.run.smithi045.stdout:(127/159): python3-jaraco-text-3.2.0-6.el9.noar 588 kB/s | 20 kB 00:00 2024-06-06T04:03:41.282 INFO:teuthology.orchestra.run.smithi045.stdout:(128/159): python3-kubernetes-26.1.0-2.el9.noar 1.6 MB/s | 1.0 MB 00:00 2024-06-06T04:03:41.341 INFO:teuthology.orchestra.run.smithi045.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 795 kB/s | 46 kB 00:00 2024-06-06T04:03:41.416 INFO:teuthology.orchestra.run.smithi045.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-06-06T04:03:41.483 INFO:teuthology.orchestra.run.smithi045.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 864 kB/s | 58 kB 00:00 2024-06-06T04:03:41.708 INFO:teuthology.orchestra.run.smithi045.stdout:(132/159): python3-pecan-1.4.2-3.el9.noarch.rpm 1.2 MB/s | 272 kB 00:00 2024-06-06T04:03:41.742 INFO:teuthology.orchestra.run.smithi045.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 492 kB/s | 16 kB 00:00 2024-06-06T04:03:41.826 INFO:teuthology.orchestra.run.smithi045.stdout:(134/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-06-06T04:03:41.867 INFO:teuthology.orchestra.run.smithi045.stdout:(135/159): python3-pyperclip-1.8.2-2.el9.noarch 680 kB/s | 28 kB 00:00 2024-06-06T04:03:41.909 INFO:teuthology.orchestra.run.smithi045.stdout:(136/159): python3-repoze-lru-0.7-16.el9.noarch 738 kB/s | 31 kB 00:00 2024-06-06T04:03:42.060 INFO:teuthology.orchestra.run.smithi045.stdout:(137/159): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-06-06T04:03:42.127 INFO:teuthology.orchestra.run.smithi045.stdout:(138/159): python3-rsa-4.9-2.el9.noarch.rpm 886 kB/s | 59 kB 00:00 2024-06-06T04:03:42.177 INFO:teuthology.orchestra.run.smithi045.stdout:(139/159): python3-tempora-5.0.0-2.el9.noarch.r 717 kB/s | 36 kB 00:00 2024-06-06T04:03:42.210 INFO:teuthology.orchestra.run.smithi045.stdout:(140/159): python3-typing-extensions-4.4.0-2.el 1.5 MB/s | 51 kB 00:00 2024-06-06T04:03:42.302 INFO:teuthology.orchestra.run.smithi045.stdout:(141/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-06-06T04:03:42.336 INFO:teuthology.orchestra.run.smithi045.stdout:(142/159): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2024-06-06T04:03:42.394 INFO:teuthology.orchestra.run.smithi045.stdout:(143/159): python3-werkzeug-2.0.3-3.el9.1.noarc 7.2 MB/s | 427 kB 00:00 2024-06-06T04:03:42.428 INFO:teuthology.orchestra.run.smithi045.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 598 kB/s | 20 kB 00:00 2024-06-06T04:03:42.511 INFO:teuthology.orchestra.run.smithi045.stdout:(145/159): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-06-06T04:03:43.720 INFO:teuthology.orchestra.run.smithi045.stdout:(146/159): thrift-0.15.0-2.el9.x86_64.rpm 1.3 MB/s | 1.6 MB 00:01 2024-06-06T04:03:44.329 INFO:teuthology.orchestra.run.smithi045.stdout:(147/159): librados2-19.0.0-2946.g3d89accf.el9. 5.6 MB/s | 3.4 MB 00:00 2024-06-06T04:03:44.587 INFO:teuthology.orchestra.run.smithi045.stdout:(148/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 12 MB/s | 3.1 MB 00:00 2024-06-06T04:03:44.762 INFO:teuthology.orchestra.run.smithi045.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-06-06T04:03:44.829 INFO:teuthology.orchestra.run.smithi045.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-06-06T04:03:44.871 INFO:teuthology.orchestra.run.smithi045.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 995 kB/s | 41 kB 00:00 2024-06-06T04:03:44.913 INFO:teuthology.orchestra.run.smithi045.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-06-06T04:03:44.947 INFO:teuthology.orchestra.run.smithi045.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 1.1 MB/s | 37 kB 00:00 2024-06-06T04:03:44.989 INFO:teuthology.orchestra.run.smithi045.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.9 MB/s | 207 kB 00:00 2024-06-06T04:03:45.030 INFO:teuthology.orchestra.run.smithi045.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 6.1 MB/s | 260 kB 00:00 2024-06-06T04:03:45.089 INFO:teuthology.orchestra.run.smithi045.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 23 MB/s | 1.3 MB 00:00 2024-06-06T04:03:45.164 INFO:teuthology.orchestra.run.smithi045.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-06-06T04:04:25.257 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 84 kB/s | 19 MB 03:52 2024-06-06T04:04:35.112 INFO:teuthology.orchestra.run.smithi154.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 23 kB/s | 1.3 MB 00:59 2024-06-06T04:04:41.940 INFO:teuthology.orchestra.run.smithi045.stdout:(158/159): python3-protobuf-3.14.0-13.el9.noarc 2.5 kB/s | 269 kB 01:46 2024-06-06T04:04:50.586 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] python3-scipy-1.6.2-8.el9.x86_64.rpm: Curl error (18): Transferred a partial file for http://mirror.servaxnet.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-scipy-1.6.2-8.el9.x86_64.rpm [transfer closed with 11030903 bytes remaining to read] 2024-06-06T04:05:07.732 INFO:teuthology.orchestra.run.smithi154.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 27 kB/s | 2.4 MB 01:31 2024-06-06T04:05:07.736 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-06-06T04:05:07.736 INFO:teuthology.orchestra.run.smithi154.stdout:Total 1.4 MB/s | 225 MB 02:35 2024-06-06T04:05:08.920 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-06-06T04:05:09.079 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-06-06T04:05:09.079 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-06-06T04:05:11.323 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-06-06T04:05:11.323 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-06-06T04:05:17.586 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-06-06T04:05:17.678 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-06T04:05:17.716 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-06T04:05:17.788 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-06T04:05:17.828 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-06T04:05:17.854 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-06T04:05:17.904 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-06T04:05:18.239 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-06T04:05:18.355 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T04:05:18.477 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T04:05:18.506 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T04:05:18.607 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T04:05:18.636 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T04:05:18.677 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-06T04:05:18.885 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-06T04:05:18.949 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-06T04:05:18.995 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T04:05:19.083 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T04:05:19.112 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T04:05:19.175 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T04:05:19.197 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T04:05:19.220 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T04:05:19.416 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T04:05:19.591 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T04:05:19.744 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T04:05:19.788 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T04:05:19.892 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T04:05:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T04:05:20.173 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T04:05:20.279 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-06T04:05:20.374 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-06T04:05:20.412 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-06T04:05:20.446 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-06T04:05:20.511 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-06T04:05:20.554 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-06T04:05:20.596 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-06T04:05:20.630 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-06T04:05:20.657 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-06T04:05:20.726 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-06T04:05:20.780 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-06T04:05:20.810 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-06T04:05:20.854 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-06T04:05:20.931 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-06T04:05:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-06T04:05:21.008 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-06T04:05:21.036 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-06T04:05:21.108 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-06T04:05:21.160 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-06T04:05:21.235 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-06T04:05:21.368 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-06T04:05:21.421 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-06T04:05:21.956 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-06T04:05:22.042 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-06T04:05:22.070 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-06T04:05:22.116 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-06T04:05:22.156 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-06T04:05:22.185 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-06T04:05:22.227 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-06T04:05:22.379 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-06T04:05:22.432 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-06T04:05:22.487 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-06T04:05:22.561 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-06T04:05:22.591 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-06T04:05:22.627 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-06T04:05:22.675 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-06T04:05:22.710 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-06T04:05:22.753 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-06T04:05:22.796 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-06T04:05:22.933 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-06T04:05:22.989 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-06T04:05:23.030 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-06T04:05:23.125 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-06T04:05:23.156 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-06T04:05:23.198 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-06T04:05:23.229 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-06T04:05:23.254 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-06T04:05:23.347 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-06T04:05:23.475 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-06T04:05:23.512 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-06T04:05:23.540 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-06T04:05:23.574 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-06T04:05:23.613 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-06T04:05:23.645 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-06T04:05:23.675 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-06T04:05:23.709 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-06T04:05:23.751 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-06T04:05:23.792 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-06T04:05:23.826 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-06T04:05:23.914 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-06T04:05:23.966 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-06T04:05:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-06T04:05:24.044 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-06T04:05:24.143 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-06T04:05:24.199 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-06T04:05:24.921 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-06T04:05:24.970 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-06T04:05:25.004 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-06T04:05:25.103 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-06T04:05:25.160 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-06T04:05:25.213 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-06T04:05:25.301 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-06T04:05:25.332 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-06T04:05:25.511 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-06T04:05:25.546 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-06T04:05:25.626 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-06T04:05:26.278 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-06T04:05:26.395 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-06T04:05:27.668 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-06T04:05:27.751 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-06T04:05:27.779 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-06T04:05:27.820 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-06T04:05:28.144 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-06T04:05:28.192 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T04:05:28.274 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T04:05:28.305 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-06T04:05:28.350 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-06T04:05:28.823 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-06T04:05:28.853 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T04:05:28.907 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T04:05:28.938 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-06T04:05:31.212 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T04:05:31.260 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T04:05:31.815 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T04:05:31.863 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T04:05:31.953 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T04:05:31.953 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-06T04:05:31.953 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-06T04:05:31.953 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:05:31.989 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T04:05:43.679 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T04:05:43.680 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /sys 2024-06-06T04:05:43.680 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /proc 2024-06-06T04:05:43.680 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /mnt 2024-06-06T04:05:43.680 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /var/tmp 2024-06-06T04:05:43.680 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /home 2024-06-06T04:05:43.680 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /root 2024-06-06T04:05:43.680 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /tmp 2024-06-06T04:05:43.680 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:05:43.845 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T04:05:43.861 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T04:05:43.861 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:05:43.861 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-06T04:05:43.861 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:05:45.358 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T04:05:45.385 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T04:05:45.385 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:05:45.385 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-06T04:05:45.385 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T04:05:45.385 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T04:05:45.385 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:05:45.609 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T04:05:45.632 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T04:05:45.632 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:05:45.632 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-06T04:05:45.632 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T04:05:45.633 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T04:05:45.633 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:05:46.084 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T04:05:46.107 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T04:05:46.107 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:05:46.107 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-06T04:05:46.107 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T04:05:46.108 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T04:05:46.108 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:05:46.215 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-06T04:05:46.272 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-06T04:05:46.329 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-06T04:05:46.515 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-06T04:05:46.569 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-06T04:05:46.647 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-06T04:05:46.741 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-06T04:05:46.885 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-06T04:05:46.925 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-06T04:05:46.957 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-06T04:05:46.985 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-06T04:05:47.414 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T04:05:47.442 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T04:05:47.473 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-06T04:05:47.536 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-06T04:05:47.565 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T04:05:48.610 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T04:05:48.649 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T04:05:49.617 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T04:05:49.641 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T04:05:49.745 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T04:05:49.904 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-06T04:05:49.953 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T04:05:49.976 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T04:05:49.976 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:05:49.976 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-06T04:05:49.976 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T04:05:49.976 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T04:05:49.976 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:05:50.028 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T04:05:50.039 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T04:05:50.137 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-06T04:05:50.196 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-06T04:05:51.094 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-06T04:05:51.121 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T04:05:51.145 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T04:05:51.145 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:05:51.145 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-06T04:05:51.145 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T04:05:51.145 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T04:05:51.145 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:05:51.197 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T04:05:51.219 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T04:05:51.219 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:05:51.219 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-06T04:05:51.220 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:05:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T04:05:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T04:05:51.287 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:05:51.287 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-06T04:05:51.287 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T04:05:51.287 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T04:05:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:05:51.590 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T04:05:51.613 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T04:05:51.614 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:05:51.614 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-06T04:05:51.614 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T04:05:51.614 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T04:05:51.614 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:05:56.143 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-06T04:05:56.185 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-06T04:05:56.225 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-06T04:05:56.257 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-06T04:05:56.284 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-06T04:05:56.470 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-06T04:05:56.514 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-06T04:05:56.538 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-06T04:05:56.578 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-06T04:05:56.608 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-06T04:05:56.646 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T04:05:57.214 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T04:05:57.252 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T04:05:57.272 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T04:05:58.113 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-06T04:05:58.198 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-06T04:05:58.239 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-06T04:05:58.330 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-06T04:05:58.360 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-06T04:05:58.384 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T04:05:58.410 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-06T04:05:58.436 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-06T04:05:58.436 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T04:05:58.521 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T04:05:58.562 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-06T04:05:58.562 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T04:05:58.646 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T04:05:58.671 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-06T04:05:58.677 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T04:05:58.677 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T04:05:58.856 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T04:05:58.856 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T04:05:58.949 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T04:05:58.973 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-06T04:05:59.040 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T04:06:04.472 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T04:06:04.472 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-06T04:06:04.472 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-06T04:06:04.472 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-06T04:06:04.472 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-06T04:06:04.472 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-06T04:06:04.472 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-06T04:06:04.472 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-06T04:06:04.473 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T04:06:04.473 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T04:06:04.473 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T04:06:04.473 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-06T04:06:04.473 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-06T04:06:04.473 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-06T04:06:04.473 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T04:06:04.473 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-06T04:06:04.473 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-06T04:06:04.473 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-06T04:06:04.473 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-06T04:06:04.473 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-06T04:06:04.473 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-06T04:06:04.474 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-06T04:06:04.474 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-06T04:06:04.474 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-06T04:06:04.474 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-06T04:06:04.474 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-06T04:06:04.474 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-06T04:06:04.474 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-06T04:06:04.474 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-06T04:06:04.474 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-06T04:06:04.474 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-06T04:06:04.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-06T04:06:04.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-06T04:06:04.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-06T04:06:04.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-06T04:06:04.486 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-06T04:06:04.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-06T04:06:04.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-06T04:06:04.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-06T04:06:04.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-06T04:06:04.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-06T04:06:04.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-06T04:06:04.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-06T04:06:04.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-06T04:06:04.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-06T04:06:04.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-06T04:06:04.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-06T04:06:04.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-06T04:06:04.487 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-06T04:06:04.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-06T04:06:04.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-06T04:06:04.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-06T04:06:04.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-06T04:06:04.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-06T04:06:04.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-06T04:06:04.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-06T04:06:04.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-06T04:06:04.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-06T04:06:04.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-06T04:06:04.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-06T04:06:04.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-06T04:06:04.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-06T04:06:04.488 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-06T04:06:04.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-06T04:06:04.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-06T04:06:04.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-06T04:06:04.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-06T04:06:04.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-06T04:06:04.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-06T04:06:04.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-06T04:06:04.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-06T04:06:04.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-06T04:06:04.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-06T04:06:04.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-06T04:06:04.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-06T04:06:04.489 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-06T04:06:04.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-06T04:06:04.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-06T04:06:04.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-06T04:06:04.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-06T04:06:04.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-06T04:06:04.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-06T04:06:04.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-06T04:06:04.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-06T04:06:04.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-06T04:06:04.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-06T04:06:04.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-06T04:06:04.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-06T04:06:04.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-06T04:06:04.490 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-06T04:06:04.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-06T04:06:04.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-06T04:06:04.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-06T04:06:04.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-06T04:06:04.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-06T04:06:04.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-06T04:06:04.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-06T04:06:04.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-06T04:06:04.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-06T04:06:04.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-06T04:06:04.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-06T04:06:04.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-06T04:06:04.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-06T04:06:04.491 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-06T04:06:04.492 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-06T04:06:04.492 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-06T04:06:04.492 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-06T04:06:04.492 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-06T04:06:04.492 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-06T04:06:04.492 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-06T04:06:04.492 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-06T04:06:04.492 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-06T04:06:04.492 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-06T04:06:04.492 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-06T04:06:04.492 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-06T04:06:04.492 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-06T04:06:04.492 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-06T04:06:04.492 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-06T04:06:04.493 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-06T04:06:04.493 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-06T04:06:04.493 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-06T04:06:04.493 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-06T04:06:04.493 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-06T04:06:04.493 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-06T04:06:04.493 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-06T04:06:04.493 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-06T04:06:04.493 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-06T04:06:04.493 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-06T04:06:04.493 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-06T04:06:04.493 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-06T04:06:04.493 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-06T04:06:04.493 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-06T04:06:04.494 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-06T04:06:04.494 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-06T04:06:04.494 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-06T04:06:04.494 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-06T04:06:04.494 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-06T04:06:04.494 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-06T04:06:04.494 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-06T04:06:04.494 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-06T04:06:04.494 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-06T04:06:04.494 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-06T04:06:04.494 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-06T04:06:04.494 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-06T04:06:04.494 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-06T04:06:04.495 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-06T04:06:04.495 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-06T04:06:04.495 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-06T04:06:04.495 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-06T04:06:04.495 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-06T04:06:04.495 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-06T04:06:04.495 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-06T04:06:04.495 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-06T04:06:04.495 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T04:06:04.495 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-06T04:06:04.495 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-06T04:06:04.495 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-06T04:06:04.495 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-06T04:06:04.495 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-06T04:06:04.495 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-06T04:06:04.496 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T04:06:04.496 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-06T04:06:04.496 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-06T04:06:04.496 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-06T04:06:04.496 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-06T04:06:04.496 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-06T04:06:04.496 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-06T04:06:04.496 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-06T04:06:04.496 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-06T04:06:04.496 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-06T04:06:07.280 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-06T04:06:07.281 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:06:07.281 INFO:teuthology.orchestra.run.smithi154.stdout:Upgraded: 2024-06-06T04:06:07.281 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-06T04:06:07.281 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-06T04:06:07.281 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-06T04:06:07.281 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-06T04:06:07.281 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-06T04:06:07.281 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-06T04:06:07.281 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-06T04:06:07.281 INFO:teuthology.orchestra.run.smithi154.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.282 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.282 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-06T04:06:07.282 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-06T04:06:07.282 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-06-06T04:06:07.282 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-06T04:06:07.282 INFO:teuthology.orchestra.run.smithi154.stdout: automake-1.16.2-8.el9.noarch 2024-06-06T04:06:07.282 INFO:teuthology.orchestra.run.smithi154.stdout: bison-3.7.4-5.el9.x86_64 2024-06-06T04:06:07.282 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-06T04:06:07.282 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.282 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.282 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.282 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.283 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:06:07.283 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.283 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.283 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.283 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:06:07.283 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:06:07.283 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:06:07.283 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:06:07.283 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:06:07.283 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.283 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.283 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:06:07.283 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.283 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.284 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.284 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:06:07.284 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:06:07.284 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.284 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.284 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:06:07.284 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-06T04:06:07.284 INFO:teuthology.orchestra.run.smithi154.stdout: flex-2.6.4-9.el9.x86_64 2024-06-06T04:06:07.284 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-06T04:06:07.284 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-06T04:06:07.284 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-06T04:06:07.284 INFO:teuthology.orchestra.run.smithi154.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-06T04:06:07.284 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-06T04:06:07.284 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-06T04:06:07.285 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-06T04:06:07.285 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-06T04:06:07.285 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-06T04:06:07.285 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.285 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.285 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.285 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-06T04:06:07.285 INFO:teuthology.orchestra.run.smithi154.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-06T04:06:07.285 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-06T04:06:07.285 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-06T04:06:07.285 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-06T04:06:07.285 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.285 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.285 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-06T04:06:07.285 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.286 INFO:teuthology.orchestra.run.smithi154.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-06T04:06:07.286 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-06T04:06:07.286 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-06T04:06:07.286 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-06T04:06:07.286 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-06T04:06:07.286 INFO:teuthology.orchestra.run.smithi154.stdout: lua-5.4.4-4.el9.x86_64 2024-06-06T04:06:07.286 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-06T04:06:07.286 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-06T04:06:07.286 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-06T04:06:07.286 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-06T04:06:07.286 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-06T04:06:07.286 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-06T04:06:07.286 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-13.14-1.el9.x86_64 2024-06-06T04:06:07.287 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-06T04:06:07.287 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-06T04:06:07.287 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-06T04:06:07.287 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-06T04:06:07.287 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-06T04:06:07.287 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-06T04:06:07.287 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-06T04:06:07.287 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-06T04:06:07.287 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-06T04:06:07.287 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-06T04:06:07.287 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.287 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.287 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.287 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-06T04:06:07.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-06T04:06:07.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-06T04:06:07.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-06T04:06:07.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-06T04:06:07.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-06T04:06:07.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-06T04:06:07.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-06T04:06:07.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-06T04:06:07.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-06T04:06:07.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-06T04:06:07.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-06T04:06:07.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-06T04:06:07.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-06T04:06:07.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-06T04:06:07.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-06T04:06:07.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-06T04:06:07.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-06T04:06:07.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-06T04:06:07.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-06T04:06:07.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-06T04:06:07.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-06T04:06:07.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-06T04:06:07.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-06T04:06:07.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-06T04:06:07.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-06T04:06:07.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-06T04:06:07.290 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-06T04:06:07.290 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-06T04:06:07.290 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-06T04:06:07.290 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply-3.11-14.el9.noarch 2024-06-06T04:06:07.290 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-06T04:06:07.290 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-06T04:06:07.290 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-06T04:06:07.290 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-06T04:06:07.290 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-06T04:06:07.290 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-06T04:06:07.291 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-06T04:06:07.291 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-06T04:06:07.291 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-06T04:06:07.291 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.291 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.291 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-06T04:06:07.291 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-06T04:06:07.291 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-06T04:06:07.291 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.291 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-06T04:06:07.291 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-06T04:06:07.292 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-06T04:06:07.292 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-06T04:06:07.292 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-06T04:06:07.292 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-06T04:06:07.292 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-06T04:06:07.292 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-06T04:06:07.292 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-06T04:06:07.293 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-06T04:06:07.293 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-06T04:06:07.293 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-06T04:06:07.293 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-06T04:06:07.293 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-06T04:06:07.293 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-06T04:06:07.293 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.294 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.294 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:06:07.294 INFO:teuthology.orchestra.run.smithi154.stdout: re2-1:20211101-3.el9.x86_64 2024-06-06T04:06:07.294 INFO:teuthology.orchestra.run.smithi154.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-06T04:06:07.294 INFO:teuthology.orchestra.run.smithi154.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-06T04:06:07.294 INFO:teuthology.orchestra.run.smithi154.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-06T04:06:07.295 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-06T04:06:07.295 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-06T04:06:07.295 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-06T04:06:07.295 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:06:07.296 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-06-06T04:06:07.669 DEBUG:teuthology.parallel:result is None 2024-06-06T04:08:41.893 INFO:teuthology.orchestra.run.smithi045.stdout:(159/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 48 kB/s | 16 MB 05:43 2024-06-06T04:08:41.895 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-06-06T04:08:41.895 INFO:teuthology.orchestra.run.smithi045.stdout:Total 442 kB/s | 225 MB 08:41 2024-06-06T04:08:43.099 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-06-06T04:08:43.274 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-06-06T04:08:43.274 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-06-06T04:08:45.531 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-06-06T04:08:45.532 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-06-06T04:08:51.903 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-06-06T04:08:51.980 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-06T04:08:52.018 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-06T04:08:52.090 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-06T04:08:52.130 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-06T04:08:52.155 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-06T04:08:52.202 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-06T04:08:52.552 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-06T04:08:52.697 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T04:08:52.822 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T04:08:52.848 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T04:08:52.924 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T04:08:52.953 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T04:08:52.997 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-06T04:08:53.215 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-06T04:08:53.276 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-06T04:08:53.303 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T04:08:53.409 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T04:08:53.437 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T04:08:53.500 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T04:08:53.522 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T04:08:53.545 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T04:08:53.742 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T04:08:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T04:08:54.077 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T04:08:54.166 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T04:08:54.263 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T04:08:54.341 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T04:08:54.544 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T04:08:54.662 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-06T04:08:54.763 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-06T04:08:54.796 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-06T04:08:54.839 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-06T04:08:54.915 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-06T04:08:54.963 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-06T04:08:55.007 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-06T04:08:55.039 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-06T04:08:55.066 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-06T04:08:55.138 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-06T04:08:55.199 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-06T04:08:55.228 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-06T04:08:55.272 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-06T04:08:55.350 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-06T04:08:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-06T04:08:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-06T04:08:55.471 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-06T04:08:55.546 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-06T04:08:55.595 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-06T04:08:55.673 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-06T04:08:55.811 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-06T04:08:55.865 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-06T04:08:56.437 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-06T04:08:56.521 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-06T04:08:56.554 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-06T04:08:56.596 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-06T04:08:56.649 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-06T04:08:56.687 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-06T04:08:56.729 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-06T04:08:56.882 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-06T04:08:56.931 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-06T04:08:56.987 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-06T04:08:57.062 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-06T04:08:57.098 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-06T04:08:57.135 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-06T04:08:57.183 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-06T04:08:57.217 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-06T04:08:57.261 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-06T04:08:57.303 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-06T04:08:57.446 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-06T04:08:57.513 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-06T04:08:57.554 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-06T04:08:57.652 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-06T04:08:57.687 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-06T04:08:57.738 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-06T04:08:57.769 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-06T04:08:57.794 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-06T04:08:57.888 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-06T04:08:58.019 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-06T04:08:58.051 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-06T04:08:58.079 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-06T04:08:58.130 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-06T04:08:58.170 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-06T04:08:58.210 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-06T04:08:58.239 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-06T04:08:58.274 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-06T04:08:58.307 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-06T04:08:58.349 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-06T04:08:58.383 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-06T04:08:58.473 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-06T04:08:58.522 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-06T04:08:58.553 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-06T04:08:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-06T04:08:58.712 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-06T04:08:58.764 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-06T04:08:59.533 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-06T04:08:59.585 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-06T04:08:59.618 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-06T04:08:59.722 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-06T04:08:59.775 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-06T04:08:59.820 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-06T04:08:59.911 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-06T04:09:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-06T04:09:00.249 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-06T04:09:00.277 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-06T04:09:00.358 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-06T04:09:01.068 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-06T04:09:01.188 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-06T04:09:02.539 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-06T04:09:02.631 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-06T04:09:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-06T04:09:02.692 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-06T04:09:03.020 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-06T04:09:03.062 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T04:09:03.154 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T04:09:03.184 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-06T04:09:03.221 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-06T04:09:03.736 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-06T04:09:03.774 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T04:09:03.845 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T04:09:03.876 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-06T04:09:06.289 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T04:09:06.330 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T04:09:06.907 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T04:09:06.950 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T04:09:07.042 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T04:09:07.042 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-06T04:09:07.042 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-06T04:09:07.043 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:09:07.076 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T04:09:19.276 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T04:09:19.276 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-06-06T04:09:19.276 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-06-06T04:09:19.277 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-06-06T04:09:19.277 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-06-06T04:09:19.277 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-06-06T04:09:19.277 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-06-06T04:09:19.277 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-06-06T04:09:19.277 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:09:19.449 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T04:09:19.465 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T04:09:19.466 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:09:19.466 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-06T04:09:19.466 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:09:20.993 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T04:09:21.021 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T04:09:21.022 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:09:21.022 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-06T04:09:21.022 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T04:09:21.022 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T04:09:21.022 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:09:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T04:09:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T04:09:21.286 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:09:21.286 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-06T04:09:21.286 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T04:09:21.286 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T04:09:21.286 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:09:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T04:09:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T04:09:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:09:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-06T04:09:21.777 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T04:09:21.778 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T04:09:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:09:21.892 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-06T04:09:21.951 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-06T04:09:22.016 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-06T04:09:22.212 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-06T04:09:22.264 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-06T04:09:22.346 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-06T04:09:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-06T04:09:22.598 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-06T04:09:22.648 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-06T04:09:22.680 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-06T04:09:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-06T04:09:23.163 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T04:09:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T04:09:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-06T04:09:23.304 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-06T04:09:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T04:09:24.426 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T04:09:24.456 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T04:09:25.475 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T04:09:25.499 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T04:09:25.609 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T04:09:25.778 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-06T04:09:25.810 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T04:09:25.835 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T04:09:25.835 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:09:25.835 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-06T04:09:25.835 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T04:09:25.835 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T04:09:25.835 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:09:25.885 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T04:09:25.896 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T04:09:25.998 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-06T04:09:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-06T04:09:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-06T04:09:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T04:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T04:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-06T04:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T04:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T04:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:09:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T04:09:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T04:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-06T04:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:09:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T04:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T04:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-06T04:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T04:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T04:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T04:09:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T04:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-06T04:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T04:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T04:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:09:32.364 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-06T04:09:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-06T04:09:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-06T04:09:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-06T04:09:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-06T04:09:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-06T04:09:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-06T04:09:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-06T04:09:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-06T04:09:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-06T04:09:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T04:09:33.529 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T04:09:33.564 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T04:09:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T04:09:34.491 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-06T04:09:34.587 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-06T04:09:34.629 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-06T04:09:34.723 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-06T04:09:34.757 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-06T04:09:34.781 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T04:09:34.807 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-06T04:09:34.833 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-06T04:09:34.833 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T04:09:34.919 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T04:09:34.960 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-06T04:09:34.960 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T04:09:35.044 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T04:09:35.069 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-06T04:09:35.075 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T04:09:35.075 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T04:09:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T04:09:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T04:09:35.331 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T04:09:35.355 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-06T04:09:35.424 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T04:09:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T04:09:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-06T04:09:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-06T04:09:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-06T04:09:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-06T04:09:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-06T04:09:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-06T04:09:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-06T04:09:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T04:09:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T04:09:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T04:09:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-06T04:09:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-06T04:09:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-06T04:09:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T04:09:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-06T04:09:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-06T04:09:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-06T04:09:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-06T04:09:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-06T04:09:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-06T04:09:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-06T04:09:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-06T04:09:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-06T04:09:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-06T04:09:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-06T04:09:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-06T04:09:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-06T04:09:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-06T04:09:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-06T04:09:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-06T04:09:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-06T04:09:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-06T04:09:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-06T04:09:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-06T04:09:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-06T04:09:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-06T04:09:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-06T04:09:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-06T04:09:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-06T04:09:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-06T04:09:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-06T04:09:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-06T04:09:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-06T04:09:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-06T04:09:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-06T04:09:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-06T04:09:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-06T04:09:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-06T04:09:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-06T04:09:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-06T04:09:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-06T04:09:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-06T04:09:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-06T04:09:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-06T04:09:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-06T04:09:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-06T04:09:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-06T04:09:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-06T04:09:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-06T04:09:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-06T04:09:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-06T04:09:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-06T04:09:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-06T04:09:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-06T04:09:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-06T04:09:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-06T04:09:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-06T04:09:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-06T04:09:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-06T04:09:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-06T04:09:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-06T04:09:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-06T04:09:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-06T04:09:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-06T04:09:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-06T04:09:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-06T04:09:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-06T04:09:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-06T04:09:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-06T04:09:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-06T04:09:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-06T04:09:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-06T04:09:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-06T04:09:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-06T04:09:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-06T04:09:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-06T04:09:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-06T04:09:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-06T04:09:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-06T04:09:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-06T04:09:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-06T04:09:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-06T04:09:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-06T04:09:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-06T04:09:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-06T04:09:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-06T04:09:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-06T04:09:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-06T04:09:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-06T04:09:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-06T04:09:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-06T04:09:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-06T04:09:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-06T04:09:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-06T04:09:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-06T04:09:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-06T04:09:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-06T04:09:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-06T04:09:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-06T04:09:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-06T04:09:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-06T04:09:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-06T04:09:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-06T04:09:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-06T04:09:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-06T04:09:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-06T04:09:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-06T04:09:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-06T04:09:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-06T04:09:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-06T04:09:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-06T04:09:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-06T04:09:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-06T04:09:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-06T04:09:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-06T04:09:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-06T04:09:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-06T04:09:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-06T04:09:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-06T04:09:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-06T04:09:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-06T04:09:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-06T04:09:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-06T04:09:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-06T04:09:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-06T04:09:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-06T04:09:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-06T04:09:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-06T04:09:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-06T04:09:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-06T04:09:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-06T04:09:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-06T04:09:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-06T04:09:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-06T04:09:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-06T04:09:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-06T04:09:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-06T04:09:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-06T04:09:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-06T04:09:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-06T04:09:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-06T04:09:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-06T04:09:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T04:09:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-06T04:09:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-06T04:09:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-06T04:09:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-06T04:09:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-06T04:09:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-06T04:09:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T04:09:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-06T04:09:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-06T04:09:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-06T04:09:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-06T04:09:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-06T04:09:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-06T04:09:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-06T04:09:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-06T04:09:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-06T04:09:43.906 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-06T04:09:43.906 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:09:43.906 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-06-06T04:09:43.906 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-06T04:09:43.906 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-06T04:09:43.907 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-06T04:09:43.907 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-06T04:09:43.907 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-06T04:09:43.907 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-06T04:09:43.907 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-06T04:09:43.907 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.907 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.907 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-06T04:09:43.907 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-06T04:09:43.907 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-06-06T04:09:43.907 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-06T04:09:43.907 INFO:teuthology.orchestra.run.smithi045.stdout: automake-1.16.2-8.el9.noarch 2024-06-06T04:09:43.908 INFO:teuthology.orchestra.run.smithi045.stdout: bison-3.7.4-5.el9.x86_64 2024-06-06T04:09:43.908 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-06T04:09:43.908 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.908 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.908 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.908 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.908 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:09:43.908 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.908 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.908 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.908 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:09:43.908 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:09:43.908 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:09:43.908 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:09:43.909 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:09:43.909 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.909 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.909 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:09:43.909 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.909 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.909 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.909 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:09:43.909 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:09:43.909 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.909 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.910 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:09:43.910 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-06T04:09:43.910 INFO:teuthology.orchestra.run.smithi045.stdout: flex-2.6.4-9.el9.x86_64 2024-06-06T04:09:43.910 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-06T04:09:43.910 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-06T04:09:43.910 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-06T04:09:43.910 INFO:teuthology.orchestra.run.smithi045.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-06T04:09:43.910 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-06T04:09:43.910 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-06T04:09:43.910 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-06T04:09:43.910 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-06T04:09:43.910 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-06T04:09:43.910 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.911 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.911 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.911 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-06T04:09:43.911 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-06T04:09:43.911 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-06T04:09:43.911 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-06T04:09:43.911 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-06T04:09:43.911 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.911 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.911 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-06T04:09:43.911 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.911 INFO:teuthology.orchestra.run.smithi045.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-06T04:09:43.911 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-06T04:09:43.911 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-06T04:09:43.912 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-06T04:09:43.912 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-06T04:09:43.912 INFO:teuthology.orchestra.run.smithi045.stdout: lua-5.4.4-4.el9.x86_64 2024-06-06T04:09:43.912 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-06T04:09:43.912 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-06T04:09:43.912 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-06T04:09:43.912 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-06T04:09:43.912 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-06T04:09:43.912 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-06T04:09:43.912 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-13.14-1.el9.x86_64 2024-06-06T04:09:43.912 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-06T04:09:43.912 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-06T04:09:43.912 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-06T04:09:43.912 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-06T04:09:43.913 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-06T04:09:43.913 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-06T04:09:43.913 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-06T04:09:43.913 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-06T04:09:43.913 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-06T04:09:43.913 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-06T04:09:43.913 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.913 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.913 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.913 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-06T04:09:43.913 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-06T04:09:43.913 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-06T04:09:43.913 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-06T04:09:43.914 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-06T04:09:43.914 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-06T04:09:43.914 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-06T04:09:43.914 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-06T04:09:43.914 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-06T04:09:43.914 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-06T04:09:43.914 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-06T04:09:43.914 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-06T04:09:43.914 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-06T04:09:43.914 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-06T04:09:43.914 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-06T04:09:43.914 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-06T04:09:43.915 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-06T04:09:43.915 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-06T04:09:43.915 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-06T04:09:43.915 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-06T04:09:43.915 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-06T04:09:43.915 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-06T04:09:43.915 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-06T04:09:43.915 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-06T04:09:43.915 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-06T04:09:43.915 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-06T04:09:43.915 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-06T04:09:43.915 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-06T04:09:43.915 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-06T04:09:43.915 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-06T04:09:43.916 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-06-06T04:09:43.916 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-06T04:09:43.916 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-06T04:09:43.916 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-06T04:09:43.916 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-06T04:09:43.916 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-06T04:09:43.916 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-06T04:09:43.916 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-06T04:09:43.916 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-06T04:09:43.916 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-06T04:09:43.916 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.916 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.917 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-06T04:09:43.917 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-06T04:09:43.917 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-06T04:09:43.917 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.917 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-06T04:09:43.917 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-06T04:09:43.917 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-06T04:09:43.917 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-06T04:09:43.917 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-06T04:09:43.917 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-06T04:09:43.917 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-06T04:09:43.917 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-06T04:09:43.917 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-06T04:09:43.918 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-06T04:09:43.918 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-06T04:09:43.918 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-06T04:09:43.918 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-06T04:09:43.918 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-06T04:09:43.918 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-06T04:09:43.918 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.918 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.918 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:09:43.918 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-3.el9.x86_64 2024-06-06T04:09:43.918 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-06T04:09:43.918 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-06T04:09:43.919 INFO:teuthology.orchestra.run.smithi045.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-06T04:09:43.919 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-06T04:09:43.919 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-06T04:09:43.919 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-06T04:09:43.919 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:09:43.919 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-06-06T04:09:44.322 DEBUG:teuthology.parallel:result is None 2024-06-06T04:10:02.859 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 19 kB/s | 6.2 MB 05:34 2024-06-06T04:10:40.449 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 467 B/s | 17 kB 00:36 2024-06-06T04:10:51.810 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 1.9 MB/s | 21 MB 00:11 2024-06-06T04:10:56.890 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-06-06T04:10:59.233 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T04:10:59.233 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T04:10:59.244 INFO:teuthology.orchestra.run.smithi086.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-06T04:10:59.245 INFO:teuthology.orchestra.run.smithi086.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-06T04:10:59.246 INFO:teuthology.orchestra.run.smithi086.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-06T04:10:59.246 INFO:teuthology.orchestra.run.smithi086.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-06T04:10:59.248 INFO:teuthology.orchestra.run.smithi086.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-06T04:10:59.250 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-06T04:10:59.251 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-06T04:10:59.314 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-06-06T04:10:59.340 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-06-06T04:10:59.340 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-06-06T04:10:59.341 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-06-06T04:10:59.341 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-06-06T04:10:59.341 INFO:teuthology.orchestra.run.smithi086.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-06T04:10:59.341 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-06T04:10:59.341 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-06T04:10:59.341 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-06T04:10:59.341 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-06T04:10:59.341 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-06T04:10:59.341 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-06T04:10:59.341 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-06T04:10:59.341 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-06T04:10:59.341 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-06T04:10:59.341 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-06T04:10:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-06T04:10:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-06T04:10:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-06T04:10:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-06T04:10:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-06T04:10:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-06T04:10:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-06T04:10:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-06T04:10:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-06T04:10:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-06T04:10:59.342 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-06T04:10:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-06T04:10:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-06T04:10:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-06T04:10:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-06T04:10:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-06T04:10:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-06T04:10:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-06T04:10:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-06T04:10:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-06T04:10:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-06T04:10:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-06T04:10:59.343 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-06T04:10:59.344 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T04:10:59.344 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-06T04:10:59.344 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-06T04:10:59.344 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-06-06T04:10:59.344 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-06T04:10:59.344 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-06T04:10:59.344 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-06T04:10:59.344 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-06T04:10:59.344 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-06T04:10:59.344 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-06T04:10:59.344 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-06T04:10:59.344 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-06T04:10:59.344 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T04:10:59.344 INFO:teuthology.orchestra.run.smithi086.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-06T04:10:59.345 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-06T04:10:59.345 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-06-06T04:10:59.345 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-06T04:10:59.345 INFO:teuthology.orchestra.run.smithi086.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-06T04:10:59.345 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-06T04:10:59.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-06T04:10:59.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-06T04:10:59.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-06T04:10:59.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-06T04:10:59.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-06T04:10:59.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-06T04:10:59.345 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-06T04:10:59.346 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-06T04:10:59.346 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-06T04:10:59.346 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-06T04:10:59.346 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-06T04:10:59.346 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-06T04:10:59.346 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-06T04:10:59.346 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-06T04:10:59.346 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-06T04:10:59.346 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-06T04:10:59.346 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-06T04:10:59.346 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-06T04:10:59.347 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-06T04:10:59.347 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-06T04:10:59.347 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-06T04:10:59.347 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-06T04:10:59.347 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-06T04:10:59.347 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-06T04:10:59.347 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-06T04:10:59.347 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-06T04:10:59.347 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-06T04:10:59.347 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-06T04:10:59.347 INFO:teuthology.orchestra.run.smithi086.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-06T04:10:59.347 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-06T04:10:59.348 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-06T04:10:59.348 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-06T04:10:59.348 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-06T04:10:59.348 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-06T04:10:59.348 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-06T04:10:59.348 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-06T04:10:59.348 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-06T04:10:59.348 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-06T04:10:59.348 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-06T04:10:59.348 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-06T04:10:59.348 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-06T04:10:59.348 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-06T04:10:59.348 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-06T04:10:59.348 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-06T04:10:59.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-06T04:10:59.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-06T04:10:59.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-06T04:10:59.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-06T04:10:59.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-06T04:10:59.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-06T04:10:59.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-06T04:10:59.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-06T04:10:59.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-06T04:10:59.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-06T04:10:59.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-06T04:10:59.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-06T04:10:59.349 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-06T04:10:59.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-06T04:10:59.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-06T04:10:59.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-06T04:10:59.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-06T04:10:59.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-06T04:10:59.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-06T04:10:59.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-06T04:10:59.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-06T04:10:59.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-06T04:10:59.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-06T04:10:59.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-06T04:10:59.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-06T04:10:59.350 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-06T04:10:59.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-06T04:10:59.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-06T04:10:59.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-06T04:10:59.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-06T04:10:59.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-06T04:10:59.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-06T04:10:59.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-06T04:10:59.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-06T04:10:59.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-06T04:10:59.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-06T04:10:59.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-06T04:10:59.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-06T04:10:59.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-06T04:10:59.351 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-06T04:10:59.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-06T04:10:59.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-06T04:10:59.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-06T04:10:59.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-06T04:10:59.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-06T04:10:59.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-06T04:10:59.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-06T04:10:59.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-06T04:10:59.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-06T04:10:59.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-06T04:10:59.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-06T04:10:59.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-06T04:10:59.352 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-06T04:10:59.353 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-06T04:10:59.353 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-06T04:10:59.353 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-06T04:10:59.353 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-06T04:10:59.353 INFO:teuthology.orchestra.run.smithi086.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-06T04:10:59.353 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-06T04:10:59.353 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-06T04:10:59.353 INFO:teuthology.orchestra.run.smithi086.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-06T04:10:59.353 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-06T04:10:59.353 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-06T04:10:59.353 INFO:teuthology.orchestra.run.smithi086.stdout:Installing weak dependencies: 2024-06-06T04:10:59.353 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-06T04:10:59.353 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T04:10:59.353 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-06-06T04:10:59.354 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-06-06T04:10:59.354 INFO:teuthology.orchestra.run.smithi086.stdout:Install 148 Packages 2024-06-06T04:10:59.354 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 11 Packages 2024-06-06T04:10:59.354 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T04:10:59.354 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 225 M 2024-06-06T04:10:59.354 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-06-06T04:11:00.191 INFO:teuthology.orchestra.run.smithi086.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-06-06T04:11:01.266 INFO:teuthology.orchestra.run.smithi086.stdout:(2/159): ceph-common-19.0.0-2946.g3d89accf.el9. 16 MB/s | 21 MB 00:01 2024-06-06T04:11:01.291 INFO:teuthology.orchestra.run.smithi086.stdout:(3/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 809 kB/s | 890 kB 00:01 2024-06-06T04:11:01.308 INFO:teuthology.orchestra.run.smithi086.stdout:(4/159): ceph-immutable-object-cache-19.0.0-294 3.4 MB/s | 145 kB 00:00 2024-06-06T04:11:01.425 INFO:teuthology.orchestra.run.smithi086.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 13 MB/s | 1.5 MB 00:00 2024-06-06T04:11:01.492 INFO:teuthology.orchestra.run.smithi086.stdout:(6/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 3.5 MB/s | 5.4 MB 00:01 2024-06-06T04:11:01.701 INFO:teuthology.orchestra.run.smithi086.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 17 MB/s | 4.7 MB 00:00 2024-06-06T04:11:02.351 INFO:teuthology.orchestra.run.smithi086.stdout:(8/159): ceph-radosgw-19.0.0-2946.g3d89accf.el9 16 MB/s | 10 MB 00:00 2024-06-06T04:11:02.409 INFO:teuthology.orchestra.run.smithi086.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 431 kB/s | 25 kB 00:00 2024-06-06T04:11:02.718 INFO:teuthology.orchestra.run.smithi086.stdout:(10/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x8 1.7 MB/s | 2.4 MB 00:01 2024-06-06T04:11:02.927 INFO:teuthology.orchestra.run.smithi086.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.0 MB/s | 221 kB 00:00 2024-06-06T04:11:03.043 INFO:teuthology.orchestra.run.smithi086.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 232 kB/s | 27 kB 00:00 2024-06-06T04:11:03.211 INFO:teuthology.orchestra.run.smithi086.stdout:(13/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 9.8 MB/s | 17 MB 00:01 2024-06-06T04:11:03.235 INFO:teuthology.orchestra.run.smithi086.stdout:(14/159): libcephfs-devel-19.0.0-2946.g3d89accf 165 kB/s | 32 kB 00:00 2024-06-06T04:11:03.319 INFO:teuthology.orchestra.run.smithi086.stdout:(15/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 6.7 MB/s | 747 kB 00:00 2024-06-06T04:11:03.403 INFO:teuthology.orchestra.run.smithi086.stdout:(16/159): libcephsqlite-19.0.0-2946.g3d89accf.e 975 kB/s | 163 kB 00:00 2024-06-06T04:11:03.429 INFO:teuthology.orchestra.run.smithi086.stdout:(17/159): librados-devel-19.0.0-2946.g3d89accf. 1.1 MB/s | 127 kB 00:00 2024-06-06T04:11:03.737 INFO:teuthology.orchestra.run.smithi086.stdout:(18/159): libradosstriper1-19.0.0-2946.g3d89acc 1.5 MB/s | 501 kB 00:00 2024-06-06T04:11:03.854 INFO:teuthology.orchestra.run.smithi086.stdout:(19/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 13 MB/s | 5.3 MB 00:00 2024-06-06T04:11:03.879 INFO:teuthology.orchestra.run.smithi086.stdout:(20/159): python3-ceph-argparse-19.0.0-2946.g3d 318 kB/s | 45 kB 00:00 2024-06-06T04:11:03.930 INFO:teuthology.orchestra.run.smithi086.stdout:(21/159): python3-ceph-common-19.0.0-2946.g3d89 1.8 MB/s | 135 kB 00:00 2024-06-06T04:11:04.013 INFO:teuthology.orchestra.run.smithi086.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 3.8 MB/s | 321 kB 00:00 2024-06-06T04:11:04.086 INFO:teuthology.orchestra.run.smithi086.stdout:(23/159): python3-cephfs-19.0.0-2946.g3d89accf. 784 kB/s | 162 kB 00:00 2024-06-06T04:11:04.122 INFO:teuthology.orchestra.run.smithi086.stdout:(24/159): python3-rbd-19.0.0-2946.g3d89accf.el9 2.7 MB/s | 300 kB 00:00 2024-06-06T04:11:04.156 INFO:teuthology.orchestra.run.smithi086.stdout:(25/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.4 MB/s | 99 kB 00:00 2024-06-06T04:11:04.214 INFO:teuthology.orchestra.run.smithi086.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 927 kB/s | 85 kB 00:00 2024-06-06T04:11:04.298 INFO:teuthology.orchestra.run.smithi086.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.0 MB/s | 171 kB 00:00 2024-06-06T04:11:04.365 INFO:teuthology.orchestra.run.smithi086.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 475 kB/s | 32 kB 00:00 2024-06-06T04:11:04.440 INFO:teuthology.orchestra.run.smithi086.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.8 MB/s | 142 kB 00:00 2024-06-06T04:11:04.724 INFO:teuthology.orchestra.run.smithi086.stdout:(30/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 12 MB/s | 3.5 MB 00:00 2024-06-06T04:11:05.291 INFO:teuthology.orchestra.run.smithi086.stdout:(31/159): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-06-06T04:11:05.374 INFO:teuthology.orchestra.run.smithi086.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.9 MB/s | 249 kB 00:00 2024-06-06T04:11:05.808 INFO:teuthology.orchestra.run.smithi086.stdout:(33/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 14 MB/s | 48 MB 00:03 2024-06-06T04:11:05.833 INFO:teuthology.orchestra.run.smithi086.stdout:(34/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 107 kB/s | 49 kB 00:00 2024-06-06T04:11:05.858 INFO:teuthology.orchestra.run.smithi086.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2946.g3 329 kB/s | 16 kB 00:00 2024-06-06T04:11:05.900 INFO:teuthology.orchestra.run.smithi086.stdout:(36/159): ceph-volume-19.0.0-2946.g3d89accf.el9 3.9 MB/s | 264 kB 00:00 2024-06-06T04:11:05.951 INFO:teuthology.orchestra.run.smithi086.stdout:(37/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 8.1 MB/s | 763 kB 00:00 2024-06-06T04:11:06.009 INFO:teuthology.orchestra.run.smithi086.stdout:(38/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 1.7 MB/s | 3.1 MB 00:01 2024-06-06T04:11:06.034 INFO:teuthology.orchestra.run.smithi086.stdout:(39/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 136 kB/s | 18 kB 00:00 2024-06-06T04:11:06.122 INFO:teuthology.orchestra.run.smithi086.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:11:06.124 INFO:teuthology.orchestra.run.smithi086.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:11:06.133 INFO:teuthology.orchestra.run.smithi086.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:11:06.193 INFO:teuthology.orchestra.run.smithi086.stdout:(40/159): fuse-2.9.9-15.el9.x86_64.rpm 332 kB/s | 80 kB 00:00 2024-06-06T04:11:06.227 INFO:teuthology.orchestra.run.smithi086.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-06-06T04:11:06.252 INFO:teuthology.orchestra.run.smithi086.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-06-06T04:11:06.294 INFO:teuthology.orchestra.run.smithi086.stdout:(43/159): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-06-06T04:11:06.319 INFO:teuthology.orchestra.run.smithi086.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-06-06T04:11:06.345 INFO:teuthology.orchestra.run.smithi086.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-06-06T04:11:06.370 INFO:teuthology.orchestra.run.smithi086.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-06-06T04:11:06.395 INFO:teuthology.orchestra.run.smithi086.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-06-06T04:11:06.396 INFO:teuthology.orchestra.run.smithi086.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-06T04:11:06.437 INFO:teuthology.orchestra.run.smithi086.stdout:(48/159): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.2 MB 00:00 2024-06-06T04:11:06.438 INFO:teuthology.orchestra.run.smithi086.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:11:06.441 INFO:teuthology.orchestra.run.smithi086.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:11:07.996 INFO:teuthology.orchestra.run.smithi086.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 69 kB/s | 107 kB 00:01 2024-06-06T04:11:08.121 INFO:teuthology.orchestra.run.smithi086.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 174 kB/s | 22 kB 00:00 2024-06-06T04:11:11.254 INFO:teuthology.orchestra.run.smithi086.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 100 kB/s | 313 kB 00:03 2024-06-06T04:11:11.780 INFO:teuthology.orchestra.run.smithi086.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 62 kB/s | 33 kB 00:00 2024-06-06T04:11:13.555 INFO:teuthology.orchestra.run.smithi086.stdout:(53/159): automake-1.16.2-8.el9.noarch.rpm 96 kB/s | 689 kB 00:07 2024-06-06T04:11:13.622 INFO:teuthology.orchestra.run.smithi086.stdout:(54/159): bison-3.7.4-5.el9.x86_64.rpm 130 kB/s | 943 kB 00:07 2024-06-06T04:11:13.689 INFO:teuthology.orchestra.run.smithi086.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 137 kB/s | 18 kB 00:00 2024-06-06T04:11:13.722 INFO:teuthology.orchestra.run.smithi086.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-06-06T04:11:14.398 INFO:teuthology.orchestra.run.smithi086.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 67 kB/s | 45 kB 00:00 2024-06-06T04:11:14.964 INFO:teuthology.orchestra.run.smithi086.stdout:(58/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 126 kB/s | 160 kB 00:01 2024-06-06T04:11:17.114 INFO:teuthology.orchestra.run.smithi086.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 244 kB/s | 662 kB 00:02 2024-06-06T04:11:17.281 INFO:teuthology.orchestra.run.smithi086.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 152 kB/s | 25 kB 00:00 2024-06-06T04:11:17.598 INFO:teuthology.orchestra.run.smithi086.stdout:(61/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 150 kB/s | 47 kB 00:00 2024-06-06T04:11:18.932 INFO:teuthology.orchestra.run.smithi086.stdout:(62/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 219 kB/s | 292 kB 00:01 2024-06-06T04:11:20.540 INFO:teuthology.orchestra.run.smithi086.stdout:(63/159): lua-5.4.4-4.el9.x86_64.rpm 117 kB/s | 188 kB 00:01 2024-06-06T04:11:21.099 INFO:teuthology.orchestra.run.smithi086.stdout:(64/159): openblas-0.3.21-2.el9.x86_64.rpm 63 kB/s | 35 kB 00:00 2024-06-06T04:11:28.324 INFO:teuthology.orchestra.run.smithi086.stdout:(65/159): libtool-2.4.6-45.el9.x86_64.rpm 44 kB/s | 582 kB 00:13 2024-06-06T04:11:42.448 INFO:teuthology.orchestra.run.smithi086.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 227 kB/s | 4.7 MB 00:21 2024-06-06T04:11:45.997 INFO:teuthology.orchestra.run.smithi086.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 458 kB/s | 1.6 MB 00:03 2024-06-06T04:11:46.297 INFO:teuthology.orchestra.run.smithi086.stdout:[MIRROR] postgresql-contrib-13.14-1.el9.x86_64.rpm: Status code: 500 for http://mirror.servaxnet.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.14-1.el9.x86_64.rpm (IP: 162.255.226.75) 2024-06-06T04:11:46.343 INFO:teuthology.orchestra.run.smithi086.stdout:[MIRROR] postgresql-contrib-13.14-1.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.14-1.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:11:50.272 INFO:teuthology.orchestra.run.smithi086.stdout:(68/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 206 kB/s | 4.4 MB 00:21 2024-06-06T04:11:51.255 INFO:teuthology.orchestra.run.smithi086.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 135 kB/s | 133 kB 00:00 2024-06-06T04:11:55.797 INFO:teuthology.orchestra.run.smithi086.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 93 kB/s | 911 kB 00:09 2024-06-06T04:12:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 52 kB/s | 1.0 MB 00:19 2024-06-06T04:12:16.812 INFO:teuthology.orchestra.run.smithi086.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 77 kB/s | 87 kB 00:01 2024-06-06T04:13:17.523 INFO:teuthology.orchestra.run.smithi086.stdout:(73/159): postgresql-server-13.14-1.el9.x86_64. 69 kB/s | 5.8 MB 01:26 2024-06-06T04:13:17.655 INFO:teuthology.orchestra.run.smithi086.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 460 kB/s | 61 kB 00:00 2024-06-06T04:13:18.331 INFO:teuthology.orchestra.run.smithi086.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 362 kB/s | 244 kB 00:00 2024-06-06T04:13:20.614 INFO:teuthology.orchestra.run.smithi086.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 109 kB/s | 248 kB 00:02 2024-06-06T04:13:21.447 INFO:teuthology.orchestra.run.smithi086.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 31 kB/s | 26 kB 00:00 2024-06-06T04:13:21.707 INFO:teuthology.orchestra.run.smithi086.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 74 kB/s | 19 kB 00:00 2024-06-06T04:13:25.872 INFO:teuthology.orchestra.run.smithi086.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 41 kB/s | 172 kB 00:04 2024-06-06T04:13:26.139 INFO:teuthology.orchestra.run.smithi086.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 130 kB/s | 35 kB 00:00 2024-06-06T04:13:51.454 INFO:teuthology.orchestra.run.smithi086.stdout:(81/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 206 kB/s | 5.1 MB 00:25 2024-06-06T04:13:54.104 INFO:teuthology.orchestra.run.smithi086.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 95 kB/s | 250 kB 00:02 2024-06-06T04:13:55.453 INFO:teuthology.orchestra.run.smithi086.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 165 kB/s | 222 kB 00:01 2024-06-06T04:13:56.954 INFO:teuthology.orchestra.run.smithi086.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 51 kB/s | 77 kB 00:01 2024-06-06T04:13:57.421 INFO:teuthology.orchestra.run.smithi086.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 90 kB/s | 42 kB 00:00 2024-06-06T04:13:59.021 INFO:teuthology.orchestra.run.smithi086.stdout:(86/159): python3-protobuf-3.14.0-13.el9.noarch 168 kB/s | 269 kB 00:01 2024-06-06T04:14:00.804 INFO:teuthology.orchestra.run.smithi086.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 89 kB/s | 159 kB 00:01 2024-06-06T04:14:02.346 INFO:teuthology.orchestra.run.smithi086.stdout:(88/159): python3-babel-2.9.1-2.el9.noarch.rpm 58 kB/s | 6.0 MB 01:45 2024-06-06T04:14:02.829 INFO:teuthology.orchestra.run.smithi086.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 105 kB/s | 51 kB 00:00 2024-06-06T04:14:04.221 INFO:teuthology.orchestra.run.smithi086.stdout:(90/159): python3-requests-oauthlib-1.3.0-12.el 39 kB/s | 54 kB 00:01 2024-06-06T04:14:04.260 INFO:teuthology.orchestra.run.smithi086.stdout:[MIRROR] python3-scipy-1.6.2-8.el9.x86_64.rpm: Status code: 500 for http://mirror.servaxnet.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-scipy-1.6.2-8.el9.x86_64.rpm (IP: 162.255.226.75) 2024-06-06T04:14:15.046 INFO:teuthology.orchestra.run.smithi086.stdout:(91/159): python3-pyasn1-modules-0.4.8-6.el9.no 20 kB/s | 279 kB 00:14 2024-06-06T04:14:17.862 INFO:teuthology.orchestra.run.smithi086.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 15 kB/s | 42 kB 00:02 2024-06-06T04:14:20.462 INFO:teuthology.orchestra.run.smithi086.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 17 kB/s | 44 kB 00:02 2024-06-06T04:14:26.679 INFO:teuthology.orchestra.run.smithi086.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 35 kB/s | 221 kB 00:06 2024-06-06T04:14:27.529 INFO:teuthology.orchestra.run.smithi086.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 42 kB/s | 36 kB 00:00 2024-06-06T04:14:28.329 INFO:teuthology.orchestra.run.smithi086.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 59 kB/s | 47 kB 00:00 2024-06-06T04:14:30.754 INFO:teuthology.orchestra.run.smithi086.stdout:(97/159): socat-1.7.4.1-5.el9.x86_64.rpm 126 kB/s | 305 kB 00:02 2024-06-06T04:14:31.337 INFO:teuthology.orchestra.run.smithi086.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 99 kB/s | 58 kB 00:00 2024-06-06T04:14:31.380 INFO:teuthology.orchestra.run.smithi086.stdout:[MIRROR] xfsprogs-devel-6.3.0-1.el9.x86_64.rpm: Status code: 500 for http://mirror.servaxnet.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/xfsprogs-devel-6.3.0-1.el9.x86_64.rpm (IP: 162.255.226.75) 2024-06-06T04:14:42.437 INFO:teuthology.orchestra.run.smithi086.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 8.4 kB/s | 94 kB 00:11 2024-06-06T04:15:39.414 INFO:teuthology.orchestra.run.smithi086.stdout:(100/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 kB/s | 64 kB 00:56 2024-06-06T04:15:47.113 INFO:teuthology.orchestra.run.smithi086.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 5.8 kB/s | 45 kB 00:07 2024-06-06T04:15:47.160 INFO:teuthology.orchestra.run.smithi086.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:15:47.530 INFO:teuthology.orchestra.run.smithi086.stdout:(102/159): lua-devel-5.4.4-4.el9.x86_64.rpm 53 kB/s | 22 kB 00:00 2024-06-06T04:15:47.533 INFO:teuthology.orchestra.run.smithi086.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T04:15:52.788 INFO:teuthology.orchestra.run.smithi086.stdout:(103/159): protobuf-compiler-3.14.0-13.el9.x86_ 164 kB/s | 863 kB 00:05 2024-06-06T04:15:53.296 INFO:teuthology.orchestra.run.smithi086.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-06-06T04:15:53.613 INFO:teuthology.orchestra.run.smithi086.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 975 kB/s | 309 kB 00:00 2024-06-06T04:15:53.647 INFO:teuthology.orchestra.run.smithi086.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 586 kB/s | 19 kB 00:00 2024-06-06T04:15:57.688 INFO:teuthology.orchestra.run.smithi086.stdout:(107/159): libarrow-9.0.0-11.el9.x86_64.rpm 1.1 MB/s | 4.4 MB 00:04 2024-06-06T04:15:57.739 INFO:teuthology.orchestra.run.smithi086.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-06-06T04:15:57.805 INFO:teuthology.orchestra.run.smithi086.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 730 kB/s | 49 kB 00:00 2024-06-06T04:15:57.897 INFO:teuthology.orchestra.run.smithi086.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 736 kB/s | 67 kB 00:00 2024-06-06T04:15:58.056 INFO:teuthology.orchestra.run.smithi086.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 955 kB/s | 151 kB 00:00 2024-06-06T04:15:58.806 INFO:teuthology.orchestra.run.smithi086.stdout:(112/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.1 MB/s | 839 kB 00:00 2024-06-06T04:15:59.281 INFO:teuthology.orchestra.run.smithi086.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 1.1 MB/s | 548 kB 00:00 2024-06-06T04:15:59.340 INFO:teuthology.orchestra.run.smithi086.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 747 kB/s | 43 kB 00:00 2024-06-06T04:15:59.390 INFO:teuthology.orchestra.run.smithi086.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 644 kB/s | 32 kB 00:00 2024-06-06T04:15:59.415 INFO:teuthology.orchestra.run.smithi086.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 600 kB/s | 15 kB 00:00 2024-06-06T04:15:59.583 INFO:teuthology.orchestra.run.smithi086.stdout:(117/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-06-06T04:15:59.883 INFO:teuthology.orchestra.run.smithi086.stdout:(118/159): python3-cherrypy-18.6.1-2.el9.noarch 1.2 MB/s | 358 kB 00:00 2024-06-06T04:16:00.133 INFO:teuthology.orchestra.run.smithi086.stdout:(119/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.2 MB/s | 303 kB 00:00 2024-06-06T04:16:00.317 INFO:teuthology.orchestra.run.smithi086.stdout:(120/159): python3-google-auth-2.29.0-1.el9.noa 1.1 MB/s | 206 kB 00:00 2024-06-06T04:16:02.075 INFO:teuthology.orchestra.run.smithi086.stdout:(121/159): python3-grpcio-1.46.7-10.el9.x86_64. 1.2 MB/s | 2.0 MB 00:01 2024-06-06T04:16:02.225 INFO:teuthology.orchestra.run.smithi086.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 965 kB/s | 144 kB 00:00 2024-06-06T04:16:02.259 INFO:teuthology.orchestra.run.smithi086.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 320 kB/s | 11 kB 00:00 2024-06-06T04:16:02.301 INFO:teuthology.orchestra.run.smithi086.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 423 kB/s | 18 kB 00:00 2024-06-06T04:16:02.343 INFO:teuthology.orchestra.run.smithi086.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 559 kB/s | 23 kB 00:00 2024-06-06T04:16:02.385 INFO:teuthology.orchestra.run.smithi086.stdout:(126/159): python3-jaraco-functools-3.5.0-2.el9 465 kB/s | 19 kB 00:00 2024-06-06T04:16:02.427 INFO:teuthology.orchestra.run.smithi086.stdout:(127/159): python3-jaraco-text-3.2.0-6.el9.noar 473 kB/s | 20 kB 00:00 2024-06-06T04:16:03.477 INFO:teuthology.orchestra.run.smithi086.stdout:(128/159): python3-kubernetes-26.1.0-2.el9.noar 996 kB/s | 1.0 MB 00:01 2024-06-06T04:16:03.552 INFO:teuthology.orchestra.run.smithi086.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 618 kB/s | 46 kB 00:00 2024-06-06T04:16:03.661 INFO:teuthology.orchestra.run.smithi086.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 726 kB/s | 79 kB 00:00 2024-06-06T04:16:03.753 INFO:teuthology.orchestra.run.smithi086.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 631 kB/s | 58 kB 00:00 2024-06-06T04:16:04.069 INFO:teuthology.orchestra.run.smithi086.stdout:(132/159): python3-pecan-1.4.2-3.el9.noarch.rpm 861 kB/s | 272 kB 00:00 2024-06-06T04:16:04.111 INFO:teuthology.orchestra.run.smithi086.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 395 kB/s | 16 kB 00:00 2024-06-06T04:16:04.237 INFO:teuthology.orchestra.run.smithi086.stdout:(134/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 719 kB/s | 90 kB 00:00 2024-06-06T04:16:04.287 INFO:teuthology.orchestra.run.smithi086.stdout:(135/159): python3-pyperclip-1.8.2-2.el9.noarch 566 kB/s | 28 kB 00:00 2024-06-06T04:16:04.345 INFO:teuthology.orchestra.run.smithi086.stdout:(136/159): python3-repoze-lru-0.7-16.el9.noarch 528 kB/s | 31 kB 00:00 2024-06-06T04:16:04.554 INFO:teuthology.orchestra.run.smithi086.stdout:(137/159): python3-routes-2.5.1-5.el9.noarch.rp 905 kB/s | 188 kB 00:00 2024-06-06T04:16:04.638 INFO:teuthology.orchestra.run.smithi086.stdout:(138/159): python3-rsa-4.9-2.el9.noarch.rpm 708 kB/s | 59 kB 00:00 2024-06-06T04:16:04.705 INFO:teuthology.orchestra.run.smithi086.stdout:(139/159): python3-tempora-5.0.0-2.el9.noarch.r 537 kB/s | 36 kB 00:00 2024-06-06T04:16:04.788 INFO:teuthology.orchestra.run.smithi086.stdout:(140/159): python3-typing-extensions-4.4.0-2.el 612 kB/s | 51 kB 00:00 2024-06-06T04:16:05.038 INFO:teuthology.orchestra.run.smithi086.stdout:(141/159): python3-webob-1.8.7-6.el9.noarch.rpm 919 kB/s | 230 kB 00:00 2024-06-06T04:16:05.163 INFO:teuthology.orchestra.run.smithi086.stdout:(142/159): python3-websocket-client-1.2.3-2.el9 718 kB/s | 90 kB 00:00 2024-06-06T04:16:05.605 INFO:teuthology.orchestra.run.smithi086.stdout:(143/159): python3-werkzeug-2.0.3-3.el9.1.noarc 967 kB/s | 427 kB 00:00 2024-06-06T04:16:05.647 INFO:teuthology.orchestra.run.smithi086.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 480 kB/s | 20 kB 00:00 2024-06-06T04:16:06.081 INFO:teuthology.orchestra.run.smithi086.stdout:(145/159): re2-20211101-3.el9.x86_64.rpm 442 kB/s | 192 kB 00:00 2024-06-06T04:16:07.764 INFO:teuthology.orchestra.run.smithi086.stdout:(146/159): thrift-0.15.0-2.el9.x86_64.rpm 962 kB/s | 1.6 MB 00:01 2024-06-06T04:16:08.148 INFO:teuthology.orchestra.run.smithi086.stdout:(147/159): librados2-19.0.0-2946.g3d89accf.el9. 8.8 MB/s | 3.4 MB 00:00 2024-06-06T04:16:08.315 INFO:teuthology.orchestra.run.smithi086.stdout:(148/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 19 MB/s | 3.1 MB 00:00 2024-06-06T04:16:08.507 INFO:teuthology.orchestra.run.smithi086.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-06-06T04:16:08.566 INFO:teuthology.orchestra.run.smithi086.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 31 MB/s | 1.8 MB 00:00 2024-06-06T04:16:08.607 INFO:teuthology.orchestra.run.smithi086.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-06-06T04:16:08.649 INFO:teuthology.orchestra.run.smithi086.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-06-06T04:16:08.675 INFO:teuthology.orchestra.run.smithi086.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 1.4 MB/s | 37 kB 00:00 2024-06-06T04:16:08.717 INFO:teuthology.orchestra.run.smithi086.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.9 MB/s | 207 kB 00:00 2024-06-06T04:16:08.759 INFO:teuthology.orchestra.run.smithi086.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 6.1 MB/s | 260 kB 00:00 2024-06-06T04:16:08.817 INFO:teuthology.orchestra.run.smithi086.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 23 MB/s | 1.3 MB 00:00 2024-06-06T04:16:08.892 INFO:teuthology.orchestra.run.smithi086.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-06-06T04:16:33.291 INFO:teuthology.orchestra.run.smithi086.stdout:[MIRROR] flexiblas-netlib-3.0.4-8.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://mirror.servaxnet.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/flexiblas-netlib-3.0.4-8.el9.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds] 2024-06-06T04:16:42.864 INFO:teuthology.orchestra.run.smithi086.stdout:(158/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 105 kB/s | 16 MB 02:38 2024-06-06T04:17:01.979 INFO:teuthology.orchestra.run.smithi086.stdout:(159/159): flexiblas-netlib-3.0.4-8.el9.x86_64. 8.7 kB/s | 3.0 MB 05:50 2024-06-06T04:17:01.982 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-06-06T04:17:01.982 INFO:teuthology.orchestra.run.smithi086.stdout:Total 636 kB/s | 225 MB 06:02 2024-06-06T04:17:03.148 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-06-06T04:17:03.307 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-06-06T04:17:03.307 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-06-06T04:17:05.477 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-06-06T04:17:05.478 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-06-06T04:17:11.673 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-06-06T04:17:11.758 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-06T04:17:11.803 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-06T04:17:11.872 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-06T04:17:11.908 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-06T04:17:11.934 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-06T04:17:11.987 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-06T04:17:12.323 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-06T04:17:12.502 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T04:17:12.623 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T04:17:12.661 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T04:17:12.736 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T04:17:12.766 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T04:17:12.808 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-06T04:17:12.996 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-06T04:17:13.055 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-06T04:17:13.083 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T04:17:13.187 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T04:17:13.209 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T04:17:13.272 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T04:17:13.293 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T04:17:13.314 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T04:17:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T04:17:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T04:17:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T04:17:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T04:17:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T04:17:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T04:17:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T04:17:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-06T04:17:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-06T04:17:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-06T04:17:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-06T04:17:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-06T04:17:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-06T04:17:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-06T04:17:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-06T04:17:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-06T04:17:14.864 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-06T04:17:14.917 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-06T04:17:14.956 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-06T04:17:14.991 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-06T04:17:15.068 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-06T04:17:15.102 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-06T04:17:15.138 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-06T04:17:15.166 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-06T04:17:15.237 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-06T04:17:15.289 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-06T04:17:15.364 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-06T04:17:15.499 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-06T04:17:15.551 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-06T04:17:16.080 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-06T04:17:16.171 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-06T04:17:16.200 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-06T04:17:16.237 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-06T04:17:16.285 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-06T04:17:16.323 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-06T04:17:16.365 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-06T04:17:16.516 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-06T04:17:16.568 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-06T04:17:16.623 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-06T04:17:16.695 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-06T04:17:16.726 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-06T04:17:16.763 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-06T04:17:16.809 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-06T04:17:16.845 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-06T04:17:16.880 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-06T04:17:16.922 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-06T04:17:17.058 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-06T04:17:17.110 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-06T04:17:17.151 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-06T04:17:17.246 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-06T04:17:17.276 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-06T04:17:17.326 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-06T04:17:17.358 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-06T04:17:17.383 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-06T04:17:17.472 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-06T04:17:17.601 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-06T04:17:17.640 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-06T04:17:17.668 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-06T04:17:17.710 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-06T04:17:17.749 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-06T04:17:17.781 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-06T04:17:17.811 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-06T04:17:17.845 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-06T04:17:17.879 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-06T04:17:17.920 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-06T04:17:17.955 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-06T04:17:18.042 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-06T04:17:18.094 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-06T04:17:18.125 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-06T04:17:18.172 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-06T04:17:18.270 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-06T04:17:18.318 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-06T04:17:19.072 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-06T04:17:19.115 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-06T04:17:19.149 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-06T04:17:19.250 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-06T04:17:19.304 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-06T04:17:19.349 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-06T04:17:19.438 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-06T04:17:19.477 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-06T04:17:19.653 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-06T04:17:19.682 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-06T04:17:19.761 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-06T04:17:20.417 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-06T04:17:20.522 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-06T04:17:21.791 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-06T04:17:21.878 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-06T04:17:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-06T04:17:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-06T04:17:22.300 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-06T04:17:22.343 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T04:17:22.417 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T04:17:22.456 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-06T04:17:22.497 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-06T04:17:22.963 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-06T04:17:23.193 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T04:17:23.351 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T04:17:23.381 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-06T04:17:25.652 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T04:17:25.695 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T04:17:26.236 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T04:17:26.273 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T04:17:26.366 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T04:17:26.366 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-06T04:17:26.366 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-06T04:17:26.366 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T04:17:26.399 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T04:17:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T04:17:38.107 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2024-06-06T04:17:38.107 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2024-06-06T04:17:38.107 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2024-06-06T04:17:38.107 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2024-06-06T04:17:38.107 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2024-06-06T04:17:38.107 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /root 2024-06-06T04:17:38.107 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2024-06-06T04:17:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T04:17:38.288 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T04:17:38.305 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T04:17:38.305 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:17:38.305 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-06T04:17:38.306 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T04:17:39.814 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T04:17:39.840 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T04:17:39.840 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:17:39.840 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-06T04:17:39.840 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T04:17:39.840 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T04:17:39.840 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T04:17:40.065 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T04:17:40.087 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T04:17:40.087 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:17:40.087 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-06T04:17:40.087 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T04:17:40.087 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T04:17:40.088 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T04:17:40.533 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T04:17:40.554 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T04:17:40.554 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:17:40.555 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-06T04:17:40.555 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T04:17:40.555 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T04:17:40.555 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T04:17:40.653 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-06T04:17:40.702 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-06T04:17:40.766 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-06T04:17:40.951 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-06T04:17:40.999 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-06T04:17:41.069 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-06T04:17:41.165 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-06T04:17:41.305 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-06T04:17:41.349 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-06T04:17:41.389 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-06T04:17:41.425 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-06T04:17:41.828 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T04:17:41.857 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T04:17:41.888 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-06T04:17:41.950 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-06T04:17:41.988 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T04:17:43.019 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T04:17:43.056 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T04:17:44.020 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T04:17:44.048 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T04:17:44.149 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T04:17:44.318 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-06T04:17:44.342 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T04:17:44.365 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T04:17:44.365 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:17:44.365 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-06T04:17:44.365 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T04:17:44.365 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T04:17:44.365 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T04:17:44.418 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T04:17:44.429 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T04:17:44.526 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-06T04:17:44.608 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-06T04:17:45.509 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-06T04:17:45.545 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T04:17:45.568 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T04:17:45.568 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:17:45.568 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-06T04:17:45.568 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T04:17:45.568 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T04:17:45.568 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T04:17:45.612 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T04:17:45.635 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T04:17:45.635 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:17:45.635 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-06T04:17:45.635 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T04:17:45.680 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T04:17:45.702 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T04:17:45.702 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:17:45.702 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-06T04:17:45.702 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T04:17:45.702 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T04:17:45.702 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T04:17:46.013 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T04:17:46.036 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T04:17:46.036 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T04:17:46.036 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-06T04:17:46.036 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T04:17:46.036 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T04:17:46.036 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T04:17:50.541 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-06T04:17:50.583 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-06T04:17:50.632 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-06T04:17:50.672 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-06T04:17:50.700 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-06T04:17:50.897 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-06T04:17:50.937 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-06T04:17:50.970 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-06T04:17:51.010 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-06T04:17:51.040 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-06T04:17:51.078 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T04:17:51.672 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T04:17:51.725 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T04:17:51.745 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T04:17:52.575 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-06T04:17:52.663 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-06T04:17:52.704 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-06T04:17:52.803 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-06T04:17:52.840 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-06T04:17:52.865 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T04:17:52.891 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-06T04:17:52.917 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-06T04:17:52.917 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T04:17:53.002 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T04:17:53.034 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-06T04:17:53.035 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T04:17:53.119 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T04:17:53.144 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-06T04:17:53.155 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T04:17:53.155 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T04:17:53.354 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T04:17:53.354 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T04:17:53.430 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T04:17:53.455 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-06T04:17:53.520 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T04:17:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T04:17:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-06T04:17:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-06T04:17:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-06T04:17:58.943 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-06T04:17:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-06T04:17:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-06T04:17:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-06T04:17:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T04:17:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T04:17:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T04:17:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-06T04:17:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-06T04:17:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-06T04:17:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T04:17:58.944 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-06T04:17:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-06T04:17:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-06T04:17:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-06T04:17:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-06T04:17:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-06T04:17:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-06T04:17:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-06T04:17:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-06T04:17:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-06T04:17:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-06T04:17:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-06T04:17:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-06T04:17:58.945 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-06T04:17:58.946 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-06T04:17:58.946 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-06T04:17:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-06T04:17:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-06T04:17:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-06T04:17:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-06T04:17:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-06T04:17:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-06T04:17:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-06T04:17:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-06T04:17:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-06T04:17:58.958 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-06T04:17:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-06T04:17:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-06T04:17:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-06T04:17:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-06T04:17:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-06T04:17:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-06T04:17:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-06T04:17:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-06T04:17:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-06T04:17:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-06T04:17:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-06T04:17:58.959 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-06T04:17:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-06T04:17:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-06T04:17:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-06T04:17:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-06T04:17:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-06T04:17:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-06T04:17:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-06T04:17:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-06T04:17:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-06T04:17:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-06T04:17:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-06T04:17:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-06T04:17:58.960 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-06T04:17:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-06T04:17:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-06T04:17:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-06T04:17:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-06T04:17:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-06T04:17:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-06T04:17:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-06T04:17:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-06T04:17:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-06T04:17:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-06T04:17:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-06T04:17:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-06T04:17:58.961 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-06T04:17:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-06T04:17:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-06T04:17:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-06T04:17:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-06T04:17:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-06T04:17:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-06T04:17:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-06T04:17:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-06T04:17:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-06T04:17:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-06T04:17:58.962 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-06T04:17:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-06T04:17:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-06T04:17:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-06T04:17:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-06T04:17:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-06T04:17:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-06T04:17:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-06T04:17:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-06T04:17:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-06T04:17:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-06T04:17:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-06T04:17:58.963 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-06T04:17:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-06T04:17:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-06T04:17:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-06T04:17:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-06T04:17:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-06T04:17:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-06T04:17:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-06T04:17:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-06T04:17:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-06T04:17:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-06T04:17:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-06T04:17:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-06T04:17:58.964 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-06T04:17:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-06T04:17:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-06T04:17:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-06T04:17:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-06T04:17:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-06T04:17:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-06T04:17:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-06T04:17:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-06T04:17:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-06T04:17:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-06T04:17:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-06T04:17:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-06T04:17:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-06T04:17:58.965 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-06T04:17:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-06T04:17:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-06T04:17:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-06T04:17:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-06T04:17:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-06T04:17:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-06T04:17:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-06T04:17:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-06T04:17:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-06T04:17:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-06T04:17:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-06T04:17:58.966 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-06T04:17:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-06T04:17:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-06T04:17:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-06T04:17:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-06T04:17:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-06T04:17:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-06T04:17:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-06T04:17:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-06T04:17:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-06T04:17:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-06T04:17:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-06T04:17:58.967 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-06T04:17:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T04:17:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-06T04:17:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-06T04:17:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-06T04:17:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-06T04:17:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-06T04:17:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-06T04:17:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T04:17:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-06T04:17:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-06T04:17:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-06T04:17:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-06T04:17:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-06T04:17:58.968 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-06T04:17:58.969 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-06T04:17:58.969 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-06T04:17:58.969 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-06T04:18:02.040 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-06T04:18:02.040 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T04:18:02.040 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-06-06T04:18:02.040 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-06T04:18:02.040 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-06T04:18:02.040 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-06T04:18:02.041 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-06T04:18:02.041 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-06T04:18:02.041 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-06T04:18:02.041 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-06T04:18:02.041 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.041 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.041 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-06T04:18:02.041 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-06T04:18:02.041 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-06-06T04:18:02.041 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-06T04:18:02.041 INFO:teuthology.orchestra.run.smithi086.stdout: automake-1.16.2-8.el9.noarch 2024-06-06T04:18:02.041 INFO:teuthology.orchestra.run.smithi086.stdout: bison-3.7.4-5.el9.x86_64 2024-06-06T04:18:02.041 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-06T04:18:02.042 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.042 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.042 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.042 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.042 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:18:02.042 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.042 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.042 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.042 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:18:02.042 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:18:02.042 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:18:02.043 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:18:02.043 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:18:02.043 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.043 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.043 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:18:02.043 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.043 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.043 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.043 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:18:02.043 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:18:02.043 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.043 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.044 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T04:18:02.044 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-06T04:18:02.044 INFO:teuthology.orchestra.run.smithi086.stdout: flex-2.6.4-9.el9.x86_64 2024-06-06T04:18:02.044 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-06T04:18:02.044 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-06T04:18:02.044 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-06T04:18:02.044 INFO:teuthology.orchestra.run.smithi086.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-06T04:18:02.044 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-06T04:18:02.044 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-06T04:18:02.044 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-06T04:18:02.044 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-06T04:18:02.045 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-06T04:18:02.045 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.045 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.045 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.045 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-06T04:18:02.045 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-06T04:18:02.045 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-06T04:18:02.045 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-06T04:18:02.045 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-06T04:18:02.045 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.045 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.045 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-06T04:18:02.046 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.046 INFO:teuthology.orchestra.run.smithi086.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-06T04:18:02.046 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-06T04:18:02.046 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-06T04:18:02.046 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-06T04:18:02.046 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-06T04:18:02.046 INFO:teuthology.orchestra.run.smithi086.stdout: lua-5.4.4-4.el9.x86_64 2024-06-06T04:18:02.046 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-06T04:18:02.046 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-06T04:18:02.046 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-06T04:18:02.046 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-06T04:18:02.046 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-06T04:18:02.046 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-06T04:18:02.047 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-13.14-1.el9.x86_64 2024-06-06T04:18:02.047 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-06T04:18:02.047 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-06T04:18:02.047 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-06T04:18:02.047 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-06T04:18:02.047 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-06T04:18:02.047 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-06T04:18:02.047 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-06T04:18:02.047 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-06T04:18:02.047 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-06T04:18:02.047 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-06T04:18:02.047 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.047 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.048 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.048 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-06T04:18:02.048 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-06T04:18:02.048 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-06T04:18:02.048 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-06T04:18:02.048 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-06T04:18:02.048 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-06T04:18:02.048 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-06T04:18:02.048 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-06T04:18:02.048 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-06T04:18:02.048 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-06T04:18:02.048 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-06T04:18:02.048 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-06T04:18:02.049 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-06T04:18:02.049 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-06T04:18:02.049 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-06T04:18:02.049 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-06T04:18:02.049 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-06T04:18:02.049 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-06T04:18:02.049 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-06T04:18:02.049 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-06T04:18:02.049 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-06T04:18:02.049 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-06T04:18:02.049 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-06T04:18:02.049 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-06T04:18:02.049 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-06T04:18:02.049 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-06T04:18:02.050 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-06T04:18:02.050 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-06T04:18:02.050 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-06T04:18:02.050 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-06T04:18:02.050 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply-3.11-14.el9.noarch 2024-06-06T04:18:02.050 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-06T04:18:02.050 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-06T04:18:02.050 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-06T04:18:02.050 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-06T04:18:02.050 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-06T04:18:02.050 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-06T04:18:02.050 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-06T04:18:02.050 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-06T04:18:02.051 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-06T04:18:02.051 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.051 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.051 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-06T04:18:02.051 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-06T04:18:02.051 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-06T04:18:02.051 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.051 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-06T04:18:02.051 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-06T04:18:02.051 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-06T04:18:02.051 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-06T04:18:02.051 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-06T04:18:02.051 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-06T04:18:02.052 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-06T04:18:02.052 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-06T04:18:02.052 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-06T04:18:02.052 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-06T04:18:02.052 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-06T04:18:02.052 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-06T04:18:02.052 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-06T04:18:02.052 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-06T04:18:02.052 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-06T04:18:02.052 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.052 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.052 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T04:18:02.052 INFO:teuthology.orchestra.run.smithi086.stdout: re2-1:20211101-3.el9.x86_64 2024-06-06T04:18:02.052 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-06T04:18:02.053 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-06T04:18:02.053 INFO:teuthology.orchestra.run.smithi086.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-06T04:18:02.053 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-06T04:18:02.053 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-06T04:18:02.053 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-06T04:18:02.053 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-06-06T04:18:02.053 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-06-06T04:18:02.476 DEBUG:teuthology.parallel:result is None 2024-06-06T04:18:02.477 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T04:18:02.618 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-06T04:18:02.647 INFO:teuthology.orchestra.run.smithi045.stdout:19.0.0-2946.g3d89accf.el9 2024-06-06T04:18:02.648 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-06T04:18:02.648 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-06T04:18:02.650 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T04:18:02.865 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-06T04:18:02.890 INFO:teuthology.orchestra.run.smithi086.stdout:19.0.0-2946.g3d89accf.el9 2024-06-06T04:18:02.890 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-06T04:18:02.890 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-06T04:18:02.892 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T04:18:03.116 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-06T04:18:03.143 INFO:teuthology.orchestra.run.smithi154.stdout:19.0.0-2946.g3d89accf.el9 2024-06-06T04:18:03.144 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-06T04:18:03.144 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-06T04:18:03.145 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-06T04:18:03.146 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:18:03.146 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-06T04:18:03.180 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-06-06T04:18:03.180 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-06T04:18:03.213 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-06-06T04:18:03.213 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-06T04:18:03.246 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-06T04:18:03.246 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:18:03.246 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-06-06T04:18:03.276 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-06T04:18:03.345 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-06-06T04:18:03.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-06-06T04:18:03.375 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-06T04:18:03.445 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-06-06T04:18:03.445 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2024-06-06T04:18:03.475 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-06T04:18:03.544 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-06T04:18:03.544 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:18:03.544 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-06T04:18:03.572 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-06T04:18:03.638 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-06-06T04:18:03.638 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-06T04:18:03.667 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-06T04:18:03.735 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-06-06T04:18:03.735 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-06T04:18:03.764 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-06T04:18:03.833 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-06T04:18:03.834 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:18:03.834 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-06-06T04:18:03.862 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-06T04:18:03.927 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-06-06T04:18:03.927 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-06-06T04:18:03.957 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-06T04:18:04.026 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-06-06T04:18:04.027 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2024-06-06T04:18:04.056 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-06T04:18:04.126 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-06T04:18:04.212 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-06-06T04:18:04.213 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T04:18:04.213 INFO:tasks.cephadm:Cluster fsid is c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:18:04.213 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-06T04:18:04.213 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.45', 'mon.b': '172.21.15.86', 'mon.c': '172.21.15.154'} 2024-06-06T04:18:04.214 INFO:tasks.cephadm:First mon is mon.a on smithi045 2024-06-06T04:18:04.214 INFO:tasks.cephadm:First mgr is x 2024-06-06T04:18:04.214 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-06T04:18:04.214 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname $(hostname -s) 2024-06-06T04:18:04.240 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname $(hostname -s) 2024-06-06T04:18:04.269 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname $(hostname -s) 2024-06-06T04:18:04.299 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-06T04:18:04.299 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T04:18:04.517 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2946-g3d89accf', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2946.g3d89accf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-30 17:04:17.570799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-06T04:18:04.737 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 3d89accfe3201286f000e065a1faea2a88c7a6ba from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T04:18:04.739 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm 2024-06-06T04:18:04.739 DEBUG:teuthology.orchestra.run.smithi045:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-06T04:18:05.178 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 6 04:18 /home/ubuntu/cephtest/cephadm 2024-06-06T04:18:05.179 DEBUG:teuthology.orchestra.run.smithi086:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-06T04:18:05.753 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 6 04:18 /home/ubuntu/cephtest/cephadm 2024-06-06T04:18:05.753 DEBUG:teuthology.orchestra.run.smithi154:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-06T04:18:06.263 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 6 04:18 /home/ubuntu/cephtest/cephadm 2024-06-06T04:18:06.263 DEBUG:teuthology.orchestra.run.smithi045:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-06T04:18:06.284 DEBUG:teuthology.orchestra.run.smithi086:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-06T04:18:06.303 DEBUG:teuthology.orchestra.run.smithi154:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-06T04:18:06.361 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on all hosts... 2024-06-06T04:18:06.361 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-06T04:18:06.364 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-06T04:18:06.366 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-06T04:18:07.177 INFO:teuthology.orchestra.run.smithi045.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T04:18:07.205 INFO:teuthology.orchestra.run.smithi154.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T04:18:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T04:18:39.061 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-06-06T04:18:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T04:18:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-06T04:18:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: "repo_digests": [ 2024-06-06T04:18:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-06T04:18:39.062 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-06-06T04:18:39.062 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-06-06T04:18:50.299 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-06-06T04:18:50.299 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T04:18:50.299 INFO:teuthology.orchestra.run.smithi154.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-06T04:18:50.299 INFO:teuthology.orchestra.run.smithi154.stdout: "repo_digests": [ 2024-06-06T04:18:50.299 INFO:teuthology.orchestra.run.smithi154.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-06T04:18:50.299 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-06-06T04:18:50.299 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-06-06T04:18:53.347 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:18:53.347 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T04:18:53.347 INFO:teuthology.orchestra.run.smithi045.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-06T04:18:53.347 INFO:teuthology.orchestra.run.smithi045.stdout: "repo_digests": [ 2024-06-06T04:18:53.347 INFO:teuthology.orchestra.run.smithi045.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-06T04:18:53.347 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-06-06T04:18:53.348 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:18:53.381 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph 2024-06-06T04:18:53.413 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph 2024-06-06T04:18:53.446 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph 2024-06-06T04:18:53.479 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /etc/ceph 2024-06-06T04:18:53.507 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /etc/ceph 2024-06-06T04:18:53.536 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /etc/ceph 2024-06-06T04:18:53.564 INFO:tasks.cephadm:Writing seed config... 2024-06-06T04:18:53.566 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-06T04:18:53.566 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-06T04:18:53.566 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-06T04:18:53.566 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-06T04:18:53.566 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-06T04:18:53.566 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-06T04:18:53.566 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-06T04:18:53.566 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-06T04:18:53.566 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-06-06T04:18:53.566 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-06-06T04:18:53.567 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-06-06T04:18:53.567 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-06-06T04:18:53.567 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-06T04:18:53.567 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-06T04:18:53.567 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-06T04:18:53.567 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-06T04:18:53.567 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-06-06T04:18:53.567 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-06T04:18:53.567 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-06T04:18:53.567 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-06T04:18:53.567 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-06-06T04:18:53.567 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-06T04:18:53.567 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-06T04:18:53.567 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-06T04:18:53.568 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-06T04:18:53.568 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-06T04:18:53.568 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-06T04:18:53.568 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-06T04:18:53.568 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-06T04:18:53.568 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-06T04:18:53.568 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-06-06T04:18:53.568 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-06T04:18:53.568 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-06-06T04:18:53.568 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-06-06T04:18:53.568 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-06T04:18:53.568 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-06T04:18:53.568 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-06-06T04:18:53.568 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-06T04:18:53.569 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-06T04:18:53.569 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-06T04:18:53.569 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-06T04:18:53.569 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-06T04:18:53.569 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-06T04:18:53.569 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-06T04:18:53.569 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-06-06T04:18:53.570 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:18:53.570 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-06T04:18:53.588 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c4f6b938-23bb-11ef-bc9b-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-06T04:18:53.588 DEBUG:teuthology.orchestra.run.smithi045:mon.a> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mon.a.service 2024-06-06T04:18:53.631 DEBUG:teuthology.orchestra.run.smithi045:mgr.x> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mgr.x.service 2024-06-06T04:18:53.673 INFO:tasks.cephadm:Bootstrapping... 2024-06-06T04:18:53.673 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba -v bootstrap --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.45 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-06T04:18:53.875 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-06-06T04:18:53.875 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba', '-v', 'bootstrap', '--fsid', 'c4f6b938-23bb-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.45', '--skip-admin-label'] 2024-06-06T04:18:53.875 INFO:teuthology.orchestra.run.smithi045.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-06T04:18:53.876 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying podman|docker is present... 2024-06-06T04:18:53.901 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 5.0.2 2024-06-06T04:18:53.901 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying lvm2 is present... 2024-06-06T04:18:53.901 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying time synchronization is in place... 2024-06-06T04:18:53.909 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-06T04:18:53.909 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-06T04:18:53.915 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-06T04:18:53.915 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-06-06T04:18:53.922 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-06-06T04:18:53.929 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-06-06T04:18:53.929 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-06-06T04:18:53.929 INFO:teuthology.orchestra.run.smithi045.stdout:Repeating the final host check... 2024-06-06T04:18:53.952 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 5.0.2 2024-06-06T04:18:53.952 INFO:teuthology.orchestra.run.smithi045.stdout:podman (/bin/podman) version 5.0.2 is present 2024-06-06T04:18:53.952 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl is present 2024-06-06T04:18:53.952 INFO:teuthology.orchestra.run.smithi045.stdout:lvcreate is present 2024-06-06T04:18:53.959 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-06T04:18:53.959 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-06T04:18:53.966 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-06T04:18:53.966 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-06-06T04:18:53.973 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-06-06T04:18:53.979 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-06-06T04:18:53.980 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-06-06T04:18:53.980 INFO:teuthology.orchestra.run.smithi045.stdout:Host looks OK 2024-06-06T04:18:53.980 INFO:teuthology.orchestra.run.smithi045.stdout:Cluster fsid: c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:18:53.980 INFO:teuthology.orchestra.run.smithi045.stdout:Acquiring lock 139744156775808 on /run/cephadm/c4f6b938-23bb-11ef-bc9b-c7b262605968.lock 2024-06-06T04:18:53.980 INFO:teuthology.orchestra.run.smithi045.stdout:Lock 139744156775808 acquired on /run/cephadm/c4f6b938-23bb-11ef-bc9b-c7b262605968.lock 2024-06-06T04:18:53.980 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 3300 ... 2024-06-06T04:18:53.980 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 6789 ... 2024-06-06T04:18:53.980 INFO:teuthology.orchestra.run.smithi045.stdout:Base mon IP(s) is [172.21.15.45:3300, 172.21.15.45:6789], mon addrv is [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-06-06T04:18:53.984 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.45 metric 100 2024-06-06T04:18:53.984 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.45 metric 100 2024-06-06T04:18:53.986 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-06T04:18:53.986 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-06-06T04:18:53.986 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-06-06T04:18:53.989 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-06T04:18:53.989 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-06T04:18:53.989 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-06T04:18:53.989 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-06-06T04:18:53.989 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-06-06T04:18:53.989 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-06T04:18:53.990 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-06-06T04:18:53.990 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-06-06T04:18:53.990 INFO:teuthology.orchestra.run.smithi045.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-06T04:18:53.990 INFO:teuthology.orchestra.run.smithi045.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-06T04:18:53.993 INFO:teuthology.orchestra.run.smithi045.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T04:18:54.533 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-06T04:18:54.533 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T04:18:54.533 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Getting image source signatures 2024-06-06T04:18:54.533 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:53b2d4191269e9d20972bfa652565296fad6514cf6a3d99f19db644875f67876 2024-06-06T04:18:54.533 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:3ca969c7b6d79946c6f10490fffc00028babe7df67f50ed53c7bc209325e0612 2024-06-06T04:18:54.534 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying config sha256:b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-06T04:18:54.534 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-06T04:18:55.388 INFO:teuthology.orchestra.run.smithi045.stdout:ceph: stdout ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T04:18:55.388 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph version: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T04:18:55.388 INFO:teuthology.orchestra.run.smithi045.stdout:Extracting ceph user uid/gid from container image... 2024-06-06T04:18:56.270 INFO:teuthology.orchestra.run.smithi045.stdout:stat: stdout 167 167 2024-06-06T04:18:56.270 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial keys... 2024-06-06T04:18:57.201 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQCwOGFmSJmwJBAA8fQRD1coma7pjoGjHgQlKg== 2024-06-06T04:18:58.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQCxOGFmu4ObIBAATRGWBfGLeXlnYQdwYykIuw== 2024-06-06T04:18:59.164 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQCyOGFmZF3rFxAABe+2VloChki+W0Ebeh9Pyg== 2024-06-06T04:18:59.165 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial monmap... 2024-06-06T04:19:00.095 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-06T04:19:00.095 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-06T04:19:00.095 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:19:00.095 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-06T04:19:00.095 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool for a [v2:172.21.15.45:3300,v1:172.21.15.45:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-06T04:19:00.095 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = quincy 2024-06-06T04:19:00.096 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: set fsid to c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:19:00.096 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-06T04:19:00.096 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:19:00.096 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mon... 2024-06-06T04:19:01.060 INFO:teuthology.orchestra.run.smithi045.stdout:create mon.a on 2024-06-06T04:19:01.268 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-06T04:19:01.429 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-06T04:19:01.690 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968.target. 2024-06-06T04:19:01.690 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968.target. 2024-06-06T04:19:01.943 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mon.a 2024-06-06T04:19:01.944 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-06-06T04:19:02.119 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968.target.wants/ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@.service. 2024-06-06T04:19:02.679 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T04:19:02.679 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-06-06T04:19:02.687 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T04:19:02.687 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-06-06T04:19:02.688 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-06-06T04:19:02.688 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-06-06T04:19:02.688 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon to start... 2024-06-06T04:19:02.688 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon... 2024-06-06T04:19:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:03 smithi045 ceph-mon[31931]: from='client.? 172.21.15.45:0/3905872947' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-06T04:19:03.760 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout cluster: 2024-06-06T04:19:03.760 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout id: c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-06T04:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-06-06T04:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout services: 2024-06-06T04:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.548414s) 2024-06-06T04:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-06T04:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-06T04:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-06-06T04:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout data: 2024-06-06T04:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-06T04:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-06T04:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-06T04:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pgs: 2024-06-06T04:19:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-06-06T04:19:03.762 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.092+0000 7f746219f700 1 Processor -- start 2024-06-06T04:19:03.762 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.093+0000 7f746219f700 1 -- start start 2024-06-06T04:19:03.762 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.093+0000 7f746219f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f745c107b80 0x7f745c107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:03.762 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.093+0000 7f746219f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f745c108570 con 0x7f745c107b80 2024-06-06T04:19:03.762 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.093+0000 7f746119d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f745c107b80 0x7f745c107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:03.762 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.093+0000 7f746119d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f745c107b80 0x7f745c107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56780/0 (socket says 172.21.15.45:56780) 2024-06-06T04:19:03.762 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.093+0000 7f746119d700 1 -- 172.21.15.45:0/2823800100 learned_addr learned my addr 172.21.15.45:0/2823800100 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:03.762 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.093+0000 7f746119d700 1 -- 172.21.15.45:0/2823800100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c108d80 con 0x7f745c107b80 2024-06-06T04:19:03.762 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.094+0000 7f746119d700 1 --2- 172.21.15.45:0/2823800100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f745c107b80 0x7f745c107fa0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f74580098a0 tx=0x7f7458009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00d0a57b0131947 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:03.762 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.094+0000 7f7453fff700 1 -- 172.21.15.45:0/2823800100 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7458004160 con 0x7f745c107b80 2024-06-06T04:19:03.762 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.094+0000 7f7453fff700 1 -- 172.21.15.45:0/2823800100 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7458004300 con 0x7f745c107b80 2024-06-06T04:19:03.762 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.094+0000 7f7453fff700 1 -- 172.21.15.45:0/2823800100 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74580036a0 con 0x7f745c107b80 2024-06-06T04:19:03.763 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.094+0000 7f746219f700 1 -- 172.21.15.45:0/2823800100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f745c107b80 msgr2=0x7f745c107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:03.763 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.094+0000 7f746219f700 1 --2- 172.21.15.45:0/2823800100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f745c107b80 0x7f745c107fa0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f74580098a0 tx=0x7f7458009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:03.763 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.094+0000 7f746219f700 1 -- 172.21.15.45:0/2823800100 shutdown_connections 2024-06-06T04:19:03.763 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.095+0000 7f746219f700 1 --2- 172.21.15.45:0/2823800100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f745c107b80 0x7f745c107fa0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:03.763 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.095+0000 7f746219f700 1 -- 172.21.15.45:0/2823800100 >> 172.21.15.45:0/2823800100 conn(0x7f745c103180 msgr2=0x7f745c1055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:03.763 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.095+0000 7f746219f700 1 -- 172.21.15.45:0/2823800100 shutdown_connections 2024-06-06T04:19:03.763 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.095+0000 7f746219f700 1 -- 172.21.15.45:0/2823800100 wait complete. 2024-06-06T04:19:03.763 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.095+0000 7f746219f700 1 Processor -- start 2024-06-06T04:19:03.763 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.095+0000 7f746219f700 1 -- start start 2024-06-06T04:19:03.763 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.096+0000 7f746219f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f745c107b80 0x7f745c07db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:03.763 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.096+0000 7f746219f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1092b0 con 0x7f745c107b80 2024-06-06T04:19:03.763 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.096+0000 7f746119d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f745c107b80 0x7f745c07db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:03.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.096+0000 7f746119d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f745c107b80 0x7f745c07db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56794/0 (socket says 172.21.15.45:56794) 2024-06-06T04:19:03.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.096+0000 7f746119d700 1 -- 172.21.15.45:0/3905872947 learned_addr learned my addr 172.21.15.45:0/3905872947 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:03.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.096+0000 7f746119d700 1 -- 172.21.15.45:0/3905872947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c07e0a0 con 0x7f745c107b80 2024-06-06T04:19:03.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.096+0000 7f746119d700 1 --2- 172.21.15.45:0/3905872947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f745c107b80 0x7f745c07db60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7458000c00 tx=0x7f7458004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:03.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.096+0000 7f74527fc700 1 -- 172.21.15.45:0/3905872947 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7458003db0 con 0x7f745c107b80 2024-06-06T04:19:03.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.096+0000 7f74527fc700 1 -- 172.21.15.45:0/3905872947 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7458011630 con 0x7f745c107b80 2024-06-06T04:19:03.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.096+0000 7f746219f700 1 -- 172.21.15.45:0/3905872947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c07c290 con 0x7f745c107b80 2024-06-06T04:19:03.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.096+0000 7f74527fc700 1 -- 172.21.15.45:0/3905872947 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7458011950 con 0x7f745c107b80 2024-06-06T04:19:03.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.096+0000 7f746219f700 1 -- 172.21.15.45:0/3905872947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c07c7d0 con 0x7f745c107b80 2024-06-06T04:19:03.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.097+0000 7f74527fc700 1 -- 172.21.15.45:0/3905872947 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7458011af0 con 0x7f745c107b80 2024-06-06T04:19:03.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.097+0000 7f74527fc700 1 -- 172.21.15.45:0/3905872947 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f745801c450 con 0x7f745c107b80 2024-06-06T04:19:03.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.097+0000 7f746219f700 1 -- 172.21.15.45:0/3905872947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f745c063af0 con 0x7f745c107b80 2024-06-06T04:19:03.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.100+0000 7f74527fc700 1 -- 172.21.15.45:0/3905872947 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7458032020 con 0x7f745c107b80 2024-06-06T04:19:03.765 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.173+0000 7f746219f700 1 -- 172.21.15.45:0/3905872947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f745c194af0 con 0x7f745c107b80 2024-06-06T04:19:03.765 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.173+0000 7f74527fc700 1 -- 172.21.15.45:0/3905872947 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f7458004260 con 0x7f745c107b80 2024-06-06T04:19:03.765 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.174+0000 7f746219f700 1 -- 172.21.15.45:0/3905872947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f745c107b80 msgr2=0x7f745c07db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:03.765 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.174+0000 7f746219f700 1 --2- 172.21.15.45:0/3905872947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f745c107b80 0x7f745c07db60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7458000c00 tx=0x7f7458004750 comp rx=0 tx=0).stop 2024-06-06T04:19:03.765 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.175+0000 7f746219f700 1 -- 172.21.15.45:0/3905872947 shutdown_connections 2024-06-06T04:19:03.765 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.175+0000 7f746219f700 1 --2- 172.21.15.45:0/3905872947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f745c107b80 0x7f745c07db60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:03.765 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.175+0000 7f746219f700 1 -- 172.21.15.45:0/3905872947 >> 172.21.15.45:0/3905872947 conn(0x7f745c103180 msgr2=0x7f745c191f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:03.765 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.175+0000 7f746219f700 1 -- 172.21.15.45:0/3905872947 shutdown_connections 2024-06-06T04:19:03.765 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:03.175+0000 7f746219f700 1 -- 172.21.15.45:0/3905872947 wait complete. 2024-06-06T04:19:03.765 INFO:teuthology.orchestra.run.smithi045.stdout:mon is available 2024-06-06T04:19:03.765 INFO:teuthology.orchestra.run.smithi045.stdout:Assimilating anything we can from ceph.conf... 2024-06-06T04:19:04.835 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-06-06T04:19:04.835 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-06-06T04:19:04.835 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:19:04.836 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-06-06T04:19:04.836 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-06T04:19:04.836 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-06T04:19:04.836 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-06T04:19:04.836 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-06T04:19:04.836 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-06-06T04:19:04.836 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-06T04:19:04.836 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-06T04:19:04.836 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-06-06T04:19:04.836 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-06-06T04:19:04.836 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-06T04:19:04.836 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-06T04:19:04.836 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-06T04:19:04.837 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-06T04:19:04.837 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-06T04:19:04.837 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.174+0000 7fe2fa774700 1 Processor -- start 2024-06-06T04:19:04.837 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.175+0000 7fe2fa774700 1 -- start start 2024-06-06T04:19:04.837 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.175+0000 7fe2fa774700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2f4108790 0x7fe2f410ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:04.837 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.175+0000 7fe2fa774700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f407bea0 con 0x7fe2f4108790 2024-06-06T04:19:04.837 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.175+0000 7fe2f3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2f4108790 0x7fe2f410ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:04.837 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.175+0000 7fe2f3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2f4108790 0x7fe2f410ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56796/0 (socket says 172.21.15.45:56796) 2024-06-06T04:19:04.837 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.175+0000 7fe2f3fff700 1 -- 172.21.15.45:0/976929769 learned_addr learned my addr 172.21.15.45:0/976929769 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:04.837 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.175+0000 7fe2f3fff700 1 -- 172.21.15.45:0/976929769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f410b0c0 con 0x7fe2f4108790 2024-06-06T04:19:04.837 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.175+0000 7fe2f3fff700 1 --2- 172.21.15.45:0/976929769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2f4108790 0x7fe2f410ab80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40098a0 tx=0x7fe2e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02fabfdd2d0c8d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:04.837 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.176+0000 7fe2f2ffd700 1 -- 172.21.15.45:0/976929769 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2e4004160 con 0x7fe2f4108790 2024-06-06T04:19:04.837 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.176+0000 7fe2f2ffd700 1 -- 172.21.15.45:0/976929769 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e4004300 con 0x7fe2f4108790 2024-06-06T04:19:04.838 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.176+0000 7fe2f2ffd700 1 -- 172.21.15.45:0/976929769 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2e40036a0 con 0x7fe2f4108790 2024-06-06T04:19:04.838 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.176+0000 7fe2fa774700 1 -- 172.21.15.45:0/976929769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2f4108790 msgr2=0x7fe2f410ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:04.838 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.176+0000 7fe2fa774700 1 --2- 172.21.15.45:0/976929769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2f4108790 0x7fe2f410ab80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40098a0 tx=0x7fe2e4009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:04.838 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.176+0000 7fe2fa774700 1 -- 172.21.15.45:0/976929769 shutdown_connections 2024-06-06T04:19:04.838 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.176+0000 7fe2fa774700 1 --2- 172.21.15.45:0/976929769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2f4108790 0x7fe2f410ab80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:04.838 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.176+0000 7fe2fa774700 1 -- 172.21.15.45:0/976929769 >> 172.21.15.45:0/976929769 conn(0x7fe2f41021e0 msgr2=0x7fe2f4104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:04.838 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.176+0000 7fe2fa774700 1 -- 172.21.15.45:0/976929769 shutdown_connections 2024-06-06T04:19:04.838 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.176+0000 7fe2fa774700 1 -- 172.21.15.45:0/976929769 wait complete. 2024-06-06T04:19:04.838 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.177+0000 7fe2fa774700 1 Processor -- start 2024-06-06T04:19:04.838 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.177+0000 7fe2fa774700 1 -- start start 2024-06-06T04:19:04.838 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.177+0000 7fe2fa774700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2f4108790 0x7fe2f419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:04.838 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.177+0000 7fe2fa774700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f410b590 con 0x7fe2f4108790 2024-06-06T04:19:04.838 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.177+0000 7fe2f3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2f4108790 0x7fe2f419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:04.839 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.177+0000 7fe2f3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2f4108790 0x7fe2f419b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56812/0 (socket says 172.21.15.45:56812) 2024-06-06T04:19:04.839 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.177+0000 7fe2f3fff700 1 -- 172.21.15.45:0/1241089059 learned_addr learned my addr 172.21.15.45:0/1241089059 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:04.839 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.178+0000 7fe2f3fff700 1 -- 172.21.15.45:0/1241089059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f419b960 con 0x7fe2f4108790 2024-06-06T04:19:04.839 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.178+0000 7fe2f3fff700 1 --2- 172.21.15.45:0/1241089059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2f4108790 0x7fe2f419b420 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4000c00 tx=0x7fe2e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:04.839 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.178+0000 7fe2f17fa700 1 -- 172.21.15.45:0/1241089059 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2e4003db0 con 0x7fe2f4108790 2024-06-06T04:19:04.839 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.178+0000 7fe2f17fa700 1 -- 172.21.15.45:0/1241089059 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e4011630 con 0x7fe2f4108790 2024-06-06T04:19:04.839 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.178+0000 7fe2fa774700 1 -- 172.21.15.45:0/1241089059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f419bbf0 con 0x7fe2f4108790 2024-06-06T04:19:04.839 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.178+0000 7fe2f17fa700 1 -- 172.21.15.45:0/1241089059 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2e4011950 con 0x7fe2f4108790 2024-06-06T04:19:04.839 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.178+0000 7fe2fa774700 1 -- 172.21.15.45:0/1241089059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f419e8e0 con 0x7fe2f4108790 2024-06-06T04:19:04.839 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.178+0000 7fe2f17fa700 1 -- 172.21.15.45:0/1241089059 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe2e4011af0 con 0x7fe2f4108790 2024-06-06T04:19:04.839 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.178+0000 7fe2f17fa700 1 -- 172.21.15.45:0/1241089059 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe2e401c450 con 0x7fe2f4108790 2024-06-06T04:19:04.839 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.179+0000 7fe2fa774700 1 -- 172.21.15.45:0/1241089059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f4063af0 con 0x7fe2f4108790 2024-06-06T04:19:04.839 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.181+0000 7fe2f17fa700 1 -- 172.21.15.45:0/1241089059 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe2e4004260 con 0x7fe2f4108790 2024-06-06T04:19:04.839 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.251+0000 7fe2fa774700 1 -- 172.21.15.45:0/1241089059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe2f402df10 con 0x7fe2f4108790 2024-06-06T04:19:04.840 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.254+0000 7fe2f17fa700 1 -- 172.21.15.45:0/1241089059 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fe2e401b450 con 0x7fe2f4108790 2024-06-06T04:19:04.840 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.255+0000 7fe2f17fa700 1 -- 172.21.15.45:0/1241089059 <== mon.0 v2:172.21.15.45:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe2e401b630 con 0x7fe2f4108790 2024-06-06T04:19:04.840 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.256+0000 7fe2fa774700 1 -- 172.21.15.45:0/1241089059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2f4108790 msgr2=0x7fe2f419b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:04.840 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.256+0000 7fe2fa774700 1 --2- 172.21.15.45:0/1241089059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2f4108790 0x7fe2f419b420 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4000c00 tx=0x7fe2e4004750 comp rx=0 tx=0).stop 2024-06-06T04:19:04.840 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.256+0000 7fe2fa774700 1 -- 172.21.15.45:0/1241089059 shutdown_connections 2024-06-06T04:19:04.840 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.256+0000 7fe2fa774700 1 --2- 172.21.15.45:0/1241089059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2f4108790 0x7fe2f419b420 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:04.840 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.256+0000 7fe2fa774700 1 -- 172.21.15.45:0/1241089059 >> 172.21.15.45:0/1241089059 conn(0x7fe2f41021e0 msgr2=0x7fe2f4104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:04.840 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.256+0000 7fe2fa774700 1 -- 172.21.15.45:0/1241089059 shutdown_connections 2024-06-06T04:19:04.840 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:04.256+0000 7fe2fa774700 1 -- 172.21.15.45:0/1241089059 wait complete. 2024-06-06T04:19:04.840 INFO:teuthology.orchestra.run.smithi045.stdout:Generating new minimal ceph.conf... 2024-06-06T04:19:04.927 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:04 smithi045 ceph-mon[31931]: from='client.? 172.21.15.45:0/1241089059' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-06T04:19:04.927 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:04 smithi045 ceph-mon[31931]: from='client.? 172.21.15.45:0/1241089059' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-06T04:19:05.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:05 smithi045 ceph-mon[31931]: from='client.? 172.21.15.45:0/3794125936' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:19:05.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.244+0000 7f403351d700 1 Processor -- start 2024-06-06T04:19:05.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.244+0000 7f403351d700 1 -- start start 2024-06-06T04:19:05.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.244+0000 7f403351d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f402c1088a0 0x7f402c10ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:05.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.244+0000 7f403351d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f402c07be20 con 0x7f402c1088a0 2024-06-06T04:19:05.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.244+0000 7f40312b9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f402c1088a0 0x7f402c10ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:05.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.245+0000 7f40312b9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f402c1088a0 0x7f402c10ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56828/0 (socket says 172.21.15.45:56828) 2024-06-06T04:19:05.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.245+0000 7f40312b9700 1 -- 172.21.15.45:0/1766662510 learned_addr learned my addr 172.21.15.45:0/1766662510 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:05.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.245+0000 7f40312b9700 1 -- 172.21.15.45:0/1766662510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c10b1d0 con 0x7f402c1088a0 2024-06-06T04:19:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.245+0000 7f40312b9700 1 --2- 172.21.15.45:0/1766662510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f402c1088a0 0x7f402c10ac90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f40280098a0 tx=0x7f4028009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0728a40fddf758 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.245+0000 7f4023fff700 1 -- 172.21.15.45:0/1766662510 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4028004160 con 0x7f402c1088a0 2024-06-06T04:19:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.245+0000 7f4023fff700 1 -- 172.21.15.45:0/1766662510 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4028004300 con 0x7f402c1088a0 2024-06-06T04:19:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.245+0000 7f4023fff700 1 -- 172.21.15.45:0/1766662510 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4028011a80 con 0x7f402c1088a0 2024-06-06T04:19:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.246+0000 7f403351d700 1 -- 172.21.15.45:0/1766662510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f402c1088a0 msgr2=0x7f402c10ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.246+0000 7f403351d700 1 --2- 172.21.15.45:0/1766662510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f402c1088a0 0x7f402c10ac90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f40280098a0 tx=0x7f4028009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.246+0000 7f403351d700 1 -- 172.21.15.45:0/1766662510 shutdown_connections 2024-06-06T04:19:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.246+0000 7f403351d700 1 --2- 172.21.15.45:0/1766662510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f402c1088a0 0x7f402c10ac90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.246+0000 7f403351d700 1 -- 172.21.15.45:0/1766662510 >> 172.21.15.45:0/1766662510 conn(0x7f402c102290 msgr2=0x7f402c1046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.246+0000 7f403351d700 1 -- 172.21.15.45:0/1766662510 shutdown_connections 2024-06-06T04:19:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.246+0000 7f403351d700 1 -- 172.21.15.45:0/1766662510 wait complete. 2024-06-06T04:19:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.246+0000 7f403351d700 1 Processor -- start 2024-06-06T04:19:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.247+0000 7f403351d700 1 -- start start 2024-06-06T04:19:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.247+0000 7f403351d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f402c1088a0 0x7f402c108120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:05.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.247+0000 7f403351d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f402c10c130 con 0x7f402c1088a0 2024-06-06T04:19:05.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.247+0000 7f40312b9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f402c1088a0 0x7f402c108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:05.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.247+0000 7f40312b9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f402c1088a0 0x7f402c108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56832/0 (socket says 172.21.15.45:56832) 2024-06-06T04:19:05.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.247+0000 7f40312b9700 1 -- 172.21.15.45:0/3794125936 learned_addr learned my addr 172.21.15.45:0/3794125936 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:05.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.247+0000 7f40312b9700 1 -- 172.21.15.45:0/3794125936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c108660 con 0x7f402c1088a0 2024-06-06T04:19:05.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.247+0000 7f40312b9700 1 --2- 172.21.15.45:0/3794125936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f402c1088a0 0x7f402c108120 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4028011a50 tx=0x7f4028011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:05.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.247+0000 7f40227fc700 1 -- 172.21.15.45:0/3794125936 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4028003a80 con 0x7f402c1088a0 2024-06-06T04:19:05.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.247+0000 7f40227fc700 1 -- 172.21.15.45:0/3794125936 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4028003c20 con 0x7f402c1088a0 2024-06-06T04:19:05.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.247+0000 7f403351d700 1 -- 172.21.15.45:0/3794125936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402c106850 con 0x7f402c1088a0 2024-06-06T04:19:05.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.247+0000 7f40227fc700 1 -- 172.21.15.45:0/3794125936 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f402801a650 con 0x7f402c1088a0 2024-06-06T04:19:05.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.248+0000 7f403351d700 1 -- 172.21.15.45:0/3794125936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402c106d90 con 0x7f402c1088a0 2024-06-06T04:19:05.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.248+0000 7f40227fc700 1 -- 172.21.15.45:0/3794125936 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f402801acb0 con 0x7f402c1088a0 2024-06-06T04:19:05.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.248+0000 7f40227fc700 1 -- 172.21.15.45:0/3794125936 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4028019700 con 0x7f402c1088a0 2024-06-06T04:19:05.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.248+0000 7f403351d700 1 -- 172.21.15.45:0/3794125936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f402c063af0 con 0x7f402c1088a0 2024-06-06T04:19:05.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.251+0000 7f40227fc700 1 -- 172.21.15.45:0/3794125936 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4028024ad0 con 0x7f402c1088a0 2024-06-06T04:19:05.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.320+0000 7f403351d700 1 -- 172.21.15.45:0/3794125936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f402c1070d0 con 0x7f402c1088a0 2024-06-06T04:19:05.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.321+0000 7f40227fc700 1 -- 172.21.15.45:0/3794125936 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f4028024440 con 0x7f402c1088a0 2024-06-06T04:19:05.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.321+0000 7f403351d700 1 -- 172.21.15.45:0/3794125936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f402c1088a0 msgr2=0x7f402c108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:05.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.321+0000 7f403351d700 1 --2- 172.21.15.45:0/3794125936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f402c1088a0 0x7f402c108120 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4028011a50 tx=0x7f4028011b30 comp rx=0 tx=0).stop 2024-06-06T04:19:05.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.321+0000 7f403351d700 1 -- 172.21.15.45:0/3794125936 shutdown_connections 2024-06-06T04:19:05.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.321+0000 7f403351d700 1 --2- 172.21.15.45:0/3794125936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f402c1088a0 0x7f402c108120 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:05.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.321+0000 7f403351d700 1 -- 172.21.15.45:0/3794125936 >> 172.21.15.45:0/3794125936 conn(0x7f402c102290 msgr2=0x7f402c102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:05.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.321+0000 7f403351d700 1 -- 172.21.15.45:0/3794125936 shutdown_connections 2024-06-06T04:19:05.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:05.321+0000 7f403351d700 1 -- 172.21.15.45:0/3794125936 wait complete. 2024-06-06T04:19:05.903 INFO:teuthology.orchestra.run.smithi045.stdout:Restarting the monitor... 2024-06-06T04:19:06.180 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:05 smithi045 systemd[1]: Stopping Ceph mon.a for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:19:06.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:06 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-a[31926]: 2024-06-06T04:19:06.179+0000 7fcb510cd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:19:06.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:06 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-a[31926]: 2024-06-06T04:19:06.179+0000 7fcb510cd700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-06T04:19:06.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:06 smithi045 podman[32175]: 2024-06-06 04:19:06.323674775 +0000 UTC m=+0.247380571 container died 94de7a7ae2976707bdf5de682535001f195e723fd44d1d02b13518e56a580640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-06-06T04:19:06.813 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:06 smithi045 podman[32175]: 2024-06-06 04:19:06.629344684 +0000 UTC m=+0.553050478 container cleanup 94de7a7ae2976707bdf5de682535001f195e723fd44d1d02b13518e56a580640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:19:06.814 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:06 smithi045 bash[32175]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-a 2024-06-06T04:19:07.156 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:06 smithi045 podman[32187]: 2024-06-06 04:19:06.813922452 +0000 UTC m=+0.493711545 container remove 94de7a7ae2976707bdf5de682535001f195e723fd44d1d02b13518e56a580640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:19:07.156 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:06 smithi045 systemd[1]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-06T04:19:07.156 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:06 smithi045 systemd[1]: Stopped Ceph mon.a for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:19:07.156 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:06 smithi045 systemd[1]: Starting Ceph mon.a for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:19:07.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 podman[32281]: 2024-06-06 04:19:07.15654819 +0000 UTC m=+0.102760108 container create 3a75fd6d5b466f236b111a97443e544e9fcd4ab5f8ed62d756fe9640754c0f9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-a, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-06T04:19:07.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 podman[32281]: 2024-06-06 04:19:07.071721981 +0000 UTC m=+0.017933871 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T04:19:07.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 podman[32281]: 2024-06-06 04:19:07.33363389 +0000 UTC m=+0.279845778 container init 3a75fd6d5b466f236b111a97443e544e9fcd4ab5f8ed62d756fe9640754c0f9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-06-06T04:19:07.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 podman[32281]: 2024-06-06 04:19:07.336723226 +0000 UTC m=+0.282935104 container start 3a75fd6d5b466f236b111a97443e544e9fcd4ab5f8ed62d756fe9640754c0f9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240528) 2024-06-06T04:19:07.409 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: set uid:gid to 167:167 (ceph:ceph) 2024-06-06T04:19:07.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-06T04:19:07.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: pidfile_write: ignore empty --pid-file 2024-06-06T04:19:07.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: load: jerasure load: lrc 2024-06-06T04:19:07.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: RocksDB version: 7.9.2 2024-06-06T04:19:07.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Git sha 0 2024-06-06T04:19:07.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-06T04:19:07.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: DB SUMMARY 2024-06-06T04:19:07.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: DB Session ID: 3L8XONGTXUT8SZ27V1QP 2024-06-06T04:19:07.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: CURRENT file: CURRENT 2024-06-06T04:19:07.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: IDENTITY file: IDENTITY 2024-06-06T04:19:07.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-06T04:19:07.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-06T04:19:07.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104352 ; 2024-06-06T04:19:07.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.error_if_exists: 0 2024-06-06T04:19:07.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.create_if_missing: 0 2024-06-06T04:19:07.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.paranoid_checks: 1 2024-06-06T04:19:07.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-06T04:19:07.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-06T04:19:07.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-06T04:19:07.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.env: 0x55eef232dc60 2024-06-06T04:19:07.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.fs: PosixFileSystem 2024-06-06T04:19:07.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.info_log: 0x55eef2dc7b20 2024-06-06T04:19:07.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_file_opening_threads: 16 2024-06-06T04:19:07.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.statistics: (nil) 2024-06-06T04:19:07.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.use_fsync: 0 2024-06-06T04:19:07.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_log_file_size: 0 2024-06-06T04:19:07.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-06T04:19:07.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-06T04:19:07.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.keep_log_file_num: 1000 2024-06-06T04:19:07.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.recycle_log_file_num: 0 2024-06-06T04:19:07.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.allow_fallocate: 1 2024-06-06T04:19:07.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.allow_mmap_reads: 0 2024-06-06T04:19:07.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.allow_mmap_writes: 0 2024-06-06T04:19:07.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.use_direct_reads: 0 2024-06-06T04:19:07.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-06T04:19:07.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.create_missing_column_families: 0 2024-06-06T04:19:07.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.db_log_dir: 2024-06-06T04:19:07.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.wal_dir: 2024-06-06T04:19:07.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-06T04:19:07.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-06T04:19:07.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-06T04:19:07.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-06T04:19:07.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-06T04:19:07.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-06T04:19:07.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.advise_random_on_open: 1 2024-06-06T04:19:07.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.db_write_buffer_size: 0 2024-06-06T04:19:07.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.write_buffer_manager: 0x55eef2d981e0 2024-06-06T04:19:07.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-06T04:19:07.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-06T04:19:07.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-06T04:19:07.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.rate_limiter: (nil) 2024-06-06T04:19:07.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-06T04:19:07.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.wal_recovery_mode: 2 2024-06-06T04:19:07.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.enable_thread_tracking: 0 2024-06-06T04:19:07.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.enable_pipelined_write: 0 2024-06-06T04:19:07.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.unordered_write: 0 2024-06-06T04:19:07.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-06T04:19:07.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-06T04:19:07.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-06T04:19:07.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-06T04:19:07.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.row_cache: None 2024-06-06T04:19:07.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.wal_filter: None 2024-06-06T04:19:07.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-06T04:19:07.415 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.allow_ingest_behind: 0 2024-06-06T04:19:07.415 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.two_write_queues: 0 2024-06-06T04:19:07.415 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.manual_wal_flush: 0 2024-06-06T04:19:07.415 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.wal_compression: 0 2024-06-06T04:19:07.415 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.atomic_flush: 0 2024-06-06T04:19:07.415 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-06T04:19:07.415 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-06T04:19:07.415 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-06T04:19:07.415 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.log_readahead_size: 0 2024-06-06T04:19:07.415 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-06T04:19:07.415 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.best_efforts_recovery: 0 2024-06-06T04:19:07.415 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-06T04:19:07.415 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-06T04:19:07.415 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.allow_data_in_errors: 0 2024-06-06T04:19:07.415 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.db_host_id: __hostname__ 2024-06-06T04:19:07.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-06T04:19:07.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_background_jobs: 2 2024-06-06T04:19:07.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_background_compactions: -1 2024-06-06T04:19:07.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_subcompactions: 1 2024-06-06T04:19:07.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-06T04:19:07.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-06T04:19:07.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-06T04:19:07.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_total_wal_size: 0 2024-06-06T04:19:07.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-06T04:19:07.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-06T04:19:07.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-06T04:19:07.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-06T04:19:07.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_open_files: -1 2024-06-06T04:19:07.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.bytes_per_sync: 0 2024-06-06T04:19:07.417 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-06T04:19:07.417 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-06T04:19:07.417 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compaction_readahead_size: 0 2024-06-06T04:19:07.417 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_background_flushes: -1 2024-06-06T04:19:07.417 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Compression algorithms supported: 2024-06-06T04:19:07.417 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-06T04:19:07.417 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: kZSTD supported: 0 2024-06-06T04:19:07.417 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: kXpressCompression supported: 0 2024-06-06T04:19:07.417 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: kLZ4HCCompression supported: 1 2024-06-06T04:19:07.417 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: kZlibCompression supported: 1 2024-06-06T04:19:07.417 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: kSnappyCompression supported: 1 2024-06-06T04:19:07.417 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: kLZ4Compression supported: 1 2024-06-06T04:19:07.417 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: kBZip2Compression supported: 0 2024-06-06T04:19:07.417 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-06T04:19:07.418 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-06T04:19:07.418 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-06T04:19:07.418 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-06T04:19:07.418 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-06T04:19:07.418 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.merge_operator: 2024-06-06T04:19:07.418 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compaction_filter: None 2024-06-06T04:19:07.418 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compaction_filter_factory: None 2024-06-06T04:19:07.418 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.sst_partitioner_factory: None 2024-06-06T04:19:07.418 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-06T04:19:07.418 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-06T04:19:07.418 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55eef2dc7c40) 2024-06-06T04:19:07.418 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks: 1 2024-06-06T04:19:07.418 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-06T04:19:07.418 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-06T04:19:07.419 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_top_level_index_and_filter: 1 2024-06-06T04:19:07.419 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_type: 0 2024-06-06T04:19:07.419 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_index_type: 0 2024-06-06T04:19:07.419 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_shortening: 1 2024-06-06T04:19:07.419 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-06T04:19:07.419 INFO:journalctl@ceph.mon.a.smithi045.stdout: checksum: 4 2024-06-06T04:19:07.419 INFO:journalctl@ceph.mon.a.smithi045.stdout: no_block_cache: 0 2024-06-06T04:19:07.419 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache: 0x55eef2dc9090 2024-06-06T04:19:07.419 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_name: BinnedLRUCache 2024-06-06T04:19:07.419 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_options: 2024-06-06T04:19:07.419 INFO:journalctl@ceph.mon.a.smithi045.stdout: capacity : 536870912 2024-06-06T04:19:07.419 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_shard_bits : 4 2024-06-06T04:19:07.419 INFO:journalctl@ceph.mon.a.smithi045.stdout: strict_capacity_limit : 0 2024-06-06T04:19:07.419 INFO:journalctl@ceph.mon.a.smithi045.stdout: high_pri_pool_ratio: 0.000 2024-06-06T04:19:07.419 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_compressed: (nil) 2024-06-06T04:19:07.420 INFO:journalctl@ceph.mon.a.smithi045.stdout: persistent_cache: (nil) 2024-06-06T04:19:07.420 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size: 4096 2024-06-06T04:19:07.420 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size_deviation: 10 2024-06-06T04:19:07.420 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_restart_interval: 16 2024-06-06T04:19:07.420 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_block_restart_interval: 1 2024-06-06T04:19:07.421 INFO:journalctl@ceph.mon.a.smithi045.stdout: metadata_block_size: 4096 2024-06-06T04:19:07.421 INFO:journalctl@ceph.mon.a.smithi045.stdout: partition_filters: 0 2024-06-06T04:19:07.421 INFO:journalctl@ceph.mon.a.smithi045.stdout: use_delta_encoding: 1 2024-06-06T04:19:07.421 INFO:journalctl@ceph.mon.a.smithi045.stdout: filter_policy: bloomfilter 2024-06-06T04:19:07.421 INFO:journalctl@ceph.mon.a.smithi045.stdout: whole_key_filtering: 1 2024-06-06T04:19:07.421 INFO:journalctl@ceph.mon.a.smithi045.stdout: verify_compression: 0 2024-06-06T04:19:07.421 INFO:journalctl@ceph.mon.a.smithi045.stdout: read_amp_bytes_per_bit: 0 2024-06-06T04:19:07.421 INFO:journalctl@ceph.mon.a.smithi045.stdout: format_version: 5 2024-06-06T04:19:07.421 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_index_compression: 1 2024-06-06T04:19:07.421 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_align: 0 2024-06-06T04:19:07.421 INFO:journalctl@ceph.mon.a.smithi045.stdout: max_auto_readahead_size: 262144 2024-06-06T04:19:07.421 INFO:journalctl@ceph.mon.a.smithi045.stdout: prepopulate_block_cache: 0 2024-06-06T04:19:07.422 INFO:journalctl@ceph.mon.a.smithi045.stdout: initial_auto_readahead_size: 8192 2024-06-06T04:19:07.422 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_file_reads_for_auto_readahead: 2 2024-06-06T04:19:07.422 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.write_buffer_size: 33554432 2024-06-06T04:19:07.422 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_write_buffer_number: 2 2024-06-06T04:19:07.422 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compression: NoCompression 2024-06-06T04:19:07.422 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.bottommost_compression: Disabled 2024-06-06T04:19:07.422 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.prefix_extractor: nullptr 2024-06-06T04:19:07.422 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-06T04:19:07.422 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.num_levels: 7 2024-06-06T04:19:07.422 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-06T04:19:07.422 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-06T04:19:07.422 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-06T04:19:07.423 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-06T04:19:07.423 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-06T04:19:07.423 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-06T04:19:07.423 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-06T04:19:07.423 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-06T04:19:07.423 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-06T04:19:07.423 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-06T04:19:07.423 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-06T04:19:07.423 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-06T04:19:07.423 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-06T04:19:07.423 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compression_opts.level: 32767 2024-06-06T04:19:07.423 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compression_opts.strategy: 0 2024-06-06T04:19:07.423 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-06T04:19:07.423 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-06T04:19:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-06T04:19:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-06T04:19:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compression_opts.enabled: false 2024-06-06T04:19:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-06T04:19:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-06T04:19:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-06T04:19:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-06T04:19:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.target_file_size_base: 67108864 2024-06-06T04:19:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-06T04:19:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-06T04:19:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-06T04:19:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-06T04:19:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-06T04:19:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-06T04:19:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-06T04:19:07.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-06T04:19:07.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-06T04:19:07.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-06T04:19:07.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-06T04:19:07.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-06T04:19:07.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-06T04:19:07.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-06T04:19:07.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.arena_block_size: 1048576 2024-06-06T04:19:07.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-06T04:19:07.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-06T04:19:07.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.disable_auto_compactions: 0 2024-06-06T04:19:07.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-06T04:19:07.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-06T04:19:07.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-06T04:19:07.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-06T04:19:07.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-06T04:19:07.426 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-06T04:19:07.426 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-06T04:19:07.426 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-06T04:19:07.426 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-06T04:19:07.426 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-06T04:19:07.426 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.table_properties_collectors: 2024-06-06T04:19:07.426 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.inplace_update_support: 0 2024-06-06T04:19:07.426 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-06T04:19:07.426 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-06T04:19:07.426 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-06T04:19:07.426 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-06T04:19:07.426 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.bloom_locality: 0 2024-06-06T04:19:07.426 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.max_successive_merges: 0 2024-06-06T04:19:07.426 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-06T04:19:07.427 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.paranoid_file_checks: 0 2024-06-06T04:19:07.427 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.force_consistency_checks: 1 2024-06-06T04:19:07.427 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.report_bg_io_stats: 0 2024-06-06T04:19:07.427 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.ttl: 2592000 2024-06-06T04:19:07.427 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-06T04:19:07.427 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-06T04:19:07.427 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-06T04:19:07.427 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.enable_blob_files: false 2024-06-06T04:19:07.427 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.min_blob_size: 0 2024-06-06T04:19:07.427 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.blob_file_size: 268435456 2024-06-06T04:19:07.427 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-06T04:19:07.427 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-06T04:19:07.427 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-06T04:19:07.427 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-06T04:19:07.428 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-06T04:19:07.428 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.blob_file_starting_level: 0 2024-06-06T04:19:07.428 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-06T04:19:07.428 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-06T04:19:07.428 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-06T04:19:07.428 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e2b8581f-c881-40e9-9b5a-cec73e2885ae 2024-06-06T04:19:07.428 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717647547366291, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-06T04:19:07.428 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-06T04:19:07.428 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717647547367308, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99529, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97437, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14060, "raw_average_key_size": 49, "raw_value_size": 89787, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717647547, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e2b8581f-c881-40e9-9b5a-cec73e2885ae", "db_session_id": "3L8XONGTXUT8SZ27V1QP", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-06T04:19:07.428 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717647547367392, "job": 1, "event": "recovery_finished"} 2024-06-06T04:19:07.428 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-06T04:19:07.428 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-06T04:19:07.428 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55eef2eb8000 2024-06-06T04:19:07.429 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: DB pointer 0x55eef2ea2000 2024-06-06T04:19:07.429 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-06T04:19:07.429 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-06T04:19:07.429 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** DB Stats ** 2024-06-06T04:19:07.429 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-06T04:19:07.429 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-06T04:19:07.429 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-06T04:19:07.429 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-06T04:19:07.429 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-06T04:19:07.429 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-06T04:19:07.429 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-06T04:19:07.429 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-06-06T04:19:07.430 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-06-06T04:19:07.430 INFO:journalctl@ceph.mon.a.smithi045.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-06T04:19:07.430 INFO:journalctl@ceph.mon.a.smithi045.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-06T04:19:07.430 INFO:journalctl@ceph.mon.a.smithi045.stdout: L0 2/0 99.03 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T04:19:07.430 INFO:journalctl@ceph.mon.a.smithi045.stdout: Sum 2/0 99.03 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T04:19:07.430 INFO:journalctl@ceph.mon.a.smithi045.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T04:19:07.430 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-06-06T04:19:07.430 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-06-06T04:19:07.430 INFO:journalctl@ceph.mon.a.smithi045.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-06T04:19:07.430 INFO:journalctl@ceph.mon.a.smithi045.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-06T04:19:07.430 INFO:journalctl@ceph.mon.a.smithi045.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 156.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T04:19:07.430 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-06-06T04:19:07.431 INFO:journalctl@ceph.mon.a.smithi045.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-06T04:19:07.431 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-06-06T04:19:07.431 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-06T04:19:07.431 INFO:journalctl@ceph.mon.a.smithi045.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-06T04:19:07.431 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-06T04:19:07.431 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-06T04:19:07.431 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-06T04:19:07.431 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-06T04:19:07.431 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative compaction: 0.00 GB write, 30.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-06T04:19:07.431 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval compaction: 0.00 GB write, 30.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-06T04:19:07.431 INFO:journalctl@ceph.mon.a.smithi045.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-06T04:19:07.431 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache BinnedLRUCache@0x55eef2dc9090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-06-06T04:19:07.431 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-06-06T04:19:07.432 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-06-06T04:19:07.432 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-06T04:19:07.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: starting mon.a rank 0 at public addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] at bind addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:19:07.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: mon.a@-1(???) e1 preinit fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:19:07.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: mon.a@-1(???).mds e1 new map 2024-06-06T04:19:07.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: mon.a@-1(???).mds e1 print_map 2024-06-06T04:19:07.432 INFO:journalctl@ceph.mon.a.smithi045.stdout: e1 2024-06-06T04:19:07.432 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-06T04:19:07.432 INFO:journalctl@ceph.mon.a.smithi045.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T04:19:07.432 INFO:journalctl@ceph.mon.a.smithi045.stdout: legacy client fscid: -1 2024-06-06T04:19:07.432 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-06-06T04:19:07.432 INFO:journalctl@ceph.mon.a.smithi045.stdout: No filesystems configured 2024-06-06T04:19:07.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-06T04:19:07.433 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T04:19:07.433 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T04:19:07.433 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T04:19:07.433 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-06T04:19:07.433 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 bash[32281]: 3a75fd6d5b466f236b111a97443e544e9fcd4ab5f8ed62d756fe9640754c0f9c 2024-06-06T04:19:07.433 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 systemd[1]: Started Ceph mon.a for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:19:07.433 INFO:teuthology.orchestra.run.smithi045.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-06T04:19:07.686 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-06T04:19:07.686 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: monmap epoch 1 2024-06-06T04:19:07.686 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:19:07.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: last_changed 2024-06-06T04:18:59.496605+0000 2024-06-06T04:19:07.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: created 2024-06-06T04:18:59.496605+0000 2024-06-06T04:19:07.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: min_mon_release 19 (squid) 2024-06-06T04:19:07.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: election_strategy: 1 2024-06-06T04:19:07.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-06-06T04:19:07.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: fsmap 2024-06-06T04:19:07.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: osdmap e1: 0 total, 0 up, 0 in 2024-06-06T04:19:07.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:07 smithi045 ceph-mon[32299]: mgrmap e1: no daemons active 2024-06-06T04:19:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.792+0000 7fa06b189700 1 Processor -- start 2024-06-06T04:19:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.792+0000 7fa06b189700 1 -- start start 2024-06-06T04:19:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.792+0000 7fa06b189700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06407de00 0x7fa06407c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.792+0000 7fa06b189700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa06407c7f0 con 0x7fa06407de00 2024-06-06T04:19:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.792+0000 7fa068f25700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06407de00 0x7fa06407c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.792+0000 7fa068f25700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06407de00 0x7fa06407c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38388/0 (socket says 172.21.15.45:38388) 2024-06-06T04:19:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.792+0000 7fa068f25700 1 -- 172.21.15.45:0/1968961036 learned_addr learned my addr 172.21.15.45:0/1968961036 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.793+0000 7fa068f25700 1 -- 172.21.15.45:0/1968961036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06407c970 con 0x7fa06407de00 2024-06-06T04:19:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.793+0000 7fa068f25700 1 --2- 172.21.15.45:0/1968961036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06407de00 0x7fa06407c2b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa0580098a0 tx=0x7fa058009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6ce5b4870924d4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.793+0000 7fa0637fe700 1 -- 172.21.15.45:0/1968961036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa058004160 con 0x7fa06407de00 2024-06-06T04:19:08.472 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.793+0000 7fa0637fe700 1 -- 172.21.15.45:0/1968961036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa058004300 con 0x7fa06407de00 2024-06-06T04:19:08.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.793+0000 7fa0637fe700 1 -- 172.21.15.45:0/1968961036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa058011a80 con 0x7fa06407de00 2024-06-06T04:19:08.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.793+0000 7fa06b189700 1 -- 172.21.15.45:0/1968961036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06407de00 msgr2=0x7fa06407c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:08.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.793+0000 7fa06b189700 1 --2- 172.21.15.45:0/1968961036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06407de00 0x7fa06407c2b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa0580098a0 tx=0x7fa058009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:08.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.794+0000 7fa06b189700 1 -- 172.21.15.45:0/1968961036 shutdown_connections 2024-06-06T04:19:08.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.794+0000 7fa06b189700 1 --2- 172.21.15.45:0/1968961036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06407de00 0x7fa06407c2b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:08.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.794+0000 7fa06b189700 1 -- 172.21.15.45:0/1968961036 >> 172.21.15.45:0/1968961036 conn(0x7fa064103330 msgr2=0x7fa064105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:08.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.794+0000 7fa06b189700 1 -- 172.21.15.45:0/1968961036 shutdown_connections 2024-06-06T04:19:08.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.794+0000 7fa06b189700 1 -- 172.21.15.45:0/1968961036 wait complete. 2024-06-06T04:19:08.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.794+0000 7fa06b189700 1 Processor -- start 2024-06-06T04:19:08.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.794+0000 7fa06b189700 1 -- start start 2024-06-06T04:19:08.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.794+0000 7fa06b189700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06407de00 0x7fa06419de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:08.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.794+0000 7fa06b189700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa06410e340 con 0x7fa06407de00 2024-06-06T04:19:08.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.795+0000 7fa068f25700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06407de00 0x7fa06419de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:08.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.795+0000 7fa068f25700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06407de00 0x7fa06419de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38390/0 (socket says 172.21.15.45:38390) 2024-06-06T04:19:08.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.795+0000 7fa068f25700 1 -- 172.21.15.45:0/3569475562 learned_addr learned my addr 172.21.15.45:0/3569475562 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:08.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.795+0000 7fa068f25700 1 -- 172.21.15.45:0/3569475562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06419e380 con 0x7fa06407de00 2024-06-06T04:19:08.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.795+0000 7fa068f25700 1 --2- 172.21.15.45:0/3569475562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06407de00 0x7fa06419de40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa058006af0 tx=0x7fa058011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:08.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.795+0000 7fa061ffb700 1 -- 172.21.15.45:0/3569475562 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0580039e0 con 0x7fa06407de00 2024-06-06T04:19:08.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.795+0000 7fa061ffb700 1 -- 172.21.15.45:0/3569475562 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa058003b80 con 0x7fa06407de00 2024-06-06T04:19:08.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.795+0000 7fa06b189700 1 -- 172.21.15.45:0/3569475562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06419e610 con 0x7fa06407de00 2024-06-06T04:19:08.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.795+0000 7fa061ffb700 1 -- 172.21.15.45:0/3569475562 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0580196a0 con 0x7fa06407de00 2024-06-06T04:19:08.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.795+0000 7fa06b189700 1 -- 172.21.15.45:0/3569475562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0641a1300 con 0x7fa06407de00 2024-06-06T04:19:08.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.796+0000 7fa061ffb700 1 -- 172.21.15.45:0/3569475562 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa058019ce0 con 0x7fa06407de00 2024-06-06T04:19:08.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.796+0000 7fa061ffb700 1 -- 172.21.15.45:0/3569475562 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa058024730 con 0x7fa06407de00 2024-06-06T04:19:08.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.796+0000 7fa06b189700 1 -- 172.21.15.45:0/3569475562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa064063af0 con 0x7fa06407de00 2024-06-06T04:19:08.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.799+0000 7fa061ffb700 1 -- 172.21.15.45:0/3569475562 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa058043dd0 con 0x7fa06407de00 2024-06-06T04:19:08.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.869+0000 7fa06b189700 1 -- 172.21.15.45:0/3569475562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fa0641a14d0 con 0x7fa06407de00 2024-06-06T04:19:08.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.871+0000 7fa061ffb700 1 -- 172.21.15.45:0/3569475562 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fa058032020 con 0x7fa06407de00 2024-06-06T04:19:08.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.872+0000 7fa06b189700 1 -- 172.21.15.45:0/3569475562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06407de00 msgr2=0x7fa06419de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:08.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.872+0000 7fa06b189700 1 --2- 172.21.15.45:0/3569475562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06407de00 0x7fa06419de40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa058006af0 tx=0x7fa058011c40 comp rx=0 tx=0).stop 2024-06-06T04:19:08.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.872+0000 7fa06b189700 1 -- 172.21.15.45:0/3569475562 shutdown_connections 2024-06-06T04:19:08.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.872+0000 7fa06b189700 1 --2- 172.21.15.45:0/3569475562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa06407de00 0x7fa06419de40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:08.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.872+0000 7fa06b189700 1 -- 172.21.15.45:0/3569475562 >> 172.21.15.45:0/3569475562 conn(0x7fa064103330 msgr2=0x7fa064103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:08.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.872+0000 7fa06b189700 1 -- 172.21.15.45:0/3569475562 shutdown_connections 2024-06-06T04:19:08.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:07.872+0000 7fa06b189700 1 -- 172.21.15.45:0/3569475562 wait complete. 2024-06-06T04:19:08.475 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-06T04:19:08.498 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-06T04:19:08.498 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mgr... 2024-06-06T04:19:08.498 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-06T04:19:08.499 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-06T04:19:08.761 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mgr.x 2024-06-06T04:19:08.761 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-06-06T04:19:08.911 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968.target.wants/ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@.service. 2024-06-06T04:19:09.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:08 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3569475562' entity='client.admin' 2024-06-06T04:19:09.142 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:08 smithi045 systemd[1]: Starting Ceph mgr.x for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:19:09.435 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:09 smithi045 podman[32515]: 2024-06-06 04:19:09.142057166 +0000 UTC m=+0.107026433 container create 8bc2882f94572fb416ce4f76ecfc4b219074c07bd8188e377a0f415aae5cba6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-06-06T04:19:09.436 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:09 smithi045 podman[32515]: 2024-06-06 04:19:09.052436299 +0000 UTC m=+0.017405573 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T04:19:09.436 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:09 smithi045 podman[32515]: 2024-06-06 04:19:09.344199284 +0000 UTC m=+0.309168559 container init 8bc2882f94572fb416ce4f76ecfc4b219074c07bd8188e377a0f415aae5cba6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T04:19:09.436 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:09 smithi045 podman[32515]: 2024-06-06 04:19:09.347088012 +0000 UTC m=+0.312057275 container start 8bc2882f94572fb416ce4f76ecfc4b219074c07bd8188e377a0f415aae5cba6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-06T04:19:09.436 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:09 smithi045 bash[32515]: 8bc2882f94572fb416ce4f76ecfc4b219074c07bd8188e377a0f415aae5cba6b 2024-06-06T04:19:09.449 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T04:19:09.450 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-06-06T04:19:09.458 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T04:19:09.458 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-06-06T04:19:09.458 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-06-06T04:19:09.458 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-06-06T04:19:09.467 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T04:19:09.467 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-06-06T04:19:09.475 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T04:19:09.475 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-06-06T04:19:09.475 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-06-06T04:19:09.475 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-06T04:19:09.475 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr to start... 2024-06-06T04:19:09.475 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr... 2024-06-06T04:19:09.705 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:09 smithi045 systemd[1]: Started Ceph mgr.x for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:19:09.705 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:09 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:09.510+0000 7f1fe34b2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T04:19:09.705 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:09 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:09.704+0000 7f1fe34b2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T04:19:09.976 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:09 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:09.813+0000 7f1fe34b2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T04:19:10.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:10 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2989410374' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T04:19:10.367 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:10 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:10.149+0000 7f1fe34b2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T04:19:10.367 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:10 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:10.239+0000 7f1fe34b2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T04:19:10.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-06-06T04:19:10.642 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-06-06T04:19:10.642 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "c4f6b938-23bb-11ef-bc9b-c7b262605968", 2024-06-06T04:19:10.642 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T04:19:10.642 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T04:19:10.642 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T04:19:10.642 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T04:19:10.642 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:10.642 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T04:19:10.642 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T04:19:10.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-06-06T04:19:10.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-06-06T04:19:10.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T04:19:10.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T04:19:10.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-06-06T04:19:10.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-06-06T04:19:10.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T04:19:10.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T04:19:10.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T04:19:10.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T04:19:10.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:10.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T04:19:10.644 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T04:19:10.644 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T04:19:10.644 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T04:19:10.644 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T04:19:10.644 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T04:19:10.644 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T04:19:10.644 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T04:19:10.644 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:10.644 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T04:19:10.644 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T04:19:10.644 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T04:19:10.644 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T04:19:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T04:19:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T04:19:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T04:19:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T04:19:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T04:19:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T04:19:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T04:19:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T04:19:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T04:19:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T04:19:10.646 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-06T04:19:10.646 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T04:19:10.646 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T04:19:10.646 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T04:19:10.646 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T04:19:10.646 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T04:19:10.646 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-06-06T04:19:10.646 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T04:19:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T04:19:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T04:19:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T04:19:02.627392+0000", 2024-06-06T04:19:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T04:19:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T04:19:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-06-06T04:19:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.925+0000 7fe3018ad700 1 Processor -- start 2024-06-06T04:19:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.926+0000 7fe3018ad700 1 -- start start 2024-06-06T04:19:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.926+0000 7fe3018ad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2fc106c80 0x7fe2fc1070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.926+0000 7fe3018ad700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc107670 con 0x7fe2fc106c80 2024-06-06T04:19:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.926+0000 7fe2faffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2fc106c80 0x7fe2fc1070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.926+0000 7fe2faffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2fc106c80 0x7fe2fc1070a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38426/0 (socket says 172.21.15.45:38426) 2024-06-06T04:19:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.926+0000 7fe2faffd700 1 -- 172.21.15.45:0/2161633097 learned_addr learned my addr 172.21.15.45:0/2161633097 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.926+0000 7fe2faffd700 1 -- 172.21.15.45:0/2161633097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2fc107e80 con 0x7fe2fc106c80 2024-06-06T04:19:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.927+0000 7fe2faffd700 1 --2- 172.21.15.45:0/2161633097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2fc106c80 0x7fe2fc1070a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0098a0 tx=0x7fe2ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98392b1841966645 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.927+0000 7fe2f9ffb700 1 -- 172.21.15.45:0/2161633097 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2ec004160 con 0x7fe2fc106c80 2024-06-06T04:19:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.927+0000 7fe2f9ffb700 1 -- 172.21.15.45:0/2161633097 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe2ec004300 con 0x7fe2fc106c80 2024-06-06T04:19:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.927+0000 7fe2f9ffb700 1 -- 172.21.15.45:0/2161633097 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2ec011a80 con 0x7fe2fc106c80 2024-06-06T04:19:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.928+0000 7fe3018ad700 1 -- 172.21.15.45:0/2161633097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2fc106c80 msgr2=0x7fe2fc1070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.928+0000 7fe3018ad700 1 --2- 172.21.15.45:0/2161633097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2fc106c80 0x7fe2fc1070a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0098a0 tx=0x7fe2ec009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.928+0000 7fe3018ad700 1 -- 172.21.15.45:0/2161633097 shutdown_connections 2024-06-06T04:19:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.928+0000 7fe3018ad700 1 --2- 172.21.15.45:0/2161633097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2fc106c80 0x7fe2fc1070a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.928+0000 7fe3018ad700 1 -- 172.21.15.45:0/2161633097 >> 172.21.15.45:0/2161633097 conn(0x7fe2fc102240 msgr2=0x7fe2fc104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.928+0000 7fe3018ad700 1 -- 172.21.15.45:0/2161633097 shutdown_connections 2024-06-06T04:19:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.928+0000 7fe3018ad700 1 -- 172.21.15.45:0/2161633097 wait complete. 2024-06-06T04:19:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.928+0000 7fe3018ad700 1 Processor -- start 2024-06-06T04:19:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.929+0000 7fe3018ad700 1 -- start start 2024-06-06T04:19:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.929+0000 7fe3018ad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2fc106c80 0x7fe2fc19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.929+0000 7fe3018ad700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc10b230 con 0x7fe2fc106c80 2024-06-06T04:19:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.929+0000 7fe2faffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2fc106c80 0x7fe2fc19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.929+0000 7fe2faffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2fc106c80 0x7fe2fc19bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38436/0 (socket says 172.21.15.45:38436) 2024-06-06T04:19:10.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.929+0000 7fe2faffd700 1 -- 172.21.15.45:0/2989410374 learned_addr learned my addr 172.21.15.45:0/2989410374 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:10.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.929+0000 7fe2faffd700 1 -- 172.21.15.45:0/2989410374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2fc19c270 con 0x7fe2fc106c80 2024-06-06T04:19:10.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.929+0000 7fe2faffd700 1 --2- 172.21.15.45:0/2989410374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2fc106c80 0x7fe2fc19bd30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec000c00 tx=0x7fe2ec011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:10.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.929+0000 7fe3008ab700 1 -- 172.21.15.45:0/2989410374 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2ec0039f0 con 0x7fe2fc106c80 2024-06-06T04:19:10.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.929+0000 7fe3008ab700 1 -- 172.21.15.45:0/2989410374 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe2ec003b90 con 0x7fe2fc106c80 2024-06-06T04:19:10.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.930+0000 7fe3018ad700 1 -- 172.21.15.45:0/2989410374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2fc19c500 con 0x7fe2fc106c80 2024-06-06T04:19:10.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.930+0000 7fe3008ab700 1 -- 172.21.15.45:0/2989410374 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2ec01a740 con 0x7fe2fc106c80 2024-06-06T04:19:10.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.930+0000 7fe3018ad700 1 -- 172.21.15.45:0/2989410374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2fc19f1f0 con 0x7fe2fc106c80 2024-06-06T04:19:10.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.930+0000 7fe3008ab700 1 -- 172.21.15.45:0/2989410374 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe2ec02d3f0 con 0x7fe2fc106c80 2024-06-06T04:19:10.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.930+0000 7fe3008ab700 1 -- 172.21.15.45:0/2989410374 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe2ec019700 con 0x7fe2fc106c80 2024-06-06T04:19:10.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.931+0000 7fe3018ad700 1 -- 172.21.15.45:0/2989410374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2fc063af0 con 0x7fe2fc106c80 2024-06-06T04:19:10.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:09.933+0000 7fe3008ab700 1 -- 172.21.15.45:0/2989410374 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe2ec024af0 con 0x7fe2fc106c80 2024-06-06T04:19:10.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:10.006+0000 7fe3018ad700 1 -- 172.21.15.45:0/2989410374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe2fc02de80 con 0x7fe2fc106c80 2024-06-06T04:19:10.656 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:10.007+0000 7fe3008ab700 1 -- 172.21.15.45:0/2989410374 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe2ec0199a0 con 0x7fe2fc106c80 2024-06-06T04:19:10.656 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:10.008+0000 7fe3018ad700 1 -- 172.21.15.45:0/2989410374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2fc106c80 msgr2=0x7fe2fc19bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:10.656 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:10.008+0000 7fe3018ad700 1 --2- 172.21.15.45:0/2989410374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2fc106c80 0x7fe2fc19bd30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec000c00 tx=0x7fe2ec011b00 comp rx=0 tx=0).stop 2024-06-06T04:19:10.656 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:10.008+0000 7fe3018ad700 1 -- 172.21.15.45:0/2989410374 shutdown_connections 2024-06-06T04:19:10.656 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:10.008+0000 7fe3018ad700 1 --2- 172.21.15.45:0/2989410374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2fc106c80 0x7fe2fc19bd30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:10.656 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:10.008+0000 7fe3018ad700 1 -- 172.21.15.45:0/2989410374 >> 172.21.15.45:0/2989410374 conn(0x7fe2fc102240 msgr2=0x7fe2fc1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:10.656 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:10.008+0000 7fe3018ad700 1 -- 172.21.15.45:0/2989410374 shutdown_connections 2024-06-06T04:19:10.656 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:10.008+0000 7fe3018ad700 1 -- 172.21.15.45:0/2989410374 wait complete. 2024-06-06T04:19:10.656 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (1/15)... 2024-06-06T04:19:11.859 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:11 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:11.507+0000 7f1fe34b2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T04:19:11.859 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:11 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:11.682+0000 7f1fe34b2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T04:19:11.859 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:11 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:11.858+0000 7f1fe34b2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T04:19:12.160 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:12 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:12.159+0000 7f1fe34b2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T04:19:12.466 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:12 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:12.255+0000 7f1fe34b2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T04:19:12.466 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:12 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:12.465+0000 7f1fe34b2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T04:19:12.726 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:12 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:12.670+0000 7f1fe34b2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T04:19:13.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:13 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3200695526' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T04:19:13.682 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:13 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:13.315+0000 7f1fe34b2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T04:19:13.686 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-06-06T04:19:13.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-06-06T04:19:13.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "c4f6b938-23bb-11ef-bc9b-c7b262605968", 2024-06-06T04:19:13.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T04:19:13.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T04:19:13.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T04:19:13.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T04:19:13.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:13.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T04:19:13.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T04:19:13.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-06-06T04:19:13.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-06-06T04:19:13.687 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T04:19:13.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T04:19:13.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-06-06T04:19:13.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-06-06T04:19:13.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T04:19:13.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T04:19:13.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T04:19:13.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T04:19:13.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:13.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T04:19:13.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T04:19:13.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T04:19:13.688 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T04:19:13.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T04:19:13.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T04:19:13.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T04:19:13.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T04:19:13.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:13.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T04:19:13.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T04:19:13.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T04:19:13.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T04:19:13.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T04:19:13.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T04:19:13.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T04:19:13.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T04:19:13.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T04:19:13.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:13.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T04:19:13.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T04:19:13.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T04:19:13.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T04:19:13.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:13.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T04:19:13.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-06T04:19:13.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T04:19:13.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T04:19:13.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T04:19:13.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T04:19:13.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T04:19:13.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-06-06T04:19:13.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T04:19:13.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:13.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T04:19:13.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T04:19:13.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T04:19:02.627392+0000", 2024-06-06T04:19:13.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T04:19:13.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:13.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T04:19:13.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-06-06T04:19:13.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.038+0000 7fc641b1f700 1 Processor -- start 2024-06-06T04:19:13.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.038+0000 7fc641b1f700 1 -- start start 2024-06-06T04:19:13.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.038+0000 7fc641b1f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c106c60 0x7fc63c107080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:13.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.038+0000 7fc641b1f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c107650 con 0x7fc63c106c60 2024-06-06T04:19:13.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.038+0000 7fc63b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c106c60 0x7fc63c107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:13.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.038+0000 7fc63b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c106c60 0x7fc63c107080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38440/0 (socket says 172.21.15.45:38440) 2024-06-06T04:19:13.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.038+0000 7fc63b7fe700 1 -- 172.21.15.45:0/4242990670 learned_addr learned my addr 172.21.15.45:0/4242990670 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:13.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.038+0000 7fc63b7fe700 1 -- 172.21.15.45:0/4242990670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63c107e60 con 0x7fc63c106c60 2024-06-06T04:19:13.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.039+0000 7fc63b7fe700 1 --2- 172.21.15.45:0/4242990670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c106c60 0x7fc63c107080 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00bb40 tx=0x7fc62c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabeccb8bdecf749 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:13.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.039+0000 7fc63a7fc700 1 -- 172.21.15.45:0/4242990670 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc62c00dbe0 con 0x7fc63c106c60 2024-06-06T04:19:13.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.039+0000 7fc63a7fc700 1 -- 172.21.15.45:0/4242990670 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc62c0044c0 con 0x7fc63c106c60 2024-06-06T04:19:13.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.039+0000 7fc63a7fc700 1 -- 172.21.15.45:0/4242990670 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc62c010470 con 0x7fc63c106c60 2024-06-06T04:19:13.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.040+0000 7fc641b1f700 1 -- 172.21.15.45:0/4242990670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c106c60 msgr2=0x7fc63c107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:13.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.040+0000 7fc641b1f700 1 --2- 172.21.15.45:0/4242990670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c106c60 0x7fc63c107080 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00bb40 tx=0x7fc62c00be50 comp rx=0 tx=0).stop 2024-06-06T04:19:13.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.040+0000 7fc641b1f700 1 -- 172.21.15.45:0/4242990670 shutdown_connections 2024-06-06T04:19:13.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.040+0000 7fc641b1f700 1 --2- 172.21.15.45:0/4242990670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c106c60 0x7fc63c107080 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:13.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.040+0000 7fc641b1f700 1 -- 172.21.15.45:0/4242990670 >> 172.21.15.45:0/4242990670 conn(0x7fc63c1021e0 msgr2=0x7fc63c104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:13.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.040+0000 7fc641b1f700 1 -- 172.21.15.45:0/4242990670 shutdown_connections 2024-06-06T04:19:13.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.040+0000 7fc641b1f700 1 -- 172.21.15.45:0/4242990670 wait complete. 2024-06-06T04:19:13.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.040+0000 7fc641b1f700 1 Processor -- start 2024-06-06T04:19:13.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.041+0000 7fc641b1f700 1 -- start start 2024-06-06T04:19:13.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.041+0000 7fc641b1f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c106c60 0x7fc63c19df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:13.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.041+0000 7fc641b1f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c07afc0 con 0x7fc63c106c60 2024-06-06T04:19:13.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.041+0000 7fc63b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c106c60 0x7fc63c19df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:13.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.041+0000 7fc63b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c106c60 0x7fc63c19df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38454/0 (socket says 172.21.15.45:38454) 2024-06-06T04:19:13.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.041+0000 7fc63b7fe700 1 -- 172.21.15.45:0/3200695526 learned_addr learned my addr 172.21.15.45:0/3200695526 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:13.701 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.041+0000 7fc63b7fe700 1 -- 172.21.15.45:0/3200695526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63c19e470 con 0x7fc63c106c60 2024-06-06T04:19:13.702 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.041+0000 7fc63b7fe700 1 --2- 172.21.15.45:0/3200695526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c106c60 0x7fc63c19df30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00d650 tx=0x7fc62c003fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:13.702 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.041+0000 7fc638ff9700 1 -- 172.21.15.45:0/3200695526 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc62c00de20 con 0x7fc63c106c60 2024-06-06T04:19:13.702 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.041+0000 7fc638ff9700 1 -- 172.21.15.45:0/3200695526 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc62c01d070 con 0x7fc63c106c60 2024-06-06T04:19:13.702 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.041+0000 7fc641b1f700 1 -- 172.21.15.45:0/3200695526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63c19e700 con 0x7fc63c106c60 2024-06-06T04:19:13.702 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.041+0000 7fc638ff9700 1 -- 172.21.15.45:0/3200695526 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc62c019bd0 con 0x7fc63c106c60 2024-06-06T04:19:13.702 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.042+0000 7fc641b1f700 1 -- 172.21.15.45:0/3200695526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63c1a13a0 con 0x7fc63c106c60 2024-06-06T04:19:13.702 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.042+0000 7fc638ff9700 1 -- 172.21.15.45:0/3200695526 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc62c028060 con 0x7fc63c106c60 2024-06-06T04:19:13.702 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.042+0000 7fc638ff9700 1 -- 172.21.15.45:0/3200695526 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc62c021750 con 0x7fc63c106c60 2024-06-06T04:19:13.702 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.042+0000 7fc641b1f700 1 -- 172.21.15.45:0/3200695526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc63c063af0 con 0x7fc63c106c60 2024-06-06T04:19:13.702 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.045+0000 7fc638ff9700 1 -- 172.21.15.45:0/3200695526 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc62c040dd0 con 0x7fc63c106c60 2024-06-06T04:19:13.702 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.119+0000 7fc641b1f700 1 -- 172.21.15.45:0/3200695526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc63c197490 con 0x7fc63c106c60 2024-06-06T04:19:13.703 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.119+0000 7fc638ff9700 1 -- 172.21.15.45:0/3200695526 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc62c02f070 con 0x7fc63c106c60 2024-06-06T04:19:13.703 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.120+0000 7fc641b1f700 1 -- 172.21.15.45:0/3200695526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c106c60 msgr2=0x7fc63c19df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:13.703 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.120+0000 7fc641b1f700 1 --2- 172.21.15.45:0/3200695526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c106c60 0x7fc63c19df30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00d650 tx=0x7fc62c003fe0 comp rx=0 tx=0).stop 2024-06-06T04:19:13.703 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.120+0000 7fc641b1f700 1 -- 172.21.15.45:0/3200695526 shutdown_connections 2024-06-06T04:19:13.703 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.120+0000 7fc641b1f700 1 --2- 172.21.15.45:0/3200695526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc63c106c60 0x7fc63c19df30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:13.703 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.120+0000 7fc641b1f700 1 -- 172.21.15.45:0/3200695526 >> 172.21.15.45:0/3200695526 conn(0x7fc63c1021e0 msgr2=0x7fc63c194550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:13.703 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.120+0000 7fc641b1f700 1 -- 172.21.15.45:0/3200695526 shutdown_connections 2024-06-06T04:19:13.703 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:13.120+0000 7fc641b1f700 1 -- 172.21.15.45:0/3200695526 wait complete. 2024-06-06T04:19:13.703 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (2/15)... 2024-06-06T04:19:13.976 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:13 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:13.759+0000 7f1fe34b2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T04:19:13.977 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:13 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:13.860+0000 7f1fe34b2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T04:19:14.476 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:14 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:14.069+0000 7f1fe34b2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T04:19:14.963 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:14 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:14.613+0000 7f1fe34b2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T04:19:14.963 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:14 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:14.709+0000 7f1fe34b2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T04:19:15.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:14 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:14.961+0000 7f1fe34b2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T04:19:15.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:15 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:15.100+0000 7f1fe34b2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T04:19:15.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:15 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:15.199+0000 7f1fe34b2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T04:19:15.632 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:15 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:15.319+0000 7f1fe34b2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T04:19:15.632 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:15 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:15.630+0000 7f1fe34b2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T04:19:16.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:16 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2119203239' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T04:19:16.477 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:16 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:16.284+0000 7f1fe34b2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T04:19:16.716 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-06-06T04:19:16.716 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-06-06T04:19:16.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "c4f6b938-23bb-11ef-bc9b-c7b262605968", 2024-06-06T04:19:16.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T04:19:16.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T04:19:16.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T04:19:16.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T04:19:16.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:16.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T04:19:16.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T04:19:16.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-06-06T04:19:16.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-06-06T04:19:16.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T04:19:16.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T04:19:16.717 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-06-06T04:19:16.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-06-06T04:19:16.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T04:19:16.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T04:19:16.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T04:19:16.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T04:19:16.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:16.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T04:19:16.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T04:19:16.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T04:19:16.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T04:19:16.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T04:19:16.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T04:19:16.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T04:19:16.718 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T04:19:16.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:16.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T04:19:16.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T04:19:16.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T04:19:16.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T04:19:16.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T04:19:16.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T04:19:16.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T04:19:16.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T04:19:16.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T04:19:16.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:16.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T04:19:16.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T04:19:16.719 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T04:19:16.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T04:19:16.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:16.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T04:19:16.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-06T04:19:16.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T04:19:16.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T04:19:16.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T04:19:16.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T04:19:16.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T04:19:16.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-06-06T04:19:16.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T04:19:16.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:16.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T04:19:16.720 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T04:19:16.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T04:19:02.627392+0000", 2024-06-06T04:19:16.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T04:19:16.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:16.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T04:19:16.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-06-06T04:19:16.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.101+0000 7fe913930700 1 Processor -- start 2024-06-06T04:19:16.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.101+0000 7fe913930700 1 -- start start 2024-06-06T04:19:16.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.101+0000 7fe913930700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90c106630 0x7fe90c106a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:16.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.101+0000 7fe913930700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c07be20 con 0x7fe90c106630 2024-06-06T04:19:16.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.101+0000 7fe9116cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90c106630 0x7fe90c106a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:16.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.101+0000 7fe9116cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90c106630 0x7fe90c106a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38460/0 (socket says 172.21.15.45:38460) 2024-06-06T04:19:16.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.101+0000 7fe9116cc700 1 -- 172.21.15.45:0/4228044916 learned_addr learned my addr 172.21.15.45:0/4228044916 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:16.721 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.102+0000 7fe9116cc700 1 -- 172.21.15.45:0/4228044916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90c106f90 con 0x7fe90c106630 2024-06-06T04:19:16.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.102+0000 7fe9116cc700 1 --2- 172.21.15.45:0/4228044916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90c106630 0x7fe90c106a50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe9080098a0 tx=0x7fe908009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b35880a225f93 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:16.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.102+0000 7fe903fff700 1 -- 172.21.15.45:0/4228044916 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe908004160 con 0x7fe90c106630 2024-06-06T04:19:16.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.102+0000 7fe903fff700 1 -- 172.21.15.45:0/4228044916 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe908004300 con 0x7fe90c106630 2024-06-06T04:19:16.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.102+0000 7fe903fff700 1 -- 172.21.15.45:0/4228044916 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe908011a80 con 0x7fe90c106630 2024-06-06T04:19:16.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.103+0000 7fe913930700 1 -- 172.21.15.45:0/4228044916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90c106630 msgr2=0x7fe90c106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:16.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.103+0000 7fe913930700 1 --2- 172.21.15.45:0/4228044916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90c106630 0x7fe90c106a50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe9080098a0 tx=0x7fe908009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:16.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.103+0000 7fe913930700 1 -- 172.21.15.45:0/4228044916 shutdown_connections 2024-06-06T04:19:16.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.103+0000 7fe913930700 1 --2- 172.21.15.45:0/4228044916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90c106630 0x7fe90c106a50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:16.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.103+0000 7fe913930700 1 -- 172.21.15.45:0/4228044916 >> 172.21.15.45:0/4228044916 conn(0x7fe90c1021f0 msgr2=0x7fe90c104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:16.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.103+0000 7fe913930700 1 -- 172.21.15.45:0/4228044916 shutdown_connections 2024-06-06T04:19:16.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.103+0000 7fe913930700 1 -- 172.21.15.45:0/4228044916 wait complete. 2024-06-06T04:19:16.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.103+0000 7fe913930700 1 Processor -- start 2024-06-06T04:19:16.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.104+0000 7fe913930700 1 -- start start 2024-06-06T04:19:16.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.104+0000 7fe913930700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90c106630 0x7fe90c19de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:16.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.104+0000 7fe913930700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c107fc0 con 0x7fe90c106630 2024-06-06T04:19:16.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.104+0000 7fe9116cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90c106630 0x7fe90c19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:16.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.104+0000 7fe9116cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90c106630 0x7fe90c19de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38476/0 (socket says 172.21.15.45:38476) 2024-06-06T04:19:16.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.104+0000 7fe9116cc700 1 -- 172.21.15.45:0/2119203239 learned_addr learned my addr 172.21.15.45:0/2119203239 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:16.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.104+0000 7fe9116cc700 1 -- 172.21.15.45:0/2119203239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90c19e3a0 con 0x7fe90c106630 2024-06-06T04:19:16.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.104+0000 7fe9116cc700 1 --2- 172.21.15.45:0/2119203239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90c106630 0x7fe90c19de60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe908006af0 tx=0x7fe908011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:16.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.104+0000 7fe9027fc700 1 -- 172.21.15.45:0/2119203239 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9080039e0 con 0x7fe90c106630 2024-06-06T04:19:16.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.104+0000 7fe9027fc700 1 -- 172.21.15.45:0/2119203239 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe908003b80 con 0x7fe90c106630 2024-06-06T04:19:16.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.104+0000 7fe913930700 1 -- 172.21.15.45:0/2119203239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90c19e630 con 0x7fe90c106630 2024-06-06T04:19:16.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.105+0000 7fe9027fc700 1 -- 172.21.15.45:0/2119203239 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9080196a0 con 0x7fe90c106630 2024-06-06T04:19:16.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.105+0000 7fe913930700 1 -- 172.21.15.45:0/2119203239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90c1a1320 con 0x7fe90c106630 2024-06-06T04:19:16.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.105+0000 7fe9027fc700 1 -- 172.21.15.45:0/2119203239 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe90802d440 con 0x7fe90c106630 2024-06-06T04:19:16.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.105+0000 7fe9027fc700 1 -- 172.21.15.45:0/2119203239 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe908024770 con 0x7fe90c106630 2024-06-06T04:19:16.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.106+0000 7fe913930700 1 -- 172.21.15.45:0/2119203239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe90c063af0 con 0x7fe90c106630 2024-06-06T04:19:16.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.108+0000 7fe9027fc700 1 -- 172.21.15.45:0/2119203239 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe908043dd0 con 0x7fe90c106630 2024-06-06T04:19:16.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.182+0000 7fe913930700 1 -- 172.21.15.45:0/2119203239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe90c02dea0 con 0x7fe90c106630 2024-06-06T04:19:16.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.182+0000 7fe9027fc700 1 -- 172.21.15.45:0/2119203239 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe908032020 con 0x7fe90c106630 2024-06-06T04:19:16.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.183+0000 7fe913930700 1 -- 172.21.15.45:0/2119203239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90c106630 msgr2=0x7fe90c19de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:16.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.183+0000 7fe913930700 1 --2- 172.21.15.45:0/2119203239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90c106630 0x7fe90c19de60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe908006af0 tx=0x7fe908011c40 comp rx=0 tx=0).stop 2024-06-06T04:19:16.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.183+0000 7fe913930700 1 -- 172.21.15.45:0/2119203239 shutdown_connections 2024-06-06T04:19:16.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.183+0000 7fe913930700 1 --2- 172.21.15.45:0/2119203239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90c106630 0x7fe90c19de60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:16.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.183+0000 7fe913930700 1 -- 172.21.15.45:0/2119203239 >> 172.21.15.45:0/2119203239 conn(0x7fe90c1021f0 msgr2=0x7fe90c194a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:16.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.184+0000 7fe913930700 1 -- 172.21.15.45:0/2119203239 shutdown_connections 2024-06-06T04:19:16.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:16.184+0000 7fe913930700 1 -- 172.21.15.45:0/2119203239 wait complete. 2024-06-06T04:19:16.724 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (3/15)... 2024-06-06T04:19:16.810 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:16 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:16.511+0000 7f1fe34b2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T04:19:17.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:17 smithi045 ceph-mon[32299]: Activating manager daemon x 2024-06-06T04:19:17.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:17 smithi045 ceph-mon[32299]: mgrmap e2: x(active, starting, since 0.00308457s) 2024-06-06T04:19:17.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:17 smithi045 ceph-mon[32299]: from='mgr.14100 172.21.15.45:0/2772242109' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T04:19:17.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:17 smithi045 ceph-mon[32299]: from='mgr.14100 172.21.15.45:0/2772242109' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T04:19:17.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:17 smithi045 ceph-mon[32299]: from='mgr.14100 172.21.15.45:0/2772242109' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T04:19:17.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:17 smithi045 ceph-mon[32299]: from='mgr.14100 172.21.15.45:0/2772242109' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T04:19:17.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:17 smithi045 ceph-mon[32299]: from='mgr.14100 172.21.15.45:0/2772242109' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T04:19:17.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:17 smithi045 ceph-mon[32299]: Manager daemon x is now available 2024-06-06T04:19:17.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:17 smithi045 ceph-mon[32299]: from='mgr.14100 172.21.15.45:0/2772242109' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T04:19:17.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:17 smithi045 ceph-mon[32299]: from='mgr.14100 172.21.15.45:0/2772242109' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T04:19:17.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:17 smithi045 ceph-mon[32299]: from='mgr.14100 172.21.15.45:0/2772242109' entity='mgr.x' 2024-06-06T04:19:17.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:17 smithi045 ceph-mon[32299]: from='mgr.14100 172.21.15.45:0/2772242109' entity='mgr.x' 2024-06-06T04:19:17.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:17 smithi045 ceph-mon[32299]: from='mgr.14100 172.21.15.45:0/2772242109' entity='mgr.x' 2024-06-06T04:19:18.842 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:18 smithi045 ceph-mon[32299]: mgrmap e3: x(active, since 1.00725s) 2024-06-06T04:19:19.832 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:19 smithi045 ceph-mon[32299]: mgrmap e4: x(active, since 2s) 2024-06-06T04:19:22.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:22 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/187029846' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T04:19:27.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-06-06T04:19:27.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-06-06T04:19:27.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "c4f6b938-23bb-11ef-bc9b-c7b262605968", 2024-06-06T04:19:27.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T04:19:27.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T04:19:27.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T04:19:27.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T04:19:27.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:27.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T04:19:27.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T04:19:27.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-06-06T04:19:27.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-06-06T04:19:27.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T04:19:27.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T04:19:27.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-06-06T04:19:27.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-06-06T04:19:27.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T04:19:27.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T04:19:27.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T04:19:27.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T04:19:27.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:27.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T04:19:27.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T04:19:27.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T04:19:27.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T04:19:27.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T04:19:27.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T04:19:27.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T04:19:27.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T04:19:27.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:27.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T04:19:27.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T04:19:27.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T04:19:27.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T04:19:27.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T04:19:27.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T04:19:27.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T04:19:27.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T04:19:27.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T04:19:27.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:27.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T04:19:27.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T04:19:27.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T04:19:27.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T04:19:27.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:27.915 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T04:19:27.915 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-06T04:19:27.915 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T04:19:27.915 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T04:19:27.915 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T04:19:27.915 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T04:19:27.915 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T04:19:27.915 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-06-06T04:19:27.915 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T04:19:27.915 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:27.915 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T04:19:27.915 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T04:19:27.916 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T04:19:02.627392+0000", 2024-06-06T04:19:27.916 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T04:19:27.916 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-06-06T04:19:27.916 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T04:19:27.916 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-06-06T04:19:27.916 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.778+0000 7f4073c1f700 1 Processor -- start 2024-06-06T04:19:27.916 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.778+0000 7f4073c1f700 1 -- start start 2024-06-06T04:19:27.916 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.778+0000 7f4073c1f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f406c106c50 0x7f406c107070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:27.916 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.778+0000 7f4073c1f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f406c107640 con 0x7f406c106c50 2024-06-06T04:19:27.916 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.778+0000 7f40719bb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f406c106c50 0x7f406c107070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:27.916 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.779+0000 7f40719bb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f406c106c50 0x7f406c107070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43634/0 (socket says 172.21.15.45:43634) 2024-06-06T04:19:27.916 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.779+0000 7f40719bb700 1 -- 172.21.15.45:0/2211023278 learned_addr learned my addr 172.21.15.45:0/2211023278 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:27.916 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.779+0000 7f40719bb700 1 -- 172.21.15.45:0/2211023278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c107e50 con 0x7f406c106c50 2024-06-06T04:19:27.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.779+0000 7f40719bb700 1 --2- 172.21.15.45:0/2211023278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f406c106c50 0x7f406c107070 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f40680098a0 tx=0x7f4068009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aadc88c03dcefab server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:27.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.779+0000 7f40709b9700 1 -- 172.21.15.45:0/2211023278 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4068004160 con 0x7f406c106c50 2024-06-06T04:19:27.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.779+0000 7f40709b9700 1 -- 172.21.15.45:0/2211023278 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4068004300 con 0x7f406c106c50 2024-06-06T04:19:27.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.779+0000 7f40709b9700 1 -- 172.21.15.45:0/2211023278 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4068011a80 con 0x7f406c106c50 2024-06-06T04:19:27.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.780+0000 7f4073c1f700 1 -- 172.21.15.45:0/2211023278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f406c106c50 msgr2=0x7f406c107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:27.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.780+0000 7f4073c1f700 1 --2- 172.21.15.45:0/2211023278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f406c106c50 0x7f406c107070 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f40680098a0 tx=0x7f4068009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:27.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.780+0000 7f4073c1f700 1 -- 172.21.15.45:0/2211023278 shutdown_connections 2024-06-06T04:19:27.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.780+0000 7f4073c1f700 1 --2- 172.21.15.45:0/2211023278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f406c106c50 0x7f406c107070 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:27.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.780+0000 7f4073c1f700 1 -- 172.21.15.45:0/2211023278 >> 172.21.15.45:0/2211023278 conn(0x7f406c1021f0 msgr2=0x7f406c104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:27.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.780+0000 7f4073c1f700 1 -- 172.21.15.45:0/2211023278 shutdown_connections 2024-06-06T04:19:27.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.780+0000 7f4073c1f700 1 -- 172.21.15.45:0/2211023278 wait complete. 2024-06-06T04:19:27.917 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.780+0000 7f4073c1f700 1 Processor -- start 2024-06-06T04:19:27.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.780+0000 7f4073c1f700 1 -- start start 2024-06-06T04:19:27.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.781+0000 7f4073c1f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f406c106c50 0x7f406c191dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:27.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.781+0000 7f4073c1f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f406c10b200 con 0x7f406c106c50 2024-06-06T04:19:27.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.781+0000 7f40719bb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f406c106c50 0x7f406c191dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:27.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.781+0000 7f40719bb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f406c106c50 0x7f406c191dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43640/0 (socket says 172.21.15.45:43640) 2024-06-06T04:19:27.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.781+0000 7f40719bb700 1 -- 172.21.15.45:0/187029846 learned_addr learned my addr 172.21.15.45:0/187029846 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:27.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.781+0000 7f40719bb700 1 -- 172.21.15.45:0/187029846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c192300 con 0x7f406c106c50 2024-06-06T04:19:27.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.781+0000 7f40719bb700 1 --2- 172.21.15.45:0/187029846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f406c106c50 0x7f406c191dc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4068006af0 tx=0x7f4068011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:27.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.781+0000 7f4062ffd700 1 -- 172.21.15.45:0/187029846 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40680039e0 con 0x7f406c106c50 2024-06-06T04:19:27.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.781+0000 7f4062ffd700 1 -- 172.21.15.45:0/187029846 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4068003b80 con 0x7f406c106c50 2024-06-06T04:19:27.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.781+0000 7f4073c1f700 1 -- 172.21.15.45:0/187029846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406c1904f0 con 0x7f406c106c50 2024-06-06T04:19:27.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.781+0000 7f4062ffd700 1 -- 172.21.15.45:0/187029846 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40680196a0 con 0x7f406c106c50 2024-06-06T04:19:27.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.781+0000 7f4073c1f700 1 -- 172.21.15.45:0/187029846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406c190a30 con 0x7f406c106c50 2024-06-06T04:19:27.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.782+0000 7f4062ffd700 1 -- 172.21.15.45:0/187029846 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f406801a430 con 0x7f406c106c50 2024-06-06T04:19:27.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.782+0000 7f4073c1f700 1 -- 172.21.15.45:0/187029846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406c063af0 con 0x7f406c106c50 2024-06-06T04:19:27.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.783+0000 7f4062ffd700 1 --2- 172.21.15.45:0/187029846 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f4058039000 0x7f405803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:27.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.783+0000 7f4062ffd700 1 -- 172.21.15.45:0/187029846 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f406804d520 con 0x7f406c106c50 2024-06-06T04:19:27.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.783+0000 7f40711ba700 1 --2- 172.21.15.45:0/187029846 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f4058039000 0x7f405803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:27.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.783+0000 7f40711ba700 1 --2- 172.21.15.45:0/187029846 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f4058039000 0x7f405803b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f405c006fd0 tx=0x7f405c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:27.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:21.788+0000 7f4062ffd700 1 -- 172.21.15.45:0/187029846 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4068023e00 con 0x7f406c106c50 2024-06-06T04:19:27.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:22.086+0000 7f4073c1f700 1 -- 172.21.15.45:0/187029846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f406c195b40 con 0x7f406c106c50 2024-06-06T04:19:27.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:22.086+0000 7f4062ffd700 1 -- 172.21.15.45:0/187029846 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f406804cf40 con 0x7f406c106c50 2024-06-06T04:19:27.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:22.088+0000 7f4073c1f700 1 -- 172.21.15.45:0/187029846 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f4058039000 msgr2=0x7f405803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:27.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:22.088+0000 7f4073c1f700 1 --2- 172.21.15.45:0/187029846 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f4058039000 0x7f405803b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f405c006fd0 tx=0x7f405c006e40 comp rx=0 tx=0).stop 2024-06-06T04:19:27.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:22.088+0000 7f4073c1f700 1 -- 172.21.15.45:0/187029846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f406c106c50 msgr2=0x7f406c191dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:27.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:22.088+0000 7f4073c1f700 1 --2- 172.21.15.45:0/187029846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f406c106c50 0x7f406c191dc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4068006af0 tx=0x7f4068011c40 comp rx=0 tx=0).stop 2024-06-06T04:19:27.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:22.088+0000 7f4073c1f700 1 -- 172.21.15.45:0/187029846 shutdown_connections 2024-06-06T04:19:27.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:22.088+0000 7f4073c1f700 1 --2- 172.21.15.45:0/187029846 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f4058039000 0x7f405803b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:27.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:22.088+0000 7f4073c1f700 1 --2- 172.21.15.45:0/187029846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f406c106c50 0x7f406c191dc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:27.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:22.088+0000 7f4073c1f700 1 -- 172.21.15.45:0/187029846 >> 172.21.15.45:0/187029846 conn(0x7f406c1021f0 msgr2=0x7f406c1030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:27.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:22.089+0000 7f4073c1f700 1 -- 172.21.15.45:0/187029846 shutdown_connections 2024-06-06T04:19:27.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:22.089+0000 7f4073c1f700 1 -- 172.21.15.45:0/187029846 wait complete. 2024-06-06T04:19:27.920 INFO:teuthology.orchestra.run.smithi045.stdout:mgr is available 2024-06-06T04:19:28.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:28 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1605322962' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-06T04:19:29.030 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-06-06T04:19:29.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-06-06T04:19:29.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:19:29.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-06T04:19:29.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-06T04:19:29.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-06T04:19:29.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-06T04:19:29.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-06-06T04:19:29.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-06T04:19:29.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-06T04:19:29.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-06-06T04:19:29.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-06-06T04:19:29.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-06T04:19:29.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-06T04:19:29.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-06T04:19:29.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-06T04:19:29.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-06T04:19:29.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.330+0000 7fbf076e3700 1 Processor -- start 2024-06-06T04:19:29.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.331+0000 7fbf076e3700 1 -- start start 2024-06-06T04:19:29.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.331+0000 7fbf076e3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf00107db0 0x7fbf001081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:29.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.331+0000 7fbf076e3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf001087a0 con 0x7fbf00107db0 2024-06-06T04:19:29.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.331+0000 7fbf0547f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf00107db0 0x7fbf001081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:29.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.331+0000 7fbf0547f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf00107db0 0x7fbf001081d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35140/0 (socket says 172.21.15.45:35140) 2024-06-06T04:19:29.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.331+0000 7fbf0547f700 1 -- 172.21.15.45:0/3728418301 learned_addr learned my addr 172.21.15.45:0/3728418301 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:29.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.331+0000 7fbf0547f700 1 -- 172.21.15.45:0/3728418301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf00108f60 con 0x7fbf00107db0 2024-06-06T04:19:29.038 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.332+0000 7fbf0547f700 1 --2- 172.21.15.45:0/3728418301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf00107db0 0x7fbf001081d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0098a0 tx=0x7fbefc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402ef0b712c08255 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:29.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.332+0000 7fbef7fff700 1 -- 172.21.15.45:0/3728418301 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbefc004160 con 0x7fbf00107db0 2024-06-06T04:19:29.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.332+0000 7fbef7fff700 1 -- 172.21.15.45:0/3728418301 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbefc004300 con 0x7fbf00107db0 2024-06-06T04:19:29.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.332+0000 7fbef7fff700 1 -- 172.21.15.45:0/3728418301 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbefc011a80 con 0x7fbf00107db0 2024-06-06T04:19:29.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.332+0000 7fbf076e3700 1 -- 172.21.15.45:0/3728418301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf00107db0 msgr2=0x7fbf001081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:29.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.332+0000 7fbf076e3700 1 --2- 172.21.15.45:0/3728418301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf00107db0 0x7fbf001081d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0098a0 tx=0x7fbefc009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:29.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.333+0000 7fbf076e3700 1 -- 172.21.15.45:0/3728418301 shutdown_connections 2024-06-06T04:19:29.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.333+0000 7fbf076e3700 1 --2- 172.21.15.45:0/3728418301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf00107db0 0x7fbf001081d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:29.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.333+0000 7fbf076e3700 1 -- 172.21.15.45:0/3728418301 >> 172.21.15.45:0/3728418301 conn(0x7fbf00103330 msgr2=0x7fbf00105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:29.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.333+0000 7fbf076e3700 1 -- 172.21.15.45:0/3728418301 shutdown_connections 2024-06-06T04:19:29.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.333+0000 7fbf076e3700 1 -- 172.21.15.45:0/3728418301 wait complete. 2024-06-06T04:19:29.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.333+0000 7fbf076e3700 1 Processor -- start 2024-06-06T04:19:29.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.333+0000 7fbf076e3700 1 -- start start 2024-06-06T04:19:29.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.334+0000 7fbf076e3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf00107db0 0x7fbf0019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:29.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.334+0000 7fbf076e3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0010c310 con 0x7fbf00107db0 2024-06-06T04:19:29.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.334+0000 7fbf0547f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf00107db0 0x7fbf0019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:29.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.334+0000 7fbf0547f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf00107db0 0x7fbf0019bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35142/0 (socket says 172.21.15.45:35142) 2024-06-06T04:19:29.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.334+0000 7fbf0547f700 1 -- 172.21.15.45:0/1605322962 learned_addr learned my addr 172.21.15.45:0/1605322962 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:29.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.334+0000 7fbf0547f700 1 -- 172.21.15.45:0/1605322962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0019c1f0 con 0x7fbf00107db0 2024-06-06T04:19:29.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.334+0000 7fbf0547f700 1 --2- 172.21.15.45:0/1605322962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf00107db0 0x7fbf0019bcb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbefc006af0 tx=0x7fbefc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:29.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.334+0000 7fbef67fc700 1 -- 172.21.15.45:0/1605322962 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbefc0039e0 con 0x7fbf00107db0 2024-06-06T04:19:29.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.334+0000 7fbef67fc700 1 -- 172.21.15.45:0/1605322962 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbefc003b80 con 0x7fbf00107db0 2024-06-06T04:19:29.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.334+0000 7fbf076e3700 1 -- 172.21.15.45:0/1605322962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0019c480 con 0x7fbf00107db0 2024-06-06T04:19:29.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.334+0000 7fbef67fc700 1 -- 172.21.15.45:0/1605322962 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbefc0196a0 con 0x7fbf00107db0 2024-06-06T04:19:29.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.334+0000 7fbf076e3700 1 -- 172.21.15.45:0/1605322962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf0019f170 con 0x7fbf00107db0 2024-06-06T04:19:29.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.335+0000 7fbef67fc700 1 -- 172.21.15.45:0/1605322962 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fbefc019950 con 0x7fbf00107db0 2024-06-06T04:19:29.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.335+0000 7fbf076e3700 1 -- 172.21.15.45:0/1605322962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf00063af0 con 0x7fbf00107db0 2024-06-06T04:19:29.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.335+0000 7fbef67fc700 1 --2- 172.21.15.45:0/1605322962 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7fbeec039000 0x7fbeec03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:29.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.335+0000 7fbef67fc700 1 -- 172.21.15.45:0/1605322962 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbefc04d6a0 con 0x7fbf00107db0 2024-06-06T04:19:29.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.336+0000 7fbf04c7e700 1 --2- 172.21.15.45:0/1605322962 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7fbeec039000 0x7fbeec03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:29.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.336+0000 7fbf04c7e700 1 --2- 172.21.15.45:0/1605322962 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7fbeec039000 0x7fbeec03b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbef0006fd0 tx=0x7fbef0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:29.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.341+0000 7fbef67fc700 1 -- 172.21.15.45:0/1605322962 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbefc023e20 con 0x7fbf00107db0 2024-06-06T04:19:29.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.545+0000 7fbf076e3700 1 -- 172.21.15.45:0/1605322962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fbf0019f340 con 0x7fbf00107db0 2024-06-06T04:19:29.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.546+0000 7fbef67fc700 1 -- 172.21.15.45:0/1605322962 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fbf0019f340 con 0x7fbf00107db0 2024-06-06T04:19:29.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.549+0000 7fbf076e3700 1 -- 172.21.15.45:0/1605322962 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7fbeec039000 msgr2=0x7fbeec03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:29.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.549+0000 7fbf076e3700 1 --2- 172.21.15.45:0/1605322962 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7fbeec039000 0x7fbeec03b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbef0006fd0 tx=0x7fbef0006e40 comp rx=0 tx=0).stop 2024-06-06T04:19:29.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.549+0000 7fbf076e3700 1 -- 172.21.15.45:0/1605322962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf00107db0 msgr2=0x7fbf0019bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:29.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.549+0000 7fbf076e3700 1 --2- 172.21.15.45:0/1605322962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf00107db0 0x7fbf0019bcb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbefc006af0 tx=0x7fbefc011c40 comp rx=0 tx=0).stop 2024-06-06T04:19:29.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.549+0000 7fbf076e3700 1 -- 172.21.15.45:0/1605322962 shutdown_connections 2024-06-06T04:19:29.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.549+0000 7fbf076e3700 1 --2- 172.21.15.45:0/1605322962 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7fbeec039000 0x7fbeec03b4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:29.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.549+0000 7fbf076e3700 1 --2- 172.21.15.45:0/1605322962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf00107db0 0x7fbf0019bcb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:29.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.549+0000 7fbf076e3700 1 -- 172.21.15.45:0/1605322962 >> 172.21.15.45:0/1605322962 conn(0x7fbf00103330 msgr2=0x7fbf00103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:29.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.549+0000 7fbf076e3700 1 -- 172.21.15.45:0/1605322962 shutdown_connections 2024-06-06T04:19:29.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:28.549+0000 7fbf076e3700 1 -- 172.21.15.45:0/1605322962 wait complete. 2024-06-06T04:19:29.042 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling cephadm module... 2024-06-06T04:19:30.109 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:29 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2205917048' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-06T04:19:30.109 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:29 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: ignoring --setuser ceph since I am not root 2024-06-06T04:19:30.110 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:29 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: ignoring --setgroup ceph since I am not root 2024-06-06T04:19:30.110 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:29 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:29.975+0000 7fd83271d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T04:19:30.401 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:30 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:30.172+0000 7fd83271d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T04:19:30.401 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:30 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:30.280+0000 7fd83271d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T04:19:30.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.470+0000 7fccd8b43700 1 Processor -- start 2024-06-06T04:19:30.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.470+0000 7fccd8b43700 1 -- start start 2024-06-06T04:19:30.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.470+0000 7fccd8b43700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd4106c70 0x7fccd4107090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:30.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.470+0000 7fccd8b43700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fccd4107660 con 0x7fccd4106c70 2024-06-06T04:19:30.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.470+0000 7fccd259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd4106c70 0x7fccd4107090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:30.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.471+0000 7fccd259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd4106c70 0x7fccd4107090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35148/0 (socket says 172.21.15.45:35148) 2024-06-06T04:19:30.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.471+0000 7fccd259c700 1 -- 172.21.15.45:0/1526589908 learned_addr learned my addr 172.21.15.45:0/1526589908 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:30.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.471+0000 7fccd259c700 1 -- 172.21.15.45:0/1526589908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd4107e70 con 0x7fccd4106c70 2024-06-06T04:19:30.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.471+0000 7fccd259c700 1 --2- 172.21.15.45:0/1526589908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd4106c70 0x7fccd4107090 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fccc40098a0 tx=0x7fccc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd48da647ed35de6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:30.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.471+0000 7fccd159a700 1 -- 172.21.15.45:0/1526589908 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccc4004160 con 0x7fccd4106c70 2024-06-06T04:19:30.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.471+0000 7fccd159a700 1 -- 172.21.15.45:0/1526589908 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fccc4004300 con 0x7fccd4106c70 2024-06-06T04:19:30.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.471+0000 7fccd159a700 1 -- 172.21.15.45:0/1526589908 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccc4011a80 con 0x7fccd4106c70 2024-06-06T04:19:30.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.472+0000 7fccd8b43700 1 -- 172.21.15.45:0/1526589908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd4106c70 msgr2=0x7fccd4107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:30.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.472+0000 7fccd8b43700 1 --2- 172.21.15.45:0/1526589908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd4106c70 0x7fccd4107090 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fccc40098a0 tx=0x7fccc4009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:30.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.472+0000 7fccd8b43700 1 -- 172.21.15.45:0/1526589908 shutdown_connections 2024-06-06T04:19:30.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.472+0000 7fccd8b43700 1 --2- 172.21.15.45:0/1526589908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd4106c70 0x7fccd4107090 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:30.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.472+0000 7fccd8b43700 1 -- 172.21.15.45:0/1526589908 >> 172.21.15.45:0/1526589908 conn(0x7fccd4102210 msgr2=0x7fccd4104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:30.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.472+0000 7fccd8b43700 1 -- 172.21.15.45:0/1526589908 shutdown_connections 2024-06-06T04:19:30.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.472+0000 7fccd8b43700 1 -- 172.21.15.45:0/1526589908 wait complete. 2024-06-06T04:19:30.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.473+0000 7fccd8b43700 1 Processor -- start 2024-06-06T04:19:30.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.473+0000 7fccd8b43700 1 -- start start 2024-06-06T04:19:30.410 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.473+0000 7fccd8b43700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd4106c70 0x7fccd4191de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:30.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.473+0000 7fccd8b43700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fccd410b220 con 0x7fccd4106c70 2024-06-06T04:19:30.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.473+0000 7fccd259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd4106c70 0x7fccd4191de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:30.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.473+0000 7fccd259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd4106c70 0x7fccd4191de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35152/0 (socket says 172.21.15.45:35152) 2024-06-06T04:19:30.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.473+0000 7fccd259c700 1 -- 172.21.15.45:0/2205917048 learned_addr learned my addr 172.21.15.45:0/2205917048 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:30.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.473+0000 7fccd259c700 1 -- 172.21.15.45:0/2205917048 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd4192320 con 0x7fccd4106c70 2024-06-06T04:19:30.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.474+0000 7fccd259c700 1 --2- 172.21.15.45:0/2205917048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd4106c70 0x7fccd4191de0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fccc4000c00 tx=0x7fccc4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:30.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.474+0000 7fcccb7fe700 1 -- 172.21.15.45:0/2205917048 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccc40039f0 con 0x7fccd4106c70 2024-06-06T04:19:30.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.474+0000 7fcccb7fe700 1 -- 172.21.15.45:0/2205917048 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fccc4003b90 con 0x7fccd4106c70 2024-06-06T04:19:30.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.474+0000 7fccd8b43700 1 -- 172.21.15.45:0/2205917048 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd4190510 con 0x7fccd4106c70 2024-06-06T04:19:30.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.474+0000 7fcccb7fe700 1 -- 172.21.15.45:0/2205917048 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccc401a740 con 0x7fccd4106c70 2024-06-06T04:19:30.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.474+0000 7fccd8b43700 1 -- 172.21.15.45:0/2205917048 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd4190a50 con 0x7fccd4106c70 2024-06-06T04:19:30.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.475+0000 7fcccb7fe700 1 -- 172.21.15.45:0/2205917048 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fccc401a9f0 con 0x7fccd4106c70 2024-06-06T04:19:30.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.475+0000 7fccd8b43700 1 -- 172.21.15.45:0/2205917048 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd4063af0 con 0x7fccd4106c70 2024-06-06T04:19:30.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.475+0000 7fcccb7fe700 1 --2- 172.21.15.45:0/2205917048 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7fccc0039050 0x7fccc003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:30.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.475+0000 7fcccb7fe700 1 -- 172.21.15.45:0/2205917048 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fccc404d5a0 con 0x7fccd4106c70 2024-06-06T04:19:30.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.475+0000 7fccd1d9b700 1 --2- 172.21.15.45:0/2205917048 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7fccc0039050 0x7fccc003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:30.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.476+0000 7fccd1d9b700 1 --2- 172.21.15.45:0/2205917048 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7fccc0039050 0x7fccc003b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fccbc006fd0 tx=0x7fccbc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:30.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.480+0000 7fcccb7fe700 1 -- 172.21.15.45:0/2205917048 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fccc400fdd0 con 0x7fccd4106c70 2024-06-06T04:19:30.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.746+0000 7fccd8b43700 1 -- 172.21.15.45:0/2205917048 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fccd4195b80 con 0x7fccd4106c70 2024-06-06T04:19:30.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.798+0000 7fcccb7fe700 1 -- 172.21.15.45:0/2205917048 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fccc404cfc0 con 0x7fccd4106c70 2024-06-06T04:19:30.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.799+0000 7fcccb7fe700 1 -- 172.21.15.45:0/2205917048 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fccc402d910 con 0x7fccd4106c70 2024-06-06T04:19:30.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.801+0000 7fccd8b43700 1 -- 172.21.15.45:0/2205917048 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7fccc0039050 msgr2=0x7fccc003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:30.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.801+0000 7fccd8b43700 1 --2- 172.21.15.45:0/2205917048 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7fccc0039050 0x7fccc003b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fccbc006fd0 tx=0x7fccbc006e40 comp rx=0 tx=0).stop 2024-06-06T04:19:30.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.801+0000 7fccd8b43700 1 -- 172.21.15.45:0/2205917048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd4106c70 msgr2=0x7fccd4191de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:30.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.801+0000 7fccd8b43700 1 --2- 172.21.15.45:0/2205917048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd4106c70 0x7fccd4191de0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fccc4000c00 tx=0x7fccc4011b00 comp rx=0 tx=0).stop 2024-06-06T04:19:30.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.801+0000 7fccd8b43700 1 -- 172.21.15.45:0/2205917048 shutdown_connections 2024-06-06T04:19:30.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.801+0000 7fccd8b43700 1 --2- 172.21.15.45:0/2205917048 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7fccc0039050 0x7fccc003b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:30.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.801+0000 7fccd8b43700 1 --2- 172.21.15.45:0/2205917048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd4106c70 0x7fccd4191de0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:30.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.801+0000 7fccd8b43700 1 -- 172.21.15.45:0/2205917048 >> 172.21.15.45:0/2205917048 conn(0x7fccd4102210 msgr2=0x7fccd4103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:30.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.801+0000 7fccd8b43700 1 -- 172.21.15.45:0/2205917048 shutdown_connections 2024-06-06T04:19:30.413 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:29.801+0000 7fccd8b43700 1 -- 172.21.15.45:0/2205917048 wait complete. 2024-06-06T04:19:30.701 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:30 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:30.610+0000 7fd83271d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T04:19:30.701 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:30 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:30.700+0000 7fd83271d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T04:19:30.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:30 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2205917048' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-06T04:19:30.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:30 smithi045 ceph-mon[32299]: mgrmap e5: x(active, since 13s) 2024-06-06T04:19:31.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:31 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3911396938' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-06T04:19:32.258 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:31 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:31.981+0000 7fd83271d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T04:19:32.258 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:32 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:32.161+0000 7fd83271d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T04:19:32.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-06-06T04:19:32.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-06T04:19:32.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-06T04:19:32.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-06T04:19:32.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-06T04:19:32.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-06-06T04:19:32.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.373+0000 7f1b55293700 1 Processor -- start 2024-06-06T04:19:32.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.373+0000 7f1b55293700 1 -- start start 2024-06-06T04:19:32.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.373+0000 7f1b55293700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b50108830 0x7f1b5010ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:32.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.373+0000 7f1b55293700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5007be60 con 0x7f1b50108830 2024-06-06T04:19:32.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.373+0000 7f1b4effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b50108830 0x7f1b5010ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:32.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.374+0000 7f1b4effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b50108830 0x7f1b5010ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35174/0 (socket says 172.21.15.45:35174) 2024-06-06T04:19:32.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.374+0000 7f1b4effd700 1 -- 172.21.15.45:0/3231088193 learned_addr learned my addr 172.21.15.45:0/3231088193 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:32.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.374+0000 7f1b4effd700 1 -- 172.21.15.45:0/3231088193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5010b160 con 0x7f1b50108830 2024-06-06T04:19:32.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.374+0000 7f1b4effd700 1 --2- 172.21.15.45:0/3231088193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b50108830 0x7f1b5010ac20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1b400098a0 tx=0x7f1b40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0fd39da1dd620f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:32.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.374+0000 7f1b4dffb700 1 -- 172.21.15.45:0/3231088193 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b40004160 con 0x7f1b50108830 2024-06-06T04:19:32.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.374+0000 7f1b4dffb700 1 -- 172.21.15.45:0/3231088193 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1b40004300 con 0x7f1b50108830 2024-06-06T04:19:32.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.374+0000 7f1b4dffb700 1 -- 172.21.15.45:0/3231088193 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b40011a80 con 0x7f1b50108830 2024-06-06T04:19:32.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.375+0000 7f1b55293700 1 -- 172.21.15.45:0/3231088193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b50108830 msgr2=0x7f1b5010ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:32.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.375+0000 7f1b55293700 1 --2- 172.21.15.45:0/3231088193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b50108830 0x7f1b5010ac20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1b400098a0 tx=0x7f1b40009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:32.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.375+0000 7f1b55293700 1 -- 172.21.15.45:0/3231088193 shutdown_connections 2024-06-06T04:19:32.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.375+0000 7f1b55293700 1 --2- 172.21.15.45:0/3231088193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b50108830 0x7f1b5010ac20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:32.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.375+0000 7f1b55293700 1 -- 172.21.15.45:0/3231088193 >> 172.21.15.45:0/3231088193 conn(0x7f1b50102240 msgr2=0x7f1b50104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:32.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.375+0000 7f1b55293700 1 -- 172.21.15.45:0/3231088193 shutdown_connections 2024-06-06T04:19:32.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.375+0000 7f1b55293700 1 -- 172.21.15.45:0/3231088193 wait complete. 2024-06-06T04:19:32.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.376+0000 7f1b55293700 1 Processor -- start 2024-06-06T04:19:32.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.376+0000 7f1b55293700 1 -- start start 2024-06-06T04:19:32.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.376+0000 7f1b55293700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b50108830 0x7f1b5019bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:32.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.376+0000 7f1b55293700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5010c120 con 0x7f1b50108830 2024-06-06T04:19:32.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.376+0000 7f1b4effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b50108830 0x7f1b5019bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:32.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.376+0000 7f1b4effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b50108830 0x7f1b5019bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35176/0 (socket says 172.21.15.45:35176) 2024-06-06T04:19:32.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.376+0000 7f1b4effd700 1 -- 172.21.15.45:0/3911396938 learned_addr learned my addr 172.21.15.45:0/3911396938 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:32.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.377+0000 7f1b4effd700 1 -- 172.21.15.45:0/3911396938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5019c1b0 con 0x7f1b50108830 2024-06-06T04:19:32.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.377+0000 7f1b4effd700 1 --2- 172.21.15.45:0/3911396938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b50108830 0x7f1b5019bc70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1b40006af0 tx=0x7f1b40011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:32.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.377+0000 7f1b37fff700 1 -- 172.21.15.45:0/3911396938 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b400039e0 con 0x7f1b50108830 2024-06-06T04:19:32.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.377+0000 7f1b37fff700 1 -- 172.21.15.45:0/3911396938 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1b40003b80 con 0x7f1b50108830 2024-06-06T04:19:32.269 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.377+0000 7f1b55293700 1 -- 172.21.15.45:0/3911396938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5019c440 con 0x7f1b50108830 2024-06-06T04:19:32.269 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.377+0000 7f1b37fff700 1 -- 172.21.15.45:0/3911396938 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b400196a0 con 0x7f1b50108830 2024-06-06T04:19:32.269 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.377+0000 7f1b55293700 1 -- 172.21.15.45:0/3911396938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5019f130 con 0x7f1b50108830 2024-06-06T04:19:32.269 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.378+0000 7f1b37fff700 1 -- 172.21.15.45:0/3911396938 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f1b400199a0 con 0x7f1b50108830 2024-06-06T04:19:32.269 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.378+0000 7f1b55293700 1 -- 172.21.15.45:0/3911396938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b50063af0 con 0x7f1b50108830 2024-06-06T04:19:32.269 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.378+0000 7f1b37fff700 1 --2- 172.21.15.45:0/3911396938 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f1b38039000 0x7f1b3803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:32.269 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.378+0000 7f1b37fff700 1 -- 172.21.15.45:0/3911396938 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1b4004d660 con 0x7f1b50108830 2024-06-06T04:19:32.269 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.378+0000 7f1b4e7fc700 1 -- 172.21.15.45:0/3911396938 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f1b38039000 msgr2=0x7f1b3803b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1562511056 2024-06-06T04:19:32.269 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.378+0000 7f1b4e7fc700 1 --2- 172.21.15.45:0/3911396938 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f1b38039000 0x7f1b3803b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T04:19:32.269 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.383+0000 7f1b37fff700 1 -- 172.21.15.45:0/3911396938 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1b40016dd0 con 0x7f1b50108830 2024-06-06T04:19:32.269 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.579+0000 7f1b4e7fc700 1 -- 172.21.15.45:0/3911396938 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f1b38039000 msgr2=0x7f1b3803b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1562511056 2024-06-06T04:19:32.269 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.579+0000 7f1b4e7fc700 1 --2- 172.21.15.45:0/3911396938 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f1b38039000 0x7f1b3803b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T04:19:32.269 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.654+0000 7f1b55293700 1 -- 172.21.15.45:0/3911396938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1b5019f300 con 0x7f1b50108830 2024-06-06T04:19:32.269 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.654+0000 7f1b37fff700 1 -- 172.21.15.45:0/3911396938 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f1b5019f300 con 0x7f1b50108830 2024-06-06T04:19:32.270 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.656+0000 7f1b55293700 1 -- 172.21.15.45:0/3911396938 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f1b38039000 msgr2=0x7f1b3803b4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T04:19:32.270 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.656+0000 7f1b55293700 1 --2- 172.21.15.45:0/3911396938 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f1b38039000 0x7f1b3803b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:32.270 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.656+0000 7f1b55293700 1 -- 172.21.15.45:0/3911396938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b50108830 msgr2=0x7f1b5019bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:32.270 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.656+0000 7f1b55293700 1 --2- 172.21.15.45:0/3911396938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b50108830 0x7f1b5019bc70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1b40006af0 tx=0x7f1b40011c40 comp rx=0 tx=0).stop 2024-06-06T04:19:32.270 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.657+0000 7f1b55293700 1 -- 172.21.15.45:0/3911396938 shutdown_connections 2024-06-06T04:19:32.270 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.657+0000 7f1b55293700 1 --2- 172.21.15.45:0/3911396938 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f1b38039000 0x7f1b3803b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:32.270 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.657+0000 7f1b55293700 1 --2- 172.21.15.45:0/3911396938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b50108830 0x7f1b5019bc70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:32.270 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.657+0000 7f1b55293700 1 -- 172.21.15.45:0/3911396938 >> 172.21.15.45:0/3911396938 conn(0x7f1b50102240 msgr2=0x7f1b50102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:32.270 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.657+0000 7f1b55293700 1 -- 172.21.15.45:0/3911396938 shutdown_connections 2024-06-06T04:19:32.270 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:31.657+0000 7f1b55293700 1 -- 172.21.15.45:0/3911396938 wait complete. 2024-06-06T04:19:32.270 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-06-06T04:19:32.270 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 5... 2024-06-06T04:19:32.577 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:32 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:32.353+0000 7fd83271d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T04:19:32.886 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:32 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:32.665+0000 7fd83271d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T04:19:32.886 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:32 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:32.764+0000 7fd83271d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T04:19:33.187 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:32 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:32.983+0000 7fd83271d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T04:19:33.187 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:33 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:33.186+0000 7fd83271d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T04:19:34.088 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:33 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:33.827+0000 7fd83271d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T04:19:34.375 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:34 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:34.272+0000 7fd83271d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T04:19:34.375 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:34 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:34.374+0000 7fd83271d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T04:19:34.726 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:34 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:34.586+0000 7fd83271d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T04:19:35.476 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:35 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:35.136+0000 7fd83271d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T04:19:35.476 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:35 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:35.233+0000 7fd83271d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T04:19:35.851 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:35 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:35.488+0000 7fd83271d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T04:19:35.851 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:35 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:35.630+0000 7fd83271d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T04:19:35.851 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:35 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:35.729+0000 7fd83271d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T04:19:35.851 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:35 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:35.850+0000 7fd83271d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T04:19:36.476 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:36 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:36.161+0000 7fd83271d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T04:19:37.077 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:36 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:36.795+0000 7fd83271d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T04:19:37.077 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:37 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:37.024+0000 7fd83271d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T04:19:37.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:37 smithi045 ceph-mon[32299]: Active manager daemon x restarted 2024-06-06T04:19:37.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:37 smithi045 ceph-mon[32299]: Activating manager daemon x 2024-06-06T04:19:37.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:37 smithi045 ceph-mon[32299]: osdmap e2: 0 total, 0 up, 0 in 2024-06-06T04:19:37.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:37 smithi045 ceph-mon[32299]: mgrmap e6: x(active, starting, since 0.00337916s) 2024-06-06T04:19:37.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:37 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T04:19:37.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:37 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T04:19:37.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:37 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T04:19:37.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:37 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T04:19:37.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:37 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T04:19:37.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:37 smithi045 ceph-mon[32299]: Manager daemon x is now available 2024-06-06T04:19:37.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:37 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' 2024-06-06T04:19:37.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:37 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' 2024-06-06T04:19:37.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:37 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:19:37.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:37 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:19:37.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:37 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T04:19:38.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:38 smithi045 ceph-mon[32299]: Found migration_current of "None". Setting to last migration. 2024-06-06T04:19:38.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:38 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T04:19:38.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:38 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' 2024-06-06T04:19:38.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:38 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' 2024-06-06T04:19:38.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:38 smithi045 ceph-mon[32299]: mgrmap e7: x(active, since 1.00566s) 2024-06-06T04:19:38.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-06-06T04:19:38.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-06T04:19:38.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-06T04:19:38.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-06-06T04:19:38.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.680+0000 7f0226566700 1 Processor -- start 2024-06-06T04:19:38.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.680+0000 7f0226566700 1 -- start start 2024-06-06T04:19:38.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.681+0000 7f0226566700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220108870 0x7f022010ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:38.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.681+0000 7f0226566700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f022007be20 con 0x7f0220108870 2024-06-06T04:19:38.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.681+0000 7f021ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220108870 0x7f022010ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.681+0000 7f021ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220108870 0x7f022010ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35192/0 (socket says 172.21.15.45:35192) 2024-06-06T04:19:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.681+0000 7f021ffff700 1 -- 172.21.15.45:0/4118974644 learned_addr learned my addr 172.21.15.45:0/4118974644 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.681+0000 7f021ffff700 1 -- 172.21.15.45:0/4118974644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022010b1a0 con 0x7f0220108870 2024-06-06T04:19:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.681+0000 7f021ffff700 1 --2- 172.21.15.45:0/4118974644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220108870 0x7f022010ac60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f02100098a0 tx=0x7f0210009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c092eba0da46ab server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.681+0000 7f021effd700 1 -- 172.21.15.45:0/4118974644 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0210004160 con 0x7f0220108870 2024-06-06T04:19:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.682+0000 7f021effd700 1 -- 172.21.15.45:0/4118974644 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0210004300 con 0x7f0220108870 2024-06-06T04:19:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.682+0000 7f021effd700 1 -- 172.21.15.45:0/4118974644 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0210011a80 con 0x7f0220108870 2024-06-06T04:19:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.682+0000 7f0226566700 1 -- 172.21.15.45:0/4118974644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220108870 msgr2=0x7f022010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.682+0000 7f0226566700 1 --2- 172.21.15.45:0/4118974644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220108870 0x7f022010ac60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f02100098a0 tx=0x7f0210009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.682+0000 7f0226566700 1 -- 172.21.15.45:0/4118974644 shutdown_connections 2024-06-06T04:19:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.682+0000 7f0226566700 1 --2- 172.21.15.45:0/4118974644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220108870 0x7f022010ac60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.682+0000 7f0226566700 1 -- 172.21.15.45:0/4118974644 >> 172.21.15.45:0/4118974644 conn(0x7f0220102240 msgr2=0x7f0220104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:38.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.682+0000 7f0226566700 1 -- 172.21.15.45:0/4118974644 shutdown_connections 2024-06-06T04:19:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.682+0000 7f0226566700 1 -- 172.21.15.45:0/4118974644 wait complete. 2024-06-06T04:19:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.683+0000 7f0226566700 1 Processor -- start 2024-06-06T04:19:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.683+0000 7f0226566700 1 -- start start 2024-06-06T04:19:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.683+0000 7f0226566700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220108870 0x7f02201080f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.683+0000 7f0226566700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f022010c100 con 0x7f0220108870 2024-06-06T04:19:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.683+0000 7f021ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220108870 0x7f02201080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.683+0000 7f021ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220108870 0x7f02201080f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35208/0 (socket says 172.21.15.45:35208) 2024-06-06T04:19:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.683+0000 7f021ffff700 1 -- 172.21.15.45:0/2035706510 learned_addr learned my addr 172.21.15.45:0/2035706510 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.684+0000 7f021ffff700 1 -- 172.21.15.45:0/2035706510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0220108630 con 0x7f0220108870 2024-06-06T04:19:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.684+0000 7f021ffff700 1 --2- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220108870 0x7f02201080f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0210006af0 tx=0x7f0210011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.684+0000 7f021d7fa700 1 -- 172.21.15.45:0/2035706510 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02100039e0 con 0x7f0220108870 2024-06-06T04:19:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.684+0000 7f021d7fa700 1 -- 172.21.15.45:0/2035706510 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0210003b80 con 0x7f0220108870 2024-06-06T04:19:38.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.684+0000 7f0226566700 1 -- 172.21.15.45:0/2035706510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0220106820 con 0x7f0220108870 2024-06-06T04:19:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.684+0000 7f021d7fa700 1 -- 172.21.15.45:0/2035706510 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02100196a0 con 0x7f0220108870 2024-06-06T04:19:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.684+0000 7f0226566700 1 -- 172.21.15.45:0/2035706510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0220106d60 con 0x7f0220108870 2024-06-06T04:19:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.685+0000 7f021d7fa700 1 -- 172.21.15.45:0/2035706510 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f0210019950 con 0x7f0220108870 2024-06-06T04:19:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.685+0000 7f021d7fa700 1 --2- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f0208039050 0x7f020803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.685+0000 7f0226566700 1 -- 172.21.15.45:0/2035706510 --> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0220063af0 con 0x7f0208039050 2024-06-06T04:19:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.685+0000 7f021d7fa700 1 -- 172.21.15.45:0/2035706510 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f021004d710 con 0x7f0220108870 2024-06-06T04:19:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.685+0000 7f021f7fe700 1 -- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f0208039050 msgr2=0x7f020803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1562511056 2024-06-06T04:19:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.685+0000 7f021f7fe700 1 --2- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f0208039050 0x7f020803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T04:19:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.886+0000 7f021f7fe700 1 -- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f0208039050 msgr2=0x7f020803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1562511056 2024-06-06T04:19:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:32.886+0000 7f021f7fe700 1 --2- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f0208039050 0x7f020803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T04:19:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:33.287+0000 7f021f7fe700 1 -- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f0208039050 msgr2=0x7f020803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1562511056 2024-06-06T04:19:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:33.287+0000 7f021f7fe700 1 --2- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f0208039050 0x7f020803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-06T04:19:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:34.088+0000 7f021f7fe700 1 -- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f0208039050 msgr2=0x7f020803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1562511056 2024-06-06T04:19:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:34.088+0000 7f021f7fe700 1 --2- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f0208039050 0x7f020803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-06T04:19:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:35.689+0000 7f021f7fe700 1 -- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f0208039050 msgr2=0x7f020803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1562511056 2024-06-06T04:19:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:35.689+0000 7f021f7fe700 1 --2- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f0208039050 0x7f020803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-06T04:19:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:37.027+0000 7f021d7fa700 1 -- 172.21.15.45:0/2035706510 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f021001a430 con 0x7f0220108870 2024-06-06T04:19:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:37.027+0000 7f021d7fa700 1 -- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f0208039050 msgr2=0x7f020803b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T04:19:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:37.027+0000 7f021d7fa700 1 --2- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f0208039050 0x7f020803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.028+0000 7f021d7fa700 1 -- 172.21.15.45:0/2035706510 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f02100178e0 con 0x7f0220108870 2024-06-06T04:19:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.028+0000 7f021d7fa700 1 --2- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f020803c790 0x7f020803eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.028+0000 7f021d7fa700 1 -- 172.21.15.45:0/2035706510 --> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0220063af0 con 0x7f020803c790 2024-06-06T04:19:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.029+0000 7f021f7fe700 1 --2- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f020803c790 0x7f020803eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.030+0000 7f021f7fe700 1 --2- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f020803c790 0x7f020803eb80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0214003a10 tx=0x7f02140092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.030+0000 7f021d7fa700 1 -- 172.21.15.45:0/2035706510 <== mgr.14120 v2:172.21.15.45:6800/3465796147 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f0220063af0 con 0x7f020803c790 2024-06-06T04:19:38.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.037+0000 7f0226566700 1 -- 172.21.15.45:0/2035706510 --> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f02201033c0 con 0x7f020803c790 2024-06-06T04:19:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.037+0000 7f021d7fa700 1 -- 172.21.15.45:0/2035706510 <== mgr.14120 v2:172.21.15.45:6800/3465796147 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f02201033c0 con 0x7f020803c790 2024-06-06T04:19:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.038+0000 7f0226566700 1 -- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f020803c790 msgr2=0x7f020803eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.038+0000 7f0226566700 1 --2- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f020803c790 0x7f020803eb80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0214003a10 tx=0x7f02140092b0 comp rx=0 tx=0).stop 2024-06-06T04:19:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.038+0000 7f0226566700 1 -- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220108870 msgr2=0x7f02201080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.038+0000 7f0226566700 1 --2- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220108870 0x7f02201080f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0210006af0 tx=0x7f0210011c40 comp rx=0 tx=0).stop 2024-06-06T04:19:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.038+0000 7f0226566700 1 -- 172.21.15.45:0/2035706510 shutdown_connections 2024-06-06T04:19:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.038+0000 7f0226566700 1 --2- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f020803c790 0x7f020803eb80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.038+0000 7f0226566700 1 --2- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220108870 0x7f02201080f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.038+0000 7f0226566700 1 --2- 172.21.15.45:0/2035706510 >> [v2:172.21.15.45:6800/1562511056,v1:172.21.15.45:6801/1562511056] conn(0x7f0208039050 0x7f020803b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.038+0000 7f0226566700 1 -- 172.21.15.45:0/2035706510 >> 172.21.15.45:0/2035706510 conn(0x7f0220102240 msgr2=0x7f0220102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.038+0000 7f0226566700 1 -- 172.21.15.45:0/2035706510 shutdown_connections 2024-06-06T04:19:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:38.038+0000 7f0226566700 1 -- 172.21.15.45:0/2035706510 wait complete. 2024-06-06T04:19:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 5 is available 2024-06-06T04:19:38.637 INFO:teuthology.orchestra.run.smithi045.stdout:Setting orchestrator backend to cephadm... 2024-06-06T04:19:39.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:39 smithi045 ceph-mon[32299]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-06T04:19:39.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:39 smithi045 ceph-mon[32299]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-06T04:19:39.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:39 smithi045 ceph-mon[32299]: [06/Jun/2024:04:19:38] ENGINE Bus STARTING 2024-06-06T04:19:39.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:39 smithi045 ceph-mon[32299]: [06/Jun/2024:04:19:38] ENGINE Serving on https://172.21.15.45:7150 2024-06-06T04:19:39.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:39 smithi045 ceph-mon[32299]: [06/Jun/2024:04:19:38] ENGINE Serving on http://172.21.15.45:8765 2024-06-06T04:19:39.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:39 smithi045 ceph-mon[32299]: [06/Jun/2024:04:19:38] ENGINE Bus STARTED 2024-06-06T04:19:39.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:39 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:19:39.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.021+0000 7f7f2242e700 1 Processor -- start 2024-06-06T04:19:39.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.021+0000 7f7f2242e700 1 -- start start 2024-06-06T04:19:39.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.021+0000 7f7f2242e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1c1088a0 0x7f7f1c10ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:39.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.021+0000 7f7f2242e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c07be20 con 0x7f7f1c1088a0 2024-06-06T04:19:39.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.021+0000 7f7f1bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1c1088a0 0x7f7f1c10ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:39.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.022+0000 7f7f1bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1c1088a0 0x7f7f1c10ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60550/0 (socket says 172.21.15.45:60550) 2024-06-06T04:19:39.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.022+0000 7f7f1bfff700 1 -- 172.21.15.45:0/96857892 learned_addr learned my addr 172.21.15.45:0/96857892 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:39.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.022+0000 7f7f1bfff700 1 -- 172.21.15.45:0/96857892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1c10b1d0 con 0x7f7f1c1088a0 2024-06-06T04:19:39.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.022+0000 7f7f1bfff700 1 --2- 172.21.15.45:0/96857892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1c1088a0 0x7f7f1c10ac90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0098a0 tx=0x7f7f0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294a92289859a6dc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:39.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.022+0000 7f7f1affd700 1 -- 172.21.15.45:0/96857892 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f0c004160 con 0x7f7f1c1088a0 2024-06-06T04:19:39.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.022+0000 7f7f1affd700 1 -- 172.21.15.45:0/96857892 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7f0c004300 con 0x7f7f1c1088a0 2024-06-06T04:19:39.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.022+0000 7f7f1affd700 1 -- 172.21.15.45:0/96857892 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f0c011a80 con 0x7f7f1c1088a0 2024-06-06T04:19:39.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.023+0000 7f7f2242e700 1 -- 172.21.15.45:0/96857892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1c1088a0 msgr2=0x7f7f1c10ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:39.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.023+0000 7f7f2242e700 1 --2- 172.21.15.45:0/96857892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1c1088a0 0x7f7f1c10ac90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0098a0 tx=0x7f7f0c009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:39.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.023+0000 7f7f2242e700 1 -- 172.21.15.45:0/96857892 shutdown_connections 2024-06-06T04:19:39.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.023+0000 7f7f2242e700 1 --2- 172.21.15.45:0/96857892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1c1088a0 0x7f7f1c10ac90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:39.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.023+0000 7f7f2242e700 1 -- 172.21.15.45:0/96857892 >> 172.21.15.45:0/96857892 conn(0x7f7f1c102290 msgr2=0x7f7f1c1046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:39.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.023+0000 7f7f2242e700 1 -- 172.21.15.45:0/96857892 shutdown_connections 2024-06-06T04:19:39.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.023+0000 7f7f2242e700 1 -- 172.21.15.45:0/96857892 wait complete. 2024-06-06T04:19:39.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.023+0000 7f7f2242e700 1 Processor -- start 2024-06-06T04:19:39.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.023+0000 7f7f2242e700 1 -- start start 2024-06-06T04:19:39.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.024+0000 7f7f2242e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1c1088a0 0x7f7f1c108120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:39.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.024+0000 7f7f2242e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c10c130 con 0x7f7f1c1088a0 2024-06-06T04:19:39.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.024+0000 7f7f1bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1c1088a0 0x7f7f1c108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:39.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.024+0000 7f7f1bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1c1088a0 0x7f7f1c108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60566/0 (socket says 172.21.15.45:60566) 2024-06-06T04:19:39.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.024+0000 7f7f1bfff700 1 -- 172.21.15.45:0/2390830532 learned_addr learned my addr 172.21.15.45:0/2390830532 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:39.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.024+0000 7f7f1bfff700 1 -- 172.21.15.45:0/2390830532 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1c108660 con 0x7f7f1c1088a0 2024-06-06T04:19:39.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.024+0000 7f7f1bfff700 1 --2- 172.21.15.45:0/2390830532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1c1088a0 0x7f7f1c108120 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c011a50 tx=0x7f7f0c011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:39.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.024+0000 7f7f197fa700 1 -- 172.21.15.45:0/2390830532 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f0c003a80 con 0x7f7f1c1088a0 2024-06-06T04:19:39.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.024+0000 7f7f2242e700 1 -- 172.21.15.45:0/2390830532 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1c106850 con 0x7f7f1c1088a0 2024-06-06T04:19:39.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.025+0000 7f7f197fa700 1 -- 172.21.15.45:0/2390830532 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7f0c003c20 con 0x7f7f1c1088a0 2024-06-06T04:19:39.847 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.025+0000 7f7f197fa700 1 -- 172.21.15.45:0/2390830532 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f0c01a650 con 0x7f7f1c1088a0 2024-06-06T04:19:39.847 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.025+0000 7f7f2242e700 1 -- 172.21.15.45:0/2390830532 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f1c106d90 con 0x7f7f1c1088a0 2024-06-06T04:19:39.847 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.026+0000 7f7f197fa700 1 -- 172.21.15.45:0/2390830532 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f7f0c01a950 con 0x7f7f1c1088a0 2024-06-06T04:19:39.847 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.026+0000 7f7f2242e700 1 -- 172.21.15.45:0/2390830532 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f1c063af0 con 0x7f7f1c1088a0 2024-06-06T04:19:39.847 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.026+0000 7f7f197fa700 1 --2- 172.21.15.45:0/2390830532 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f7f04038f30 0x7f7f0403b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:39.847 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.026+0000 7f7f197fa700 1 -- 172.21.15.45:0/2390830532 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7f0c04d390 con 0x7f7f1c1088a0 2024-06-06T04:19:39.847 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.026+0000 7f7f1b7fe700 1 --2- 172.21.15.45:0/2390830532 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f7f04038f30 0x7f7f0403b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:39.847 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.026+0000 7f7f1b7fe700 1 --2- 172.21.15.45:0/2390830532 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f7f04038f30 0x7f7f0403b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7f10006fd0 tx=0x7f7f10006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:39.847 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.031+0000 7f7f197fa700 1 -- 172.21.15.45:0/2390830532 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7f0c016df0 con 0x7f7f1c1088a0 2024-06-06T04:19:39.847 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.081+0000 7f7f197fa700 1 -- 172.21.15.45:0/2390830532 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7f0c024ad0 con 0x7f7f1c1088a0 2024-06-06T04:19:39.847 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.256+0000 7f7f2242e700 1 -- 172.21.15.45:0/2390830532 --> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f7f1c103440 con 0x7f7f04038f30 2024-06-06T04:19:39.847 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.261+0000 7f7f197fa700 1 -- 172.21.15.45:0/2390830532 <== mgr.14120 v2:172.21.15.45:6800/3465796147 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7f1c103440 con 0x7f7f04038f30 2024-06-06T04:19:39.847 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.263+0000 7f7f2242e700 1 -- 172.21.15.45:0/2390830532 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f7f04038f30 msgr2=0x7f7f0403b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:39.848 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.263+0000 7f7f2242e700 1 --2- 172.21.15.45:0/2390830532 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f7f04038f30 0x7f7f0403b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7f10006fd0 tx=0x7f7f10006e40 comp rx=0 tx=0).stop 2024-06-06T04:19:39.848 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.263+0000 7f7f2242e700 1 -- 172.21.15.45:0/2390830532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1c1088a0 msgr2=0x7f7f1c108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:39.848 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.263+0000 7f7f2242e700 1 --2- 172.21.15.45:0/2390830532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1c1088a0 0x7f7f1c108120 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c011a50 tx=0x7f7f0c011b30 comp rx=0 tx=0).stop 2024-06-06T04:19:39.848 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.263+0000 7f7f2242e700 1 -- 172.21.15.45:0/2390830532 shutdown_connections 2024-06-06T04:19:39.848 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.263+0000 7f7f2242e700 1 --2- 172.21.15.45:0/2390830532 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f7f04038f30 0x7f7f0403b3f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:39.848 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.263+0000 7f7f2242e700 1 --2- 172.21.15.45:0/2390830532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1c1088a0 0x7f7f1c108120 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:39.848 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.263+0000 7f7f2242e700 1 -- 172.21.15.45:0/2390830532 >> 172.21.15.45:0/2390830532 conn(0x7f7f1c102290 msgr2=0x7f7f1c102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:39.848 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.264+0000 7f7f2242e700 1 -- 172.21.15.45:0/2390830532 shutdown_connections 2024-06-06T04:19:39.848 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:39.264+0000 7f7f2242e700 1 -- 172.21.15.45:0/2390830532 wait complete. 2024-06-06T04:19:40.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:40 smithi045 ceph-mon[32299]: mgrmap e8: x(active, since 2s) 2024-06-06T04:19:40.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:40 smithi045 ceph-mon[32299]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:19:40.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:40 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' 2024-06-06T04:19:40.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:40 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:19:41.000 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-06T04:19:41.000 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.229+0000 7f70e04b6700 1 Processor -- start 2024-06-06T04:19:41.000 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.229+0000 7f70e04b6700 1 -- start start 2024-06-06T04:19:41.000 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.229+0000 7f70e04b6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d8108860 0x7f70d810ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:41.000 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.229+0000 7f70e04b6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70d807be60 con 0x7f70d8108860 2024-06-06T04:19:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.230+0000 7f70de252700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d8108860 0x7f70d810ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.230+0000 7f70de252700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d8108860 0x7f70d810ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60580/0 (socket says 172.21.15.45:60580) 2024-06-06T04:19:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.230+0000 7f70de252700 1 -- 172.21.15.45:0/3558247050 learned_addr learned my addr 172.21.15.45:0/3558247050 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.230+0000 7f70de252700 1 -- 172.21.15.45:0/3558247050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d810b190 con 0x7f70d8108860 2024-06-06T04:19:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.230+0000 7f70de252700 1 --2- 172.21.15.45:0/3558247050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d8108860 0x7f70d810ac50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f70d40098a0 tx=0x7f70d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0907457754fbaa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.230+0000 7f70dd250700 1 -- 172.21.15.45:0/3558247050 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70d4004160 con 0x7f70d8108860 2024-06-06T04:19:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.230+0000 7f70dd250700 1 -- 172.21.15.45:0/3558247050 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f70d4004300 con 0x7f70d8108860 2024-06-06T04:19:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.230+0000 7f70dd250700 1 -- 172.21.15.45:0/3558247050 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70d4011a80 con 0x7f70d8108860 2024-06-06T04:19:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.231+0000 7f70e04b6700 1 -- 172.21.15.45:0/3558247050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d8108860 msgr2=0x7f70d810ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.231+0000 7f70e04b6700 1 --2- 172.21.15.45:0/3558247050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d8108860 0x7f70d810ac50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f70d40098a0 tx=0x7f70d4009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.231+0000 7f70e04b6700 1 -- 172.21.15.45:0/3558247050 shutdown_connections 2024-06-06T04:19:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.231+0000 7f70e04b6700 1 --2- 172.21.15.45:0/3558247050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d8108860 0x7f70d810ac50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.231+0000 7f70e04b6700 1 -- 172.21.15.45:0/3558247050 >> 172.21.15.45:0/3558247050 conn(0x7f70d8102290 msgr2=0x7f70d81046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.231+0000 7f70e04b6700 1 -- 172.21.15.45:0/3558247050 shutdown_connections 2024-06-06T04:19:41.002 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.231+0000 7f70e04b6700 1 -- 172.21.15.45:0/3558247050 wait complete. 2024-06-06T04:19:41.002 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.232+0000 7f70e04b6700 1 Processor -- start 2024-06-06T04:19:41.002 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.232+0000 7f70e04b6700 1 -- start start 2024-06-06T04:19:41.002 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.232+0000 7f70e04b6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d8108860 0x7f70d819bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:41.002 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.232+0000 7f70e04b6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70d810c150 con 0x7f70d8108860 2024-06-06T04:19:41.002 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.232+0000 7f70de252700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d8108860 0x7f70d819bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:41.002 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.232+0000 7f70de252700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d8108860 0x7f70d819bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60596/0 (socket says 172.21.15.45:60596) 2024-06-06T04:19:41.002 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.232+0000 7f70de252700 1 -- 172.21.15.45:0/1234999275 learned_addr learned my addr 172.21.15.45:0/1234999275 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:41.002 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.232+0000 7f70de252700 1 -- 172.21.15.45:0/1234999275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d819c1e0 con 0x7f70d8108860 2024-06-06T04:19:41.002 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.232+0000 7f70de252700 1 --2- 172.21.15.45:0/1234999275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d8108860 0x7f70d819bca0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f70d4006af0 tx=0x7f70d4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:41.002 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.233+0000 7f70cf7fe700 1 -- 172.21.15.45:0/1234999275 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70d40039e0 con 0x7f70d8108860 2024-06-06T04:19:41.002 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.233+0000 7f70cf7fe700 1 -- 172.21.15.45:0/1234999275 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f70d4003b80 con 0x7f70d8108860 2024-06-06T04:19:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.233+0000 7f70e04b6700 1 -- 172.21.15.45:0/1234999275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d819c470 con 0x7f70d8108860 2024-06-06T04:19:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.233+0000 7f70cf7fe700 1 -- 172.21.15.45:0/1234999275 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70d40196a0 con 0x7f70d8108860 2024-06-06T04:19:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.233+0000 7f70e04b6700 1 -- 172.21.15.45:0/1234999275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d819f160 con 0x7f70d8108860 2024-06-06T04:19:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.233+0000 7f70cf7fe700 1 -- 172.21.15.45:0/1234999275 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f70d401a430 con 0x7f70d8108860 2024-06-06T04:19:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.234+0000 7f70e04b6700 1 -- 172.21.15.45:0/1234999275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d8063af0 con 0x7f70d8108860 2024-06-06T04:19:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.234+0000 7f70cf7fe700 1 --2- 172.21.15.45:0/1234999275 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f70c4039000 0x7f70c403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.234+0000 7f70cf7fe700 1 -- 172.21.15.45:0/1234999275 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f70d404d620 con 0x7f70d8108860 2024-06-06T04:19:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.234+0000 7f70dda51700 1 --2- 172.21.15.45:0/1234999275 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f70c4039000 0x7f70c403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.234+0000 7f70dda51700 1 --2- 172.21.15.45:0/1234999275 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f70c4039000 0x7f70c403b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f70c8006fd0 tx=0x7f70c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.239+0000 7f70cf7fe700 1 -- 172.21.15.45:0/1234999275 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f70d402de00 con 0x7f70d8108860 2024-06-06T04:19:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.448+0000 7f70e04b6700 1 -- 172.21.15.45:0/1234999275 --> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f70d8103410 con 0x7f70c4039000 2024-06-06T04:19:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.448+0000 7f70cf7fe700 1 -- 172.21.15.45:0/1234999275 <== mgr.14120 v2:172.21.15.45:6800/3465796147 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f70d8103410 con 0x7f70c4039000 2024-06-06T04:19:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.451+0000 7f70e04b6700 1 -- 172.21.15.45:0/1234999275 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f70c4039000 msgr2=0x7f70c403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:41.003 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.451+0000 7f70e04b6700 1 --2- 172.21.15.45:0/1234999275 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f70c4039000 0x7f70c403b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f70c8006fd0 tx=0x7f70c8006e40 comp rx=0 tx=0).stop 2024-06-06T04:19:41.004 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.451+0000 7f70e04b6700 1 -- 172.21.15.45:0/1234999275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d8108860 msgr2=0x7f70d819bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:41.004 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.451+0000 7f70e04b6700 1 --2- 172.21.15.45:0/1234999275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d8108860 0x7f70d819bca0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f70d4006af0 tx=0x7f70d4011c40 comp rx=0 tx=0).stop 2024-06-06T04:19:41.004 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.451+0000 7f70e04b6700 1 -- 172.21.15.45:0/1234999275 shutdown_connections 2024-06-06T04:19:41.004 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.451+0000 7f70e04b6700 1 --2- 172.21.15.45:0/1234999275 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f70c4039000 0x7f70c403b4c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:41.004 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.451+0000 7f70e04b6700 1 --2- 172.21.15.45:0/1234999275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d8108860 0x7f70d819bca0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:41.004 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.451+0000 7f70e04b6700 1 -- 172.21.15.45:0/1234999275 >> 172.21.15.45:0/1234999275 conn(0x7f70d8102290 msgr2=0x7f70d8102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:41.004 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.451+0000 7f70e04b6700 1 -- 172.21.15.45:0/1234999275 shutdown_connections 2024-06-06T04:19:41.004 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:40.451+0000 7f70e04b6700 1 -- 172.21.15.45:0/1234999275 wait complete. 2024-06-06T04:19:41.004 INFO:teuthology.orchestra.run.smithi045.stdout:Generating ssh key... 2024-06-06T04:19:41.279 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-mon[32299]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:19:42.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:42 smithi045 ceph-mon[32299]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:19:42.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:42 smithi045 ceph-mon[32299]: Generating ssh key... 2024-06-06T04:19:42.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:42 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' 2024-06-06T04:19:42.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:42 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' 2024-06-06T04:19:42.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: Generating public/private rsa key pair. 2024-06-06T04:19:42.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: Your identification has been saved in /tmp/tmppxi2z2or/key. 2024-06-06T04:19:42.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: Your public key has been saved in /tmp/tmppxi2z2or/key.pub. 2024-06-06T04:19:42.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: The key fingerprint is: 2024-06-06T04:19:42.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: SHA256:FbrvXWCa/Kr7wPtNPNsqyS/DcMKk2kxaVIjOrA0FuwA ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:19:42.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: The key's randomart image is: 2024-06-06T04:19:42.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: +---[RSA 3072]----+ 2024-06-06T04:19:42.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: |E .. . . . | 2024-06-06T04:19:42.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: |. .o . .. . | 2024-06-06T04:19:42.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: |. .= .. . | 2024-06-06T04:19:42.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: | ...+ . .o | 2024-06-06T04:19:42.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: | .+ . +S o | 2024-06-06T04:19:42.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: | . . +.+o.= . | 2024-06-06T04:19:42.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: | B o*=.+ . | 2024-06-06T04:19:42.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: | o o +B= = | 2024-06-06T04:19:42.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: | +==*B.. | 2024-06-06T04:19:42.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: +----[SHA256]-----+ 2024-06-06T04:19:42.549 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.386+0000 7f1363a49700 1 Processor -- start 2024-06-06T04:19:42.549 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.386+0000 7f1363a49700 1 -- start start 2024-06-06T04:19:42.549 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.387+0000 7f1363a49700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f135c107df0 0x7f135c108210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:42.549 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.387+0000 7f1363a49700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1087e0 con 0x7f135c107df0 2024-06-06T04:19:42.549 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.387+0000 7f13617e5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f135c107df0 0x7f135c108210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:42.549 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.387+0000 7f13617e5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f135c107df0 0x7f135c108210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60612/0 (socket says 172.21.15.45:60612) 2024-06-06T04:19:42.549 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.387+0000 7f13617e5700 1 -- 172.21.15.45:0/3773469278 learned_addr learned my addr 172.21.15.45:0/3773469278 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:42.549 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.387+0000 7f13617e5700 1 -- 172.21.15.45:0/3773469278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135c108f50 con 0x7f135c107df0 2024-06-06T04:19:42.549 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.387+0000 7f13617e5700 1 --2- 172.21.15.45:0/3773469278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f135c107df0 0x7f135c108210 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f134c009e80 tx=0x7f134c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d065c599518c1120 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:42.549 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.387+0000 7f135bfff700 1 -- 172.21.15.45:0/3773469278 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f134c017070 con 0x7f135c107df0 2024-06-06T04:19:42.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.387+0000 7f135bfff700 1 -- 172.21.15.45:0/3773469278 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f134c00b8c0 con 0x7f135c107df0 2024-06-06T04:19:42.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.387+0000 7f135bfff700 1 -- 172.21.15.45:0/3773469278 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f134c004520 con 0x7f135c107df0 2024-06-06T04:19:42.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.388+0000 7f1363a49700 1 -- 172.21.15.45:0/3773469278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f135c107df0 msgr2=0x7f135c108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:42.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.388+0000 7f1363a49700 1 --2- 172.21.15.45:0/3773469278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f135c107df0 0x7f135c108210 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f134c009e80 tx=0x7f134c00b330 comp rx=0 tx=0).stop 2024-06-06T04:19:42.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.388+0000 7f1363a49700 1 -- 172.21.15.45:0/3773469278 shutdown_connections 2024-06-06T04:19:42.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.388+0000 7f1363a49700 1 --2- 172.21.15.45:0/3773469278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f135c107df0 0x7f135c108210 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:42.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.388+0000 7f1363a49700 1 -- 172.21.15.45:0/3773469278 >> 172.21.15.45:0/3773469278 conn(0x7f135c1033d0 msgr2=0x7f135c1057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:42.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.388+0000 7f1363a49700 1 -- 172.21.15.45:0/3773469278 shutdown_connections 2024-06-06T04:19:42.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.388+0000 7f1363a49700 1 -- 172.21.15.45:0/3773469278 wait complete. 2024-06-06T04:19:42.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.389+0000 7f1363a49700 1 Processor -- start 2024-06-06T04:19:42.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.389+0000 7f1363a49700 1 -- start start 2024-06-06T04:19:42.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.389+0000 7f1363a49700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f135c107df0 0x7f135c19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:42.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.389+0000 7f1363a49700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f135c10c300 con 0x7f135c107df0 2024-06-06T04:19:42.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.389+0000 7f13617e5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f135c107df0 0x7f135c19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:42.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.389+0000 7f13617e5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f135c107df0 0x7f135c19bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60628/0 (socket says 172.21.15.45:60628) 2024-06-06T04:19:42.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.389+0000 7f13617e5700 1 -- 172.21.15.45:0/1135987809 learned_addr learned my addr 172.21.15.45:0/1135987809 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:42.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.389+0000 7f13617e5700 1 -- 172.21.15.45:0/1135987809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135c19c1b0 con 0x7f135c107df0 2024-06-06T04:19:42.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.390+0000 7f13617e5700 1 --2- 172.21.15.45:0/1135987809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f135c107df0 0x7f135c19bc70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f134c009e50 tx=0x7f134c004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:42.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.390+0000 7f135a7fc700 1 -- 172.21.15.45:0/1135987809 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f134c020020 con 0x7f135c107df0 2024-06-06T04:19:42.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.390+0000 7f135a7fc700 1 -- 172.21.15.45:0/1135987809 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f134c00bba0 con 0x7f135c107df0 2024-06-06T04:19:42.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.390+0000 7f1363a49700 1 -- 172.21.15.45:0/1135987809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135c19c440 con 0x7f135c107df0 2024-06-06T04:19:42.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.390+0000 7f135a7fc700 1 -- 172.21.15.45:0/1135987809 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f134c017040 con 0x7f135c107df0 2024-06-06T04:19:42.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.390+0000 7f1363a49700 1 -- 172.21.15.45:0/1135987809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135c19f130 con 0x7f135c107df0 2024-06-06T04:19:42.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.391+0000 7f135a7fc700 1 -- 172.21.15.45:0/1135987809 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f134c023020 con 0x7f135c107df0 2024-06-06T04:19:42.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.391+0000 7f1363a49700 1 -- 172.21.15.45:0/1135987809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f135c063af0 con 0x7f135c107df0 2024-06-06T04:19:42.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.391+0000 7f135a7fc700 1 --2- 172.21.15.45:0/1135987809 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f1344039000 0x7f134403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:42.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.391+0000 7f135a7fc700 1 -- 172.21.15.45:0/1135987809 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f134c00f050 con 0x7f135c107df0 2024-06-06T04:19:42.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.391+0000 7f1360fe4700 1 --2- 172.21.15.45:0/1135987809 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f1344039000 0x7f134403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:42.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.392+0000 7f1360fe4700 1 --2- 172.21.15.45:0/1135987809 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f1344039000 0x7f134403b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1350006fd0 tx=0x7f1350006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:42.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.396+0000 7f135a7fc700 1 -- 172.21.15.45:0/1135987809 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f134c02a020 con 0x7f135c107df0 2024-06-06T04:19:42.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.603+0000 7f1363a49700 1 -- 172.21.15.45:0/1135987809 --> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f135c1045d0 con 0x7f1344039000 2024-06-06T04:19:42.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.899+0000 7f135a7fc700 1 -- 172.21.15.45:0/1135987809 <== mgr.14120 v2:172.21.15.45:6800/3465796147 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f135c1045d0 con 0x7f1344039000 2024-06-06T04:19:42.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.902+0000 7f1363a49700 1 -- 172.21.15.45:0/1135987809 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f1344039000 msgr2=0x7f134403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:42.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.902+0000 7f1363a49700 1 --2- 172.21.15.45:0/1135987809 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f1344039000 0x7f134403b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1350006fd0 tx=0x7f1350006e40 comp rx=0 tx=0).stop 2024-06-06T04:19:42.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.902+0000 7f1363a49700 1 -- 172.21.15.45:0/1135987809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f135c107df0 msgr2=0x7f135c19bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:42.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.902+0000 7f1363a49700 1 --2- 172.21.15.45:0/1135987809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f135c107df0 0x7f135c19bc70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f134c009e50 tx=0x7f134c004670 comp rx=0 tx=0).stop 2024-06-06T04:19:42.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.902+0000 7f1363a49700 1 -- 172.21.15.45:0/1135987809 shutdown_connections 2024-06-06T04:19:42.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.902+0000 7f1363a49700 1 --2- 172.21.15.45:0/1135987809 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f1344039000 0x7f134403b4c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:42.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.902+0000 7f1363a49700 1 --2- 172.21.15.45:0/1135987809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f135c107df0 0x7f135c19bc70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:42.553 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.902+0000 7f1363a49700 1 -- 172.21.15.45:0/1135987809 >> 172.21.15.45:0/1135987809 conn(0x7f135c1033d0 msgr2=0x7f135c103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:42.553 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.902+0000 7f1363a49700 1 -- 172.21.15.45:0/1135987809 shutdown_connections 2024-06-06T04:19:42.553 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:41.902+0000 7f1363a49700 1 -- 172.21.15.45:0/1135987809 wait complete. 2024-06-06T04:19:43.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaQ80a1msjimWFH42LBkkRUHyrBG+CxhI6lol3TFACE1Nzq2HjAR5zPbKduXNJkpsaAxWkQeZrTuIDzp2q0xY7yCVt5NBvt3KyjMKdaxZ38WMdu1TUYtkPBf+7RwZ/xY5uFsNlY15sglJoV2OfKAqp8Gra1kzPny1SuFFAlcmCbbBf079j7hYTRW5nO2hIdM4Ar9Tjrzd52WnniuLeX37zAlr1s35VkdznYI81ZZqNJkk01NhgICJnLl2P2Hn/ysoEbnQ2UwXjq4BlCjebWRcXHVTmnQ34t0L3nBJqIVhfidqJ5IB6XV9TUxoDxH/72sFFhqDridkkSb0nzWbc07Wym8JIgRDvaf313FK3Kx/nF8/C7xozl5OuFGwtzRnXyFswoA8Hw1vHwQaRJXeTjyWFTdKKAOpNse+LoZYix2t/0410nBe750lPkvhI5eZ5Votz3P3gzEDR5XUI50HEIfWGGEFXrrVwlLNmPgVeC8qhU8Pu7WCMTxFHLF0alnntCc8= ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:19:43.722 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.949+0000 7f2374154700 1 Processor -- start 2024-06-06T04:19:43.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.949+0000 7f2374154700 1 -- start start 2024-06-06T04:19:43.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.949+0000 7f2374154700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236c07de70 0x7f236c07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:43.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.949+0000 7f2374154700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f236c07c8a0 con 0x7f236c07de70 2024-06-06T04:19:43.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.950+0000 7f2371ef0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236c07de70 0x7f236c07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:43.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.950+0000 7f2371ef0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236c07de70 0x7f236c07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60634/0 (socket says 172.21.15.45:60634) 2024-06-06T04:19:43.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.950+0000 7f2371ef0700 1 -- 172.21.15.45:0/692041145 learned_addr learned my addr 172.21.15.45:0/692041145 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:43.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.950+0000 7f2371ef0700 1 -- 172.21.15.45:0/692041145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c07ca20 con 0x7f236c07de70 2024-06-06T04:19:43.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.950+0000 7f2371ef0700 1 --2- 172.21.15.45:0/692041145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236c07de70 0x7f236c07c2d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23680098a0 tx=0x7f2368009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf10f22446f825c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:43.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.950+0000 7f2370eee700 1 -- 172.21.15.45:0/692041145 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2368004160 con 0x7f236c07de70 2024-06-06T04:19:43.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.950+0000 7f2370eee700 1 -- 172.21.15.45:0/692041145 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2368004300 con 0x7f236c07de70 2024-06-06T04:19:43.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.950+0000 7f2370eee700 1 -- 172.21.15.45:0/692041145 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2368011a80 con 0x7f236c07de70 2024-06-06T04:19:43.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.951+0000 7f2374154700 1 -- 172.21.15.45:0/692041145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236c07de70 msgr2=0x7f236c07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:43.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.951+0000 7f2374154700 1 --2- 172.21.15.45:0/692041145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236c07de70 0x7f236c07c2d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23680098a0 tx=0x7f2368009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:43.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.951+0000 7f2374154700 1 -- 172.21.15.45:0/692041145 shutdown_connections 2024-06-06T04:19:43.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.951+0000 7f2374154700 1 --2- 172.21.15.45:0/692041145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236c07de70 0x7f236c07c2d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:43.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.951+0000 7f2374154700 1 -- 172.21.15.45:0/692041145 >> 172.21.15.45:0/692041145 conn(0x7f236c103310 msgr2=0x7f236c105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:43.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.951+0000 7f2374154700 1 -- 172.21.15.45:0/692041145 shutdown_connections 2024-06-06T04:19:43.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.951+0000 7f2374154700 1 -- 172.21.15.45:0/692041145 wait complete. 2024-06-06T04:19:43.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.952+0000 7f2374154700 1 Processor -- start 2024-06-06T04:19:43.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.952+0000 7f2374154700 1 -- start start 2024-06-06T04:19:43.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.952+0000 7f2374154700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236c07de70 0x7f236c19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:43.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.952+0000 7f2374154700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f236c10e320 con 0x7f236c07de70 2024-06-06T04:19:43.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.952+0000 7f2371ef0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236c07de70 0x7f236c19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:43.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.952+0000 7f2371ef0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236c07de70 0x7f236c19de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60636/0 (socket says 172.21.15.45:60636) 2024-06-06T04:19:43.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.952+0000 7f2371ef0700 1 -- 172.21.15.45:0/3879027157 learned_addr learned my addr 172.21.15.45:0/3879027157 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:43.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.952+0000 7f2371ef0700 1 -- 172.21.15.45:0/3879027157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c19e360 con 0x7f236c07de70 2024-06-06T04:19:43.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.952+0000 7f2371ef0700 1 --2- 172.21.15.45:0/3879027157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236c07de70 0x7f236c19de20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2368011a50 tx=0x7f2368011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:43.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.953+0000 7f2362ffd700 1 -- 172.21.15.45:0/3879027157 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2368003a80 con 0x7f236c07de70 2024-06-06T04:19:43.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.953+0000 7f2362ffd700 1 -- 172.21.15.45:0/3879027157 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2368003c20 con 0x7f236c07de70 2024-06-06T04:19:43.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.953+0000 7f2374154700 1 -- 172.21.15.45:0/3879027157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236c19e5f0 con 0x7f236c07de70 2024-06-06T04:19:43.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.953+0000 7f2362ffd700 1 -- 172.21.15.45:0/3879027157 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f236801a650 con 0x7f236c07de70 2024-06-06T04:19:43.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.953+0000 7f2374154700 1 -- 172.21.15.45:0/3879027157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236c1a12e0 con 0x7f236c07de70 2024-06-06T04:19:43.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.954+0000 7f2374154700 1 -- 172.21.15.45:0/3879027157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f236c063af0 con 0x7f236c07de70 2024-06-06T04:19:43.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.954+0000 7f2362ffd700 1 -- 172.21.15.45:0/3879027157 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f236801a900 con 0x7f236c07de70 2024-06-06T04:19:43.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.954+0000 7f2362ffd700 1 --2- 172.21.15.45:0/3879027157 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f23580390a0 0x7f235803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:43.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.954+0000 7f2362ffd700 1 -- 172.21.15.45:0/3879027157 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f236804d630 con 0x7f236c07de70 2024-06-06T04:19:43.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.954+0000 7f23716ef700 1 --2- 172.21.15.45:0/3879027157 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f23580390a0 0x7f235803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:43.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.954+0000 7f23716ef700 1 --2- 172.21.15.45:0/3879027157 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f23580390a0 0x7f235803b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f235c006fd0 tx=0x7f235c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:43.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:42.959+0000 7f2362ffd700 1 -- 172.21.15.45:0/3879027157 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2368019c70 con 0x7f236c07de70 2024-06-06T04:19:43.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:43.168+0000 7f2374154700 1 -- 172.21.15.45:0/3879027157 --> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f236c104510 con 0x7f23580390a0 2024-06-06T04:19:43.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:43.169+0000 7f2362ffd700 1 -- 172.21.15.45:0/3879027157 <== mgr.14120 v2:172.21.15.45:6800/3465796147 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f236c104510 con 0x7f23580390a0 2024-06-06T04:19:43.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:43.171+0000 7f2374154700 1 -- 172.21.15.45:0/3879027157 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f23580390a0 msgr2=0x7f235803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:43.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:43.171+0000 7f2374154700 1 --2- 172.21.15.45:0/3879027157 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f23580390a0 0x7f235803b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f235c006fd0 tx=0x7f235c006e40 comp rx=0 tx=0).stop 2024-06-06T04:19:43.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:43.171+0000 7f2374154700 1 -- 172.21.15.45:0/3879027157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236c07de70 msgr2=0x7f236c19de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:43.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:43.171+0000 7f2374154700 1 --2- 172.21.15.45:0/3879027157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236c07de70 0x7f236c19de20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2368011a50 tx=0x7f2368011b30 comp rx=0 tx=0).stop 2024-06-06T04:19:43.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:43.171+0000 7f2374154700 1 -- 172.21.15.45:0/3879027157 shutdown_connections 2024-06-06T04:19:43.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:43.171+0000 7f2374154700 1 --2- 172.21.15.45:0/3879027157 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f23580390a0 0x7f235803b560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:43.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:43.171+0000 7f2374154700 1 --2- 172.21.15.45:0/3879027157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236c07de70 0x7f236c19de20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:43.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:43.171+0000 7f2374154700 1 -- 172.21.15.45:0/3879027157 >> 172.21.15.45:0/3879027157 conn(0x7f236c103310 msgr2=0x7f236c103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:43.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:43.171+0000 7f2374154700 1 -- 172.21.15.45:0/3879027157 shutdown_connections 2024-06-06T04:19:43.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:43.171+0000 7f2374154700 1 -- 172.21.15.45:0/3879027157 wait complete. 2024-06-06T04:19:43.727 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-06T04:19:43.727 INFO:teuthology.orchestra.run.smithi045.stdout:Adding key to root@localhost authorized_keys... 2024-06-06T04:19:43.727 INFO:teuthology.orchestra.run.smithi045.stdout:Adding host smithi045... 2024-06-06T04:19:43.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:43 smithi045 ceph-mon[32299]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:19:44.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:44 smithi045 ceph-mon[32299]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:19:46.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:45 smithi045 ceph-mon[32299]: Deploying cephadm binary to smithi045 2024-06-06T04:19:49.238 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Added host 'smithi045' with addr '172.21.15.45' 2024-06-06T04:19:49.238 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.149+0000 7f5024ed8700 1 Processor -- start 2024-06-06T04:19:49.238 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.149+0000 7f5024ed8700 1 -- start start 2024-06-06T04:19:49.238 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.149+0000 7f5024ed8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5020106cb0 0x7f50201070d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:49.238 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.149+0000 7f5024ed8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50201076a0 con 0x7f5020106cb0 2024-06-06T04:19:49.238 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.149+0000 7f501e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5020106cb0 0x7f50201070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:49.238 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.149+0000 7f501e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5020106cb0 0x7f50201070d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60648/0 (socket says 172.21.15.45:60648) 2024-06-06T04:19:49.238 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.149+0000 7f501e59c700 1 -- 172.21.15.45:0/882690459 learned_addr learned my addr 172.21.15.45:0/882690459 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:49.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.150+0000 7f501e59c700 1 -- 172.21.15.45:0/882690459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5020107eb0 con 0x7f5020106cb0 2024-06-06T04:19:49.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.150+0000 7f501e59c700 1 --2- 172.21.15.45:0/882690459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5020106cb0 0x7f50201070d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5010009e80 tx=0x7f501000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad909cbaac037ca2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:49.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.150+0000 7f501d59a700 1 -- 172.21.15.45:0/882690459 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5010017070 con 0x7f5020106cb0 2024-06-06T04:19:49.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.150+0000 7f501d59a700 1 -- 172.21.15.45:0/882690459 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f501000b8c0 con 0x7f5020106cb0 2024-06-06T04:19:49.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.150+0000 7f501d59a700 1 -- 172.21.15.45:0/882690459 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5010004520 con 0x7f5020106cb0 2024-06-06T04:19:49.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.151+0000 7f5024ed8700 1 -- 172.21.15.45:0/882690459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5020106cb0 msgr2=0x7f50201070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:49.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.151+0000 7f5024ed8700 1 --2- 172.21.15.45:0/882690459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5020106cb0 0x7f50201070d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5010009e80 tx=0x7f501000b330 comp rx=0 tx=0).stop 2024-06-06T04:19:49.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.151+0000 7f5024ed8700 1 -- 172.21.15.45:0/882690459 shutdown_connections 2024-06-06T04:19:49.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.151+0000 7f5024ed8700 1 --2- 172.21.15.45:0/882690459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5020106cb0 0x7f50201070d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:49.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.151+0000 7f5024ed8700 1 -- 172.21.15.45:0/882690459 >> 172.21.15.45:0/882690459 conn(0x7f5020102290 msgr2=0x7f50201046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:49.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.151+0000 7f5024ed8700 1 -- 172.21.15.45:0/882690459 shutdown_connections 2024-06-06T04:19:49.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.151+0000 7f5024ed8700 1 -- 172.21.15.45:0/882690459 wait complete. 2024-06-06T04:19:49.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.152+0000 7f5024ed8700 1 Processor -- start 2024-06-06T04:19:49.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.152+0000 7f5024ed8700 1 -- start start 2024-06-06T04:19:49.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.152+0000 7f5024ed8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5020106cb0 0x7f502019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:49.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.152+0000 7f5024ed8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f502010b260 con 0x7f5020106cb0 2024-06-06T04:19:49.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.152+0000 7f501e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5020106cb0 0x7f502019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:49.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.152+0000 7f501e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5020106cb0 0x7f502019bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60660/0 (socket says 172.21.15.45:60660) 2024-06-06T04:19:49.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.152+0000 7f501e59c700 1 -- 172.21.15.45:0/3015841868 learned_addr learned my addr 172.21.15.45:0/3015841868 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:49.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.152+0000 7f501e59c700 1 -- 172.21.15.45:0/3015841868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502019c210 con 0x7f5020106cb0 2024-06-06T04:19:49.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.152+0000 7f501e59c700 1 --2- 172.21.15.45:0/3015841868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5020106cb0 0x7f502019bcd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5010009e50 tx=0x7f5010004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:49.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.152+0000 7f50177fe700 1 -- 172.21.15.45:0/3015841868 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5010020020 con 0x7f5020106cb0 2024-06-06T04:19:49.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.152+0000 7f50177fe700 1 -- 172.21.15.45:0/3015841868 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f501000bba0 con 0x7f5020106cb0 2024-06-06T04:19:49.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.153+0000 7f5024ed8700 1 -- 172.21.15.45:0/3015841868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502019c4a0 con 0x7f5020106cb0 2024-06-06T04:19:49.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.153+0000 7f50177fe700 1 -- 172.21.15.45:0/3015841868 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5010017040 con 0x7f5020106cb0 2024-06-06T04:19:49.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.153+0000 7f5024ed8700 1 -- 172.21.15.45:0/3015841868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f502019f180 con 0x7f5020106cb0 2024-06-06T04:19:49.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.153+0000 7f50177fe700 1 -- 172.21.15.45:0/3015841868 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5010023020 con 0x7f5020106cb0 2024-06-06T04:19:49.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.154+0000 7f5024ed8700 1 -- 172.21.15.45:0/3015841868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5020063af0 con 0x7f5020106cb0 2024-06-06T04:19:49.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.154+0000 7f50177fe700 1 --2- 172.21.15.45:0/3015841868 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f500c038fb0 0x7f500c03b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:49.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.154+0000 7f50177fe700 1 -- 172.21.15.45:0/3015841868 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f501000f050 con 0x7f5020106cb0 2024-06-06T04:19:49.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.154+0000 7f501dd9b700 1 --2- 172.21.15.45:0/3015841868 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f500c038fb0 0x7f500c03b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:49.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.154+0000 7f501dd9b700 1 --2- 172.21.15.45:0/3015841868 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f500c038fb0 0x7f500c03b470 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5008006fd0 tx=0x7f5008006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:49.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.159+0000 7f50177fe700 1 -- 172.21.15.45:0/3015841868 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f501002a020 con 0x7f5020106cb0 2024-06-06T04:19:49.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:44.371+0000 7f5024ed8700 1 -- 172.21.15.45:0/3015841868 --> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}) -- 0x7f502002e050 con 0x7f500c038fb0 2024-06-06T04:19:49.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:48.676+0000 7f50177fe700 1 -- 172.21.15.45:0/3015841868 <== mgr.14120 v2:172.21.15.45:6800/3465796147 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f502002e050 con 0x7f500c038fb0 2024-06-06T04:19:49.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:48.679+0000 7f5024ed8700 1 -- 172.21.15.45:0/3015841868 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f500c038fb0 msgr2=0x7f500c03b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:49.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:48.679+0000 7f5024ed8700 1 --2- 172.21.15.45:0/3015841868 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f500c038fb0 0x7f500c03b470 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5008006fd0 tx=0x7f5008006e40 comp rx=0 tx=0).stop 2024-06-06T04:19:49.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:48.679+0000 7f5024ed8700 1 -- 172.21.15.45:0/3015841868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5020106cb0 msgr2=0x7f502019bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:49.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:48.679+0000 7f5024ed8700 1 --2- 172.21.15.45:0/3015841868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5020106cb0 0x7f502019bcd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5010009e50 tx=0x7f5010004670 comp rx=0 tx=0).stop 2024-06-06T04:19:49.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:48.679+0000 7f5024ed8700 1 -- 172.21.15.45:0/3015841868 shutdown_connections 2024-06-06T04:19:49.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:48.679+0000 7f5024ed8700 1 --2- 172.21.15.45:0/3015841868 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f500c038fb0 0x7f500c03b470 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:49.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:48.679+0000 7f5024ed8700 1 --2- 172.21.15.45:0/3015841868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5020106cb0 0x7f502019bcd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:49.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:48.679+0000 7f5024ed8700 1 -- 172.21.15.45:0/3015841868 >> 172.21.15.45:0/3015841868 conn(0x7f5020102290 msgr2=0x7f5020103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:49.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:48.679+0000 7f5024ed8700 1 -- 172.21.15.45:0/3015841868 shutdown_connections 2024-06-06T04:19:49.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:48.679+0000 7f5024ed8700 1 -- 172.21.15.45:0/3015841868 wait complete. 2024-06-06T04:19:49.242 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mon service... 2024-06-06T04:19:49.742 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:49 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' 2024-06-06T04:19:49.743 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:49 smithi045 ceph-mon[32299]: Added host smithi045 2024-06-06T04:19:49.743 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:49 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:19:51.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:50 smithi045 ceph-mon[32299]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:19:51.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:50 smithi045 ceph-mon[32299]: Saving service mon spec with placement count:5 2024-06-06T04:19:51.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:50 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' 2024-06-06T04:19:51.503 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-06T04:19:51.503 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.732+0000 7f0b729f8700 1 Processor -- start 2024-06-06T04:19:51.503 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.732+0000 7f0b729f8700 1 -- start start 2024-06-06T04:19:51.503 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.732+0000 7f0b729f8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6c1088a0 0x7f0b6c10ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:51.503 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.732+0000 7f0b729f8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c07be20 con 0x7f0b6c1088a0 2024-06-06T04:19:51.504 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.733+0000 7f0b6bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6c1088a0 0x7f0b6c10ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:51.504 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.733+0000 7f0b6bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6c1088a0 0x7f0b6c10ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45370/0 (socket says 172.21.15.45:45370) 2024-06-06T04:19:51.504 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.733+0000 7f0b6bfff700 1 -- 172.21.15.45:0/1316711132 learned_addr learned my addr 172.21.15.45:0/1316711132 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:51.504 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.733+0000 7f0b6bfff700 1 -- 172.21.15.45:0/1316711132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c10b1d0 con 0x7f0b6c1088a0 2024-06-06T04:19:51.504 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.733+0000 7f0b6bfff700 1 --2- 172.21.15.45:0/1316711132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6c1088a0 0x7f0b6c10ac90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0098a0 tx=0x7f0b5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f8bdb2a0f9da6c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:51.504 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.733+0000 7f0b6affd700 1 -- 172.21.15.45:0/1316711132 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b5c004160 con 0x7f0b6c1088a0 2024-06-06T04:19:51.504 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.733+0000 7f0b6affd700 1 -- 172.21.15.45:0/1316711132 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0b5c004300 con 0x7f0b6c1088a0 2024-06-06T04:19:51.504 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.733+0000 7f0b6affd700 1 -- 172.21.15.45:0/1316711132 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b5c011a80 con 0x7f0b6c1088a0 2024-06-06T04:19:51.504 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.734+0000 7f0b729f8700 1 -- 172.21.15.45:0/1316711132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6c1088a0 msgr2=0x7f0b6c10ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:51.504 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.734+0000 7f0b729f8700 1 --2- 172.21.15.45:0/1316711132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6c1088a0 0x7f0b6c10ac90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0098a0 tx=0x7f0b5c009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:51.504 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.734+0000 7f0b729f8700 1 -- 172.21.15.45:0/1316711132 shutdown_connections 2024-06-06T04:19:51.504 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.734+0000 7f0b729f8700 1 --2- 172.21.15.45:0/1316711132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6c1088a0 0x7f0b6c10ac90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:51.505 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.734+0000 7f0b729f8700 1 -- 172.21.15.45:0/1316711132 >> 172.21.15.45:0/1316711132 conn(0x7f0b6c102290 msgr2=0x7f0b6c1046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:51.505 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.734+0000 7f0b729f8700 1 -- 172.21.15.45:0/1316711132 shutdown_connections 2024-06-06T04:19:51.505 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.734+0000 7f0b729f8700 1 -- 172.21.15.45:0/1316711132 wait complete. 2024-06-06T04:19:51.505 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.734+0000 7f0b729f8700 1 Processor -- start 2024-06-06T04:19:51.505 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.734+0000 7f0b729f8700 1 -- start start 2024-06-06T04:19:51.505 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.735+0000 7f0b729f8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6c1088a0 0x7f0b6c19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:51.505 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.735+0000 7f0b729f8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c10c130 con 0x7f0b6c1088a0 2024-06-06T04:19:51.505 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.735+0000 7f0b6bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6c1088a0 0x7f0b6c19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:51.505 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.735+0000 7f0b6bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6c1088a0 0x7f0b6c19bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45374/0 (socket says 172.21.15.45:45374) 2024-06-06T04:19:51.505 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.735+0000 7f0b6bfff700 1 -- 172.21.15.45:0/1763059126 learned_addr learned my addr 172.21.15.45:0/1763059126 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:51.505 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.735+0000 7f0b6bfff700 1 -- 172.21.15.45:0/1763059126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c19c1e0 con 0x7f0b6c1088a0 2024-06-06T04:19:51.505 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.735+0000 7f0b6bfff700 1 --2- 172.21.15.45:0/1763059126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6c1088a0 0x7f0b6c19bca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c011a50 tx=0x7f0b5c011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:51.505 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.735+0000 7f0b697fa700 1 -- 172.21.15.45:0/1763059126 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b5c003a80 con 0x7f0b6c1088a0 2024-06-06T04:19:51.505 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.736+0000 7f0b697fa700 1 -- 172.21.15.45:0/1763059126 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0b5c003c20 con 0x7f0b6c1088a0 2024-06-06T04:19:51.506 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.736+0000 7f0b729f8700 1 -- 172.21.15.45:0/1763059126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6c19c470 con 0x7f0b6c1088a0 2024-06-06T04:19:51.506 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.736+0000 7f0b697fa700 1 -- 172.21.15.45:0/1763059126 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b5c01a650 con 0x7f0b6c1088a0 2024-06-06T04:19:51.506 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.736+0000 7f0b729f8700 1 -- 172.21.15.45:0/1763059126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6c19f160 con 0x7f0b6c1088a0 2024-06-06T04:19:51.506 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.736+0000 7f0b697fa700 1 -- 172.21.15.45:0/1763059126 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0b5c01a900 con 0x7f0b6c1088a0 2024-06-06T04:19:51.506 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.737+0000 7f0b729f8700 1 -- 172.21.15.45:0/1763059126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b6c063af0 con 0x7f0b6c1088a0 2024-06-06T04:19:51.506 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.737+0000 7f0b697fa700 1 --2- 172.21.15.45:0/1763059126 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f0b540390a0 0x7f0b5403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:51.506 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.737+0000 7f0b697fa700 1 -- 172.21.15.45:0/1763059126 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0b5c04d6a0 con 0x7f0b6c1088a0 2024-06-06T04:19:51.506 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.737+0000 7f0b6b7fe700 1 --2- 172.21.15.45:0/1763059126 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f0b540390a0 0x7f0b5403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:51.506 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.737+0000 7f0b6b7fe700 1 --2- 172.21.15.45:0/1763059126 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f0b540390a0 0x7f0b5403b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0b60006fd0 tx=0x7f0b60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:51.506 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.742+0000 7f0b697fa700 1 -- 172.21.15.45:0/1763059126 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0b5c019c70 con 0x7f0b6c1088a0 2024-06-06T04:19:51.506 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.968+0000 7f0b729f8700 1 -- 172.21.15.45:0/1763059126 --> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0b6c103440 con 0x7f0b540390a0 2024-06-06T04:19:51.507 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.974+0000 7f0b697fa700 1 -- 172.21.15.45:0/1763059126 <== mgr.14120 v2:172.21.15.45:6800/3465796147 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0b6c103440 con 0x7f0b540390a0 2024-06-06T04:19:51.507 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.976+0000 7f0b729f8700 1 -- 172.21.15.45:0/1763059126 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f0b540390a0 msgr2=0x7f0b5403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:51.507 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.976+0000 7f0b729f8700 1 --2- 172.21.15.45:0/1763059126 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f0b540390a0 0x7f0b5403b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0b60006fd0 tx=0x7f0b60006e40 comp rx=0 tx=0).stop 2024-06-06T04:19:51.507 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.976+0000 7f0b729f8700 1 -- 172.21.15.45:0/1763059126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6c1088a0 msgr2=0x7f0b6c19bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:51.507 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.976+0000 7f0b729f8700 1 --2- 172.21.15.45:0/1763059126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6c1088a0 0x7f0b6c19bca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c011a50 tx=0x7f0b5c011b30 comp rx=0 tx=0).stop 2024-06-06T04:19:51.507 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.976+0000 7f0b729f8700 1 -- 172.21.15.45:0/1763059126 shutdown_connections 2024-06-06T04:19:51.507 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.976+0000 7f0b729f8700 1 --2- 172.21.15.45:0/1763059126 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f0b540390a0 0x7f0b5403b560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:51.507 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.976+0000 7f0b729f8700 1 --2- 172.21.15.45:0/1763059126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b6c1088a0 0x7f0b6c19bca0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:51.507 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.976+0000 7f0b729f8700 1 -- 172.21.15.45:0/1763059126 >> 172.21.15.45:0/1763059126 conn(0x7f0b6c102290 msgr2=0x7f0b6c102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:51.507 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.977+0000 7f0b729f8700 1 -- 172.21.15.45:0/1763059126 shutdown_connections 2024-06-06T04:19:51.507 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:49.977+0000 7f0b729f8700 1 -- 172.21.15.45:0/1763059126 wait complete. 2024-06-06T04:19:51.507 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mgr service... 2024-06-06T04:19:52.166 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:52 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' 2024-06-06T04:19:52.166 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:52 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' 2024-06-06T04:19:52.692 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-06T04:19:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.923+0000 7fb25f720700 1 Processor -- start 2024-06-06T04:19:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.923+0000 7fb25f720700 1 -- start start 2024-06-06T04:19:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.923+0000 7fb25f720700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb258107d80 0x7fb2581081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.923+0000 7fb25f720700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb258108770 con 0x7fb258107d80 2024-06-06T04:19:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.923+0000 7fb25d4bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb258107d80 0x7fb2581081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.923+0000 7fb25d4bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb258107d80 0x7fb2581081a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45376/0 (socket says 172.21.15.45:45376) 2024-06-06T04:19:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.923+0000 7fb25d4bc700 1 -- 172.21.15.45:0/602236747 learned_addr learned my addr 172.21.15.45:0/602236747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.924+0000 7fb25d4bc700 1 -- 172.21.15.45:0/602236747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb258108f80 con 0x7fb258107d80 2024-06-06T04:19:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.924+0000 7fb25d4bc700 1 --2- 172.21.15.45:0/602236747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb258107d80 0x7fb2581081a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb2540098a0 tx=0x7fb254009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef11db153e5d1cb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.924+0000 7fb24ffff700 1 -- 172.21.15.45:0/602236747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb254004160 con 0x7fb258107d80 2024-06-06T04:19:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.924+0000 7fb24ffff700 1 -- 172.21.15.45:0/602236747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb254004300 con 0x7fb258107d80 2024-06-06T04:19:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.924+0000 7fb24ffff700 1 -- 172.21.15.45:0/602236747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb254011a50 con 0x7fb258107d80 2024-06-06T04:19:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.925+0000 7fb25f720700 1 -- 172.21.15.45:0/602236747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb258107d80 msgr2=0x7fb2581081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.925+0000 7fb25f720700 1 --2- 172.21.15.45:0/602236747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb258107d80 0x7fb2581081a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb2540098a0 tx=0x7fb254009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.925+0000 7fb25f720700 1 -- 172.21.15.45:0/602236747 shutdown_connections 2024-06-06T04:19:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.925+0000 7fb25f720700 1 --2- 172.21.15.45:0/602236747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb258107d80 0x7fb2581081a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.925+0000 7fb25f720700 1 -- 172.21.15.45:0/602236747 >> 172.21.15.45:0/602236747 conn(0x7fb258103360 msgr2=0x7fb258105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.925+0000 7fb25f720700 1 -- 172.21.15.45:0/602236747 shutdown_connections 2024-06-06T04:19:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.925+0000 7fb25f720700 1 -- 172.21.15.45:0/602236747 wait complete. 2024-06-06T04:19:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.926+0000 7fb25f720700 1 Processor -- start 2024-06-06T04:19:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.926+0000 7fb25f720700 1 -- start start 2024-06-06T04:19:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.926+0000 7fb25f720700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb258107d80 0x7fb25807db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.926+0000 7fb25f720700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb25810c330 con 0x7fb258107d80 2024-06-06T04:19:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.926+0000 7fb25d4bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb258107d80 0x7fb25807db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.926+0000 7fb25d4bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb258107d80 0x7fb25807db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45390/0 (socket says 172.21.15.45:45390) 2024-06-06T04:19:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.926+0000 7fb25d4bc700 1 -- 172.21.15.45:0/3695801713 learned_addr learned my addr 172.21.15.45:0/3695801713 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.926+0000 7fb25d4bc700 1 -- 172.21.15.45:0/3695801713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25807e0a0 con 0x7fb258107d80 2024-06-06T04:19:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.926+0000 7fb25d4bc700 1 --2- 172.21.15.45:0/3695801713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb258107d80 0x7fb25807db60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb254011710 tx=0x7fb254011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.926+0000 7fb24e7fc700 1 -- 172.21.15.45:0/3695801713 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2540039c0 con 0x7fb258107d80 2024-06-06T04:19:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.927+0000 7fb24e7fc700 1 -- 172.21.15.45:0/3695801713 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb254003b60 con 0x7fb258107d80 2024-06-06T04:19:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.927+0000 7fb25f720700 1 -- 172.21.15.45:0/3695801713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25807c290 con 0x7fb258107d80 2024-06-06T04:19:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.927+0000 7fb24e7fc700 1 -- 172.21.15.45:0/3695801713 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb25401a690 con 0x7fb258107d80 2024-06-06T04:19:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.927+0000 7fb25f720700 1 -- 172.21.15.45:0/3695801713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb25807c7d0 con 0x7fb258107d80 2024-06-06T04:19:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.927+0000 7fb24e7fc700 1 -- 172.21.15.45:0/3695801713 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb25401a9c0 con 0x7fb258107d80 2024-06-06T04:19:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.928+0000 7fb24e7fc700 1 --2- 172.21.15.45:0/3695801713 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7fb244039050 0x7fb24403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.928+0000 7fb25f720700 1 -- 172.21.15.45:0/3695801713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb258063af0 con 0x7fb258107d80 2024-06-06T04:19:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.928+0000 7fb24e7fc700 1 -- 172.21.15.45:0/3695801713 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb25404d810 con 0x7fb258107d80 2024-06-06T04:19:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.928+0000 7fb25ccbb700 1 --2- 172.21.15.45:0/3695801713 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7fb244039050 0x7fb24403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.928+0000 7fb25ccbb700 1 --2- 172.21.15.45:0/3695801713 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7fb244039050 0x7fb24403b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb248006fd0 tx=0x7fb248006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:51.933+0000 7fb24e7fc700 1 -- 172.21.15.45:0/3695801713 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb254019c70 con 0x7fb258107d80 2024-06-06T04:19:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:52.157+0000 7fb25f720700 1 -- 172.21.15.45:0/3695801713 --> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fb25807d0e0 con 0x7fb244039050 2024-06-06T04:19:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:52.163+0000 7fb24e7fc700 1 -- 172.21.15.45:0/3695801713 <== mgr.14120 v2:172.21.15.45:6800/3465796147 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb25807d0e0 con 0x7fb244039050 2024-06-06T04:19:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:52.165+0000 7fb25f720700 1 -- 172.21.15.45:0/3695801713 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7fb244039050 msgr2=0x7fb24403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:52.165+0000 7fb25f720700 1 --2- 172.21.15.45:0/3695801713 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7fb244039050 0x7fb24403b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb248006fd0 tx=0x7fb248006e40 comp rx=0 tx=0).stop 2024-06-06T04:19:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:52.165+0000 7fb25f720700 1 -- 172.21.15.45:0/3695801713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb258107d80 msgr2=0x7fb25807db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:52.165+0000 7fb25f720700 1 --2- 172.21.15.45:0/3695801713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb258107d80 0x7fb25807db60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb254011710 tx=0x7fb254011ad0 comp rx=0 tx=0).stop 2024-06-06T04:19:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:52.166+0000 7fb25f720700 1 -- 172.21.15.45:0/3695801713 shutdown_connections 2024-06-06T04:19:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:52.166+0000 7fb25f720700 1 --2- 172.21.15.45:0/3695801713 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7fb244039050 0x7fb24403b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:52.166+0000 7fb25f720700 1 --2- 172.21.15.45:0/3695801713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb258107d80 0x7fb25807db60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:52.166+0000 7fb25f720700 1 -- 172.21.15.45:0/3695801713 >> 172.21.15.45:0/3695801713 conn(0x7fb258103360 msgr2=0x7fb258103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:52.166+0000 7fb25f720700 1 -- 172.21.15.45:0/3695801713 shutdown_connections 2024-06-06T04:19:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:52.166+0000 7fb25f720700 1 -- 172.21.15.45:0/3695801713 wait complete. 2024-06-06T04:19:53.356 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:53 smithi045 ceph-mon[32299]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:19:53.356 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:53 smithi045 ceph-mon[32299]: Saving service mgr spec with placement count:2 2024-06-06T04:19:53.356 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:53 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' 2024-06-06T04:19:54.446 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:54 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2900765356' entity='client.admin' 2024-06-06T04:19:54.446 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:54 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' 2024-06-06T04:19:54.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.119+0000 7f3622065700 1 Processor -- start 2024-06-06T04:19:54.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.119+0000 7f3622065700 1 -- start start 2024-06-06T04:19:54.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.119+0000 7f3622065700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361c07de70 0x7f361c07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:54.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.119+0000 7f3622065700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f361c07c8a0 con 0x7f361c07de70 2024-06-06T04:19:54.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.120+0000 7f361b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361c07de70 0x7f361c07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:54.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.120+0000 7f361b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361c07de70 0x7f361c07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45406/0 (socket says 172.21.15.45:45406) 2024-06-06T04:19:54.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.120+0000 7f361b7fe700 1 -- 172.21.15.45:0/2216399432 learned_addr learned my addr 172.21.15.45:0/2216399432 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:54.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.120+0000 7f361b7fe700 1 -- 172.21.15.45:0/2216399432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c07ca20 con 0x7f361c07de70 2024-06-06T04:19:54.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.120+0000 7f361b7fe700 1 --2- 172.21.15.45:0/2216399432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361c07de70 0x7f361c07c2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f360c0098a0 tx=0x7f360c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4a110640220ba8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:54.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.120+0000 7f361a7fc700 1 -- 172.21.15.45:0/2216399432 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f360c004160 con 0x7f361c07de70 2024-06-06T04:19:54.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.120+0000 7f361a7fc700 1 -- 172.21.15.45:0/2216399432 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f360c004300 con 0x7f361c07de70 2024-06-06T04:19:54.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.120+0000 7f361a7fc700 1 -- 172.21.15.45:0/2216399432 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f360c011a50 con 0x7f361c07de70 2024-06-06T04:19:54.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.121+0000 7f3622065700 1 -- 172.21.15.45:0/2216399432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361c07de70 msgr2=0x7f361c07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:54.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.121+0000 7f3622065700 1 --2- 172.21.15.45:0/2216399432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361c07de70 0x7f361c07c2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f360c0098a0 tx=0x7f360c009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:54.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.121+0000 7f3622065700 1 -- 172.21.15.45:0/2216399432 shutdown_connections 2024-06-06T04:19:54.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.121+0000 7f3622065700 1 --2- 172.21.15.45:0/2216399432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361c07de70 0x7f361c07c2d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:54.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.121+0000 7f3622065700 1 -- 172.21.15.45:0/2216399432 >> 172.21.15.45:0/2216399432 conn(0x7f361c103320 msgr2=0x7f361c105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:54.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.121+0000 7f3622065700 1 -- 172.21.15.45:0/2216399432 shutdown_connections 2024-06-06T04:19:54.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.121+0000 7f3622065700 1 -- 172.21.15.45:0/2216399432 wait complete. 2024-06-06T04:19:54.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.122+0000 7f3622065700 1 Processor -- start 2024-06-06T04:19:54.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.122+0000 7f3622065700 1 -- start start 2024-06-06T04:19:54.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.122+0000 7f3622065700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361c07de70 0x7f361c19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:54.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.122+0000 7f3622065700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f361c10e380 con 0x7f361c07de70 2024-06-06T04:19:54.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.123+0000 7f361b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361c07de70 0x7f361c19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:54.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.123+0000 7f361b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361c07de70 0x7f361c19de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45408/0 (socket says 172.21.15.45:45408) 2024-06-06T04:19:54.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.123+0000 7f361b7fe700 1 -- 172.21.15.45:0/2900765356 learned_addr learned my addr 172.21.15.45:0/2900765356 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:54.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.123+0000 7f361b7fe700 1 -- 172.21.15.45:0/2900765356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c19e390 con 0x7f361c07de70 2024-06-06T04:19:54.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.123+0000 7f361b7fe700 1 --2- 172.21.15.45:0/2900765356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361c07de70 0x7f361c19de50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f360c011a20 tx=0x7f360c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:54.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.123+0000 7f3618ff9700 1 -- 172.21.15.45:0/2900765356 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f360c0039c0 con 0x7f361c07de70 2024-06-06T04:19:54.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.123+0000 7f3622065700 1 -- 172.21.15.45:0/2900765356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361c19e620 con 0x7f361c07de70 2024-06-06T04:19:54.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.124+0000 7f3622065700 1 -- 172.21.15.45:0/2900765356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f361c1a1310 con 0x7f361c07de70 2024-06-06T04:19:54.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.124+0000 7f3618ff9700 1 -- 172.21.15.45:0/2900765356 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f360c003b60 con 0x7f361c07de70 2024-06-06T04:19:54.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.125+0000 7f3618ff9700 1 -- 172.21.15.45:0/2900765356 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f360c01a720 con 0x7f361c07de70 2024-06-06T04:19:54.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.125+0000 7f3618ff9700 1 -- 172.21.15.45:0/2900765356 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f360c01a980 con 0x7f361c07de70 2024-06-06T04:19:54.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.125+0000 7f3622065700 1 -- 172.21.15.45:0/2900765356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f361c063af0 con 0x7f361c07de70 2024-06-06T04:19:54.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.125+0000 7f3618ff9700 1 --2- 172.21.15.45:0/2900765356 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f3604038cb0 0x7f360403b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:54.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.125+0000 7f3618ff9700 1 -- 172.21.15.45:0/2900765356 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f360c04d780 con 0x7f361c07de70 2024-06-06T04:19:54.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.125+0000 7f361affd700 1 --2- 172.21.15.45:0/2900765356 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f3604038cb0 0x7f360403b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:54.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.126+0000 7f361affd700 1 --2- 172.21.15.45:0/2900765356 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f3604038cb0 0x7f360403b170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3610006fd0 tx=0x7f3610006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:54.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.130+0000 7f3618ff9700 1 -- 172.21.15.45:0/2900765356 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f360c019c70 con 0x7f361c07de70 2024-06-06T04:19:54.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.355+0000 7f3622065700 1 -- 172.21.15.45:0/2900765356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f361c1a1870 con 0x7f361c07de70 2024-06-06T04:19:54.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.357+0000 7f3618ff9700 1 -- 172.21.15.45:0/2900765356 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f360c00ba30 con 0x7f361c07de70 2024-06-06T04:19:54.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.360+0000 7f3622065700 1 -- 172.21.15.45:0/2900765356 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f3604038cb0 msgr2=0x7f360403b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:54.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.360+0000 7f3622065700 1 --2- 172.21.15.45:0/2900765356 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f3604038cb0 0x7f360403b170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3610006fd0 tx=0x7f3610006e40 comp rx=0 tx=0).stop 2024-06-06T04:19:54.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.360+0000 7f3622065700 1 -- 172.21.15.45:0/2900765356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361c07de70 msgr2=0x7f361c19de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:54.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.360+0000 7f3622065700 1 --2- 172.21.15.45:0/2900765356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361c07de70 0x7f361c19de50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f360c011a20 tx=0x7f360c011b00 comp rx=0 tx=0).stop 2024-06-06T04:19:54.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.360+0000 7f3622065700 1 -- 172.21.15.45:0/2900765356 shutdown_connections 2024-06-06T04:19:54.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.360+0000 7f3622065700 1 --2- 172.21.15.45:0/2900765356 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f3604038cb0 0x7f360403b170 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:54.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.360+0000 7f3622065700 1 --2- 172.21.15.45:0/2900765356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361c07de70 0x7f361c19de50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:54.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.360+0000 7f3622065700 1 -- 172.21.15.45:0/2900765356 >> 172.21.15.45:0/2900765356 conn(0x7f361c103320 msgr2=0x7f361c103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:54.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.360+0000 7f3622065700 1 -- 172.21.15.45:0/2900765356 shutdown_connections 2024-06-06T04:19:54.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:53.360+0000 7f3622065700 1 -- 172.21.15.45:0/2900765356 wait complete. 2024-06-06T04:19:55.735 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.876+0000 7f4f1de9f700 1 Processor -- start 2024-06-06T04:19:55.735 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.876+0000 7f4f1de9f700 1 -- start start 2024-06-06T04:19:55.735 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.876+0000 7f4f1de9f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18108840 0x7f4f1810ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:55.735 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.876+0000 7f4f1de9f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1807be20 con 0x7f4f18108840 2024-06-06T04:19:55.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.876+0000 7f4f177fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18108840 0x7f4f1810ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:55.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.876+0000 7f4f177fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18108840 0x7f4f1810ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45422/0 (socket says 172.21.15.45:45422) 2024-06-06T04:19:55.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.876+0000 7f4f177fe700 1 -- 172.21.15.45:0/3354335988 learned_addr learned my addr 172.21.15.45:0/3354335988 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:55.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.877+0000 7f4f177fe700 1 -- 172.21.15.45:0/3354335988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1810b170 con 0x7f4f18108840 2024-06-06T04:19:55.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.877+0000 7f4f177fe700 1 --2- 172.21.15.45:0/3354335988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18108840 0x7f4f1810ac30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800f960 tx=0x7f4f0800fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45b602ec4a55550 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:55.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.877+0000 7f4f167fc700 1 -- 172.21.15.45:0/3354335988 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f0801c070 con 0x7f4f18108840 2024-06-06T04:19:55.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.877+0000 7f4f167fc700 1 -- 172.21.15.45:0/3354335988 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f08004030 con 0x7f4f18108840 2024-06-06T04:19:55.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.877+0000 7f4f167fc700 1 -- 172.21.15.45:0/3354335988 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f08003c00 con 0x7f4f18108840 2024-06-06T04:19:55.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.878+0000 7f4f1de9f700 1 -- 172.21.15.45:0/3354335988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18108840 msgr2=0x7f4f1810ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:55.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.878+0000 7f4f1de9f700 1 --2- 172.21.15.45:0/3354335988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18108840 0x7f4f1810ac30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800f960 tx=0x7f4f0800fc70 comp rx=0 tx=0).stop 2024-06-06T04:19:55.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.878+0000 7f4f1de9f700 1 -- 172.21.15.45:0/3354335988 shutdown_connections 2024-06-06T04:19:55.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.878+0000 7f4f1de9f700 1 --2- 172.21.15.45:0/3354335988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18108840 0x7f4f1810ac30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:55.736 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.878+0000 7f4f1de9f700 1 -- 172.21.15.45:0/3354335988 >> 172.21.15.45:0/3354335988 conn(0x7f4f181021f0 msgr2=0x7f4f18104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:55.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.878+0000 7f4f1de9f700 1 -- 172.21.15.45:0/3354335988 shutdown_connections 2024-06-06T04:19:55.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.878+0000 7f4f1de9f700 1 -- 172.21.15.45:0/3354335988 wait complete. 2024-06-06T04:19:55.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.879+0000 7f4f1de9f700 1 Processor -- start 2024-06-06T04:19:55.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.879+0000 7f4f1de9f700 1 -- start start 2024-06-06T04:19:55.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.879+0000 7f4f1de9f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18108840 0x7f4f18199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:55.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.879+0000 7f4f1de9f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18107400 con 0x7f4f18108840 2024-06-06T04:19:55.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.879+0000 7f4f177fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18108840 0x7f4f18199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:55.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.879+0000 7f4f177fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18108840 0x7f4f18199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45426/0 (socket says 172.21.15.45:45426) 2024-06-06T04:19:55.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.879+0000 7f4f177fe700 1 -- 172.21.15.45:0/3594668278 learned_addr learned my addr 172.21.15.45:0/3594668278 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:55.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.879+0000 7f4f177fe700 1 -- 172.21.15.45:0/3594668278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f18199f80 con 0x7f4f18108840 2024-06-06T04:19:55.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.880+0000 7f4f177fe700 1 --2- 172.21.15.45:0/3594668278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18108840 0x7f4f18199a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800f5e0 tx=0x7f4f080044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:55.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.880+0000 7f4f14ff9700 1 -- 172.21.15.45:0/3594668278 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f0801c040 con 0x7f4f18108840 2024-06-06T04:19:55.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.880+0000 7f4f1de9f700 1 -- 172.21.15.45:0/3594668278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1819a210 con 0x7f4f18108840 2024-06-06T04:19:55.737 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.880+0000 7f4f14ff9700 1 -- 172.21.15.45:0/3594668278 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f08025070 con 0x7f4f18108840 2024-06-06T04:19:55.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.880+0000 7f4f14ff9700 1 -- 172.21.15.45:0/3594668278 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f080206a0 con 0x7f4f18108840 2024-06-06T04:19:55.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.880+0000 7f4f1de9f700 1 -- 172.21.15.45:0/3594668278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1819cf00 con 0x7f4f18108840 2024-06-06T04:19:55.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.881+0000 7f4f14ff9700 1 -- 172.21.15.45:0/3594668278 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4f08020950 con 0x7f4f18108840 2024-06-06T04:19:55.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.881+0000 7f4f1de9f700 1 -- 172.21.15.45:0/3594668278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f18063af0 con 0x7f4f18108840 2024-06-06T04:19:55.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.881+0000 7f4f14ff9700 1 --2- 172.21.15.45:0/3594668278 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f4f00039050 0x7f4f0003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:55.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.881+0000 7f4f14ff9700 1 -- 172.21.15.45:0/3594668278 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4f08014070 con 0x7f4f18108840 2024-06-06T04:19:55.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.882+0000 7f4f16ffd700 1 --2- 172.21.15.45:0/3594668278 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f4f00039050 0x7f4f0003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:55.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.882+0000 7f4f16ffd700 1 --2- 172.21.15.45:0/3594668278 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f4f00039050 0x7f4f0003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c006fd0 tx=0x7f4f0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:55.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:54.886+0000 7f4f14ff9700 1 -- 172.21.15.45:0/3594668278 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4f08025220 con 0x7f4f18108840 2024-06-06T04:19:55.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:55.093+0000 7f4f1de9f700 1 -- 172.21.15.45:0/3594668278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f4f1802dfa0 con 0x7f4f18108840 2024-06-06T04:19:55.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:55.095+0000 7f4f14ff9700 1 -- 172.21.15.45:0/3594668278 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f4f08027dd0 con 0x7f4f18108840 2024-06-06T04:19:55.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:55.098+0000 7f4f1de9f700 1 -- 172.21.15.45:0/3594668278 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f4f00039050 msgr2=0x7f4f0003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:55.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:55.098+0000 7f4f1de9f700 1 --2- 172.21.15.45:0/3594668278 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f4f00039050 0x7f4f0003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c006fd0 tx=0x7f4f0c006e40 comp rx=0 tx=0).stop 2024-06-06T04:19:55.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:55.098+0000 7f4f1de9f700 1 -- 172.21.15.45:0/3594668278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18108840 msgr2=0x7f4f18199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:55.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:55.098+0000 7f4f1de9f700 1 --2- 172.21.15.45:0/3594668278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18108840 0x7f4f18199a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800f5e0 tx=0x7f4f080044c0 comp rx=0 tx=0).stop 2024-06-06T04:19:55.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:55.098+0000 7f4f1de9f700 1 -- 172.21.15.45:0/3594668278 shutdown_connections 2024-06-06T04:19:55.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:55.098+0000 7f4f1de9f700 1 --2- 172.21.15.45:0/3594668278 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f4f00039050 0x7f4f0003b510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:55.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:55.098+0000 7f4f1de9f700 1 --2- 172.21.15.45:0/3594668278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f18108840 0x7f4f18199a40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:55.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:55.098+0000 7f4f1de9f700 1 -- 172.21.15.45:0/3594668278 >> 172.21.15.45:0/3594668278 conn(0x7f4f181021f0 msgr2=0x7f4f18102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:55.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:55.098+0000 7f4f1de9f700 1 -- 172.21.15.45:0/3594668278 shutdown_connections 2024-06-06T04:19:55.739 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:55.098+0000 7f4f1de9f700 1 -- 172.21.15.45:0/3594668278 wait complete. 2024-06-06T04:19:55.739 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling the dashboard module... 2024-06-06T04:19:56.397 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:56 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3594668278' entity='client.admin' 2024-06-06T04:19:56.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:56 smithi045 ceph-mon[32299]: from='mgr.14120 172.21.15.45:0/910569368' entity='mgr.x' 2024-06-06T04:19:57.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:57 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/317573542' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-06T04:19:57.329 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:57 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: ignoring --setuser ceph since I am not root 2024-06-06T04:19:57.329 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:57 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: ignoring --setgroup ceph since I am not root 2024-06-06T04:19:57.329 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:57 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:57.282+0000 7f12a8d19200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T04:19:57.584 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:57 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:57.476+0000 7f12a8d19200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T04:19:57.907 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:57 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:57.584+0000 7f12a8d19200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T04:19:57.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.159+0000 7f1997e84700 1 Processor -- start 2024-06-06T04:19:57.931 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.159+0000 7f1997e84700 1 -- start start 2024-06-06T04:19:57.931 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.159+0000 7f1997e84700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1990107dc0 0x7f19901081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:57.931 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.160+0000 7f1997e84700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19901087b0 con 0x7f1990107dc0 2024-06-06T04:19:57.931 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.160+0000 7f1995c20700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1990107dc0 0x7f19901081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:57.931 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.160+0000 7f1995c20700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1990107dc0 0x7f19901081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45442/0 (socket says 172.21.15.45:45442) 2024-06-06T04:19:57.931 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.160+0000 7f1995c20700 1 -- 172.21.15.45:0/3393924811 learned_addr learned my addr 172.21.15.45:0/3393924811 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:57.931 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.160+0000 7f1995c20700 1 -- 172.21.15.45:0/3393924811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1990108f20 con 0x7f1990107dc0 2024-06-06T04:19:57.931 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.160+0000 7f1995c20700 1 --2- 172.21.15.45:0/3393924811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1990107dc0 0x7f19901081e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f198c009e80 tx=0x7f198c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5daa1b291f47089 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:57.931 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.160+0000 7f1994c1e700 1 -- 172.21.15.45:0/3393924811 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f198c017070 con 0x7f1990107dc0 2024-06-06T04:19:57.931 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.160+0000 7f1994c1e700 1 -- 172.21.15.45:0/3393924811 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f198c00b8c0 con 0x7f1990107dc0 2024-06-06T04:19:57.931 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.160+0000 7f1994c1e700 1 -- 172.21.15.45:0/3393924811 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f198c004490 con 0x7f1990107dc0 2024-06-06T04:19:57.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.161+0000 7f1997e84700 1 -- 172.21.15.45:0/3393924811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1990107dc0 msgr2=0x7f19901081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:57.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.161+0000 7f1997e84700 1 --2- 172.21.15.45:0/3393924811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1990107dc0 0x7f19901081e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f198c009e80 tx=0x7f198c00b330 comp rx=0 tx=0).stop 2024-06-06T04:19:57.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.161+0000 7f1997e84700 1 -- 172.21.15.45:0/3393924811 shutdown_connections 2024-06-06T04:19:57.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.161+0000 7f1997e84700 1 --2- 172.21.15.45:0/3393924811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1990107dc0 0x7f19901081e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:57.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.161+0000 7f1997e84700 1 -- 172.21.15.45:0/3393924811 >> 172.21.15.45:0/3393924811 conn(0x7f1990103340 msgr2=0x7f19901057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:57.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.161+0000 7f1997e84700 1 -- 172.21.15.45:0/3393924811 shutdown_connections 2024-06-06T04:19:57.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.161+0000 7f1997e84700 1 -- 172.21.15.45:0/3393924811 wait complete. 2024-06-06T04:19:57.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.161+0000 7f1997e84700 1 Processor -- start 2024-06-06T04:19:57.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.162+0000 7f1997e84700 1 -- start start 2024-06-06T04:19:57.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.162+0000 7f1997e84700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1990107dc0 0x7f199007db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:57.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.162+0000 7f1997e84700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f199010c2d0 con 0x7f1990107dc0 2024-06-06T04:19:57.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.162+0000 7f1995c20700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1990107dc0 0x7f199007db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:57.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.162+0000 7f1995c20700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1990107dc0 0x7f199007db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45454/0 (socket says 172.21.15.45:45454) 2024-06-06T04:19:57.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.162+0000 7f1995c20700 1 -- 172.21.15.45:0/317573542 learned_addr learned my addr 172.21.15.45:0/317573542 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:57.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.162+0000 7f1995c20700 1 -- 172.21.15.45:0/317573542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199007e0a0 con 0x7f1990107dc0 2024-06-06T04:19:57.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.162+0000 7f1995c20700 1 --2- 172.21.15.45:0/317573542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1990107dc0 0x7f199007db60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f198c003c50 tx=0x7f198c003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:57.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.162+0000 7f1986ffd700 1 -- 172.21.15.45:0/317573542 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f198c020020 con 0x7f1990107dc0 2024-06-06T04:19:57.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.162+0000 7f1986ffd700 1 -- 172.21.15.45:0/317573542 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f198c00bc60 con 0x7f1990107dc0 2024-06-06T04:19:57.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.162+0000 7f1997e84700 1 -- 172.21.15.45:0/317573542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199007c290 con 0x7f1990107dc0 2024-06-06T04:19:57.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.162+0000 7f1986ffd700 1 -- 172.21.15.45:0/317573542 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f198c017040 con 0x7f1990107dc0 2024-06-06T04:19:57.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.162+0000 7f1997e84700 1 -- 172.21.15.45:0/317573542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199007c7d0 con 0x7f1990107dc0 2024-06-06T04:19:57.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.163+0000 7f1986ffd700 1 -- 172.21.15.45:0/317573542 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f198c023020 con 0x7f1990107dc0 2024-06-06T04:19:57.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.163+0000 7f1997e84700 1 -- 172.21.15.45:0/317573542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1990063af0 con 0x7f1990107dc0 2024-06-06T04:19:57.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.163+0000 7f1986ffd700 1 --2- 172.21.15.45:0/317573542 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f197c0390a0 0x7f197c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:57.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.164+0000 7f1986ffd700 1 -- 172.21.15.45:0/317573542 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f198c00f050 con 0x7f1990107dc0 2024-06-06T04:19:57.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.164+0000 7f199541f700 1 --2- 172.21.15.45:0/317573542 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f197c0390a0 0x7f197c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:57.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.164+0000 7f199541f700 1 --2- 172.21.15.45:0/317573542 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f197c0390a0 0x7f197c03b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1980006fd0 tx=0x7f1980006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:57.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.169+0000 7f1986ffd700 1 -- 172.21.15.45:0/317573542 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f198c02a020 con 0x7f1990107dc0 2024-06-06T04:19:57.934 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:56.449+0000 7f1997e84700 1 -- 172.21.15.45:0/317573542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f199007cf30 con 0x7f1990107dc0 2024-06-06T04:19:57.934 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:57.097+0000 7f1986ffd700 1 -- 172.21.15.45:0/317573542 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f198c02b260 con 0x7f1990107dc0 2024-06-06T04:19:57.934 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:57.099+0000 7f1986ffd700 1 -- 172.21.15.45:0/317573542 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f198c02d350 con 0x7f1990107dc0 2024-06-06T04:19:57.934 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:57.100+0000 7f1997e84700 1 -- 172.21.15.45:0/317573542 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f197c0390a0 msgr2=0x7f197c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:57.934 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:57.100+0000 7f1997e84700 1 --2- 172.21.15.45:0/317573542 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f197c0390a0 0x7f197c03b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1980006fd0 tx=0x7f1980006e40 comp rx=0 tx=0).stop 2024-06-06T04:19:57.934 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:57.100+0000 7f1997e84700 1 -- 172.21.15.45:0/317573542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1990107dc0 msgr2=0x7f199007db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:57.934 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:57.100+0000 7f1997e84700 1 --2- 172.21.15.45:0/317573542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1990107dc0 0x7f199007db60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f198c003c50 tx=0x7f198c003d30 comp rx=0 tx=0).stop 2024-06-06T04:19:57.934 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:57.100+0000 7f1997e84700 1 -- 172.21.15.45:0/317573542 shutdown_connections 2024-06-06T04:19:57.934 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:57.100+0000 7f1997e84700 1 --2- 172.21.15.45:0/317573542 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f197c0390a0 0x7f197c03b560 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:57.934 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:57.100+0000 7f1997e84700 1 --2- 172.21.15.45:0/317573542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1990107dc0 0x7f199007db60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:57.934 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:57.100+0000 7f1997e84700 1 -- 172.21.15.45:0/317573542 >> 172.21.15.45:0/317573542 conn(0x7f1990103340 msgr2=0x7f1990103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:57.934 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:57.100+0000 7f1997e84700 1 -- 172.21.15.45:0/317573542 shutdown_connections 2024-06-06T04:19:57.934 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:57.101+0000 7f1997e84700 1 -- 172.21.15.45:0/317573542 wait complete. 2024-06-06T04:19:58.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:58 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/317573542' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-06T04:19:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:58 smithi045 ceph-mon[32299]: mgrmap e9: x(active, since 20s) 2024-06-06T04:19:58.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:57 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:57.907+0000 7f12a8d19200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T04:19:58.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:58 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:58.000+0000 7f12a8d19200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T04:19:59.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:19:59 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/249898479' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-06T04:19:59.346 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-06-06T04:19:59.346 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-06T04:19:59.346 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-06T04:19:59.346 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-06T04:19:59.346 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-06T04:19:59.347 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-06-06T04:19:59.347 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.338+0000 7f4e5fa89700 1 Processor -- start 2024-06-06T04:19:59.347 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.338+0000 7f4e5fa89700 1 -- start start 2024-06-06T04:19:59.347 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.338+0000 7f4e5fa89700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e58107df0 0x7f4e58108210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:59.347 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.338+0000 7f4e5fa89700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581087e0 con 0x7f4e58107df0 2024-06-06T04:19:59.347 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.338+0000 7f4e5d825700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e58107df0 0x7f4e58108210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:59.347 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.338+0000 7f4e5d825700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e58107df0 0x7f4e58108210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34334/0 (socket says 172.21.15.45:34334) 2024-06-06T04:19:59.347 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.338+0000 7f4e5d825700 1 -- 172.21.15.45:0/692334489 learned_addr learned my addr 172.21.15.45:0/692334489 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:59.347 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.339+0000 7f4e5d825700 1 -- 172.21.15.45:0/692334489 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e58108f50 con 0x7f4e58107df0 2024-06-06T04:19:59.347 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.339+0000 7f4e5d825700 1 --2- 172.21.15.45:0/692334489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e58107df0 0x7f4e58108210 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4e540098a0 tx=0x7f4e54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71d727ca01c2a38 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:59.347 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.339+0000 7f4e5c823700 1 -- 172.21.15.45:0/692334489 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e54004160 con 0x7f4e58107df0 2024-06-06T04:19:59.347 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.339+0000 7f4e5c823700 1 -- 172.21.15.45:0/692334489 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e54004300 con 0x7f4e58107df0 2024-06-06T04:19:59.347 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.339+0000 7f4e5c823700 1 -- 172.21.15.45:0/692334489 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e54011a50 con 0x7f4e58107df0 2024-06-06T04:19:59.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.340+0000 7f4e5fa89700 1 -- 172.21.15.45:0/692334489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e58107df0 msgr2=0x7f4e58108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:59.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.340+0000 7f4e5fa89700 1 --2- 172.21.15.45:0/692334489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e58107df0 0x7f4e58108210 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4e540098a0 tx=0x7f4e54009bb0 comp rx=0 tx=0).stop 2024-06-06T04:19:59.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.340+0000 7f4e5fa89700 1 -- 172.21.15.45:0/692334489 shutdown_connections 2024-06-06T04:19:59.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.340+0000 7f4e5fa89700 1 --2- 172.21.15.45:0/692334489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e58107df0 0x7f4e58108210 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:59.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.340+0000 7f4e5fa89700 1 -- 172.21.15.45:0/692334489 >> 172.21.15.45:0/692334489 conn(0x7f4e581033d0 msgr2=0x7f4e581057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:59.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.340+0000 7f4e5fa89700 1 -- 172.21.15.45:0/692334489 shutdown_connections 2024-06-06T04:19:59.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.340+0000 7f4e5fa89700 1 -- 172.21.15.45:0/692334489 wait complete. 2024-06-06T04:19:59.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.341+0000 7f4e5fa89700 1 Processor -- start 2024-06-06T04:19:59.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.341+0000 7f4e5fa89700 1 -- start start 2024-06-06T04:19:59.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.341+0000 7f4e5fa89700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e58107df0 0x7f4e5807db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:59.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.341+0000 7f4e5fa89700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5810c300 con 0x7f4e58107df0 2024-06-06T04:19:59.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.341+0000 7f4e5d825700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e58107df0 0x7f4e5807db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:19:59.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.341+0000 7f4e5d825700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e58107df0 0x7f4e5807db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34346/0 (socket says 172.21.15.45:34346) 2024-06-06T04:19:59.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.341+0000 7f4e5d825700 1 -- 172.21.15.45:0/249898479 learned_addr learned my addr 172.21.15.45:0/249898479 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:19:59.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.341+0000 7f4e5d825700 1 -- 172.21.15.45:0/249898479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5807e0a0 con 0x7f4e58107df0 2024-06-06T04:19:59.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.342+0000 7f4e5d825700 1 --2- 172.21.15.45:0/249898479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e58107df0 0x7f4e5807db60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e54006ba0 tx=0x7f4e54011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:19:59.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.342+0000 7f4e4effd700 1 -- 172.21.15.45:0/249898479 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e54003af0 con 0x7f4e58107df0 2024-06-06T04:19:59.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.342+0000 7f4e4effd700 1 -- 172.21.15.45:0/249898479 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e5401a8e0 con 0x7f4e58107df0 2024-06-06T04:19:59.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.342+0000 7f4e4effd700 1 -- 172.21.15.45:0/249898479 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e54024900 con 0x7f4e58107df0 2024-06-06T04:19:59.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.342+0000 7f4e5fa89700 1 -- 172.21.15.45:0/249898479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5807c290 con 0x7f4e58107df0 2024-06-06T04:19:59.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.342+0000 7f4e5fa89700 1 -- 172.21.15.45:0/249898479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e5807c7d0 con 0x7f4e58107df0 2024-06-06T04:19:59.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.343+0000 7f4e4effd700 1 -- 172.21.15.45:0/249898479 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f4e54022020 con 0x7f4e58107df0 2024-06-06T04:19:59.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.343+0000 7f4e5fa89700 1 -- 172.21.15.45:0/249898479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e58063af0 con 0x7f4e58107df0 2024-06-06T04:19:59.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.343+0000 7f4e4effd700 1 --2- 172.21.15.45:0/249898479 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f4e440390b0 0x7f4e4403b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:19:59.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.343+0000 7f4e4effd700 1 -- 172.21.15.45:0/249898479 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4e5404d510 con 0x7f4e58107df0 2024-06-06T04:19:59.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.343+0000 7f4e5d024700 1 -- 172.21.15.45:0/249898479 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f4e440390b0 msgr2=0x7f4e4403b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3465796147 2024-06-06T04:19:59.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.343+0000 7f4e5d024700 1 --2- 172.21.15.45:0/249898479 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f4e440390b0 0x7f4e4403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T04:19:59.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.348+0000 7f4e4effd700 1 -- 172.21.15.45:0/249898479 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4e54020c10 con 0x7f4e58107df0 2024-06-06T04:19:59.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.544+0000 7f4e5d024700 1 -- 172.21.15.45:0/249898479 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f4e440390b0 msgr2=0x7f4e4403b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3465796147 2024-06-06T04:19:59.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.544+0000 7f4e5d024700 1 --2- 172.21.15.45:0/249898479 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f4e440390b0 0x7f4e4403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T04:19:59.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.665+0000 7f4e5fa89700 1 -- 172.21.15.45:0/249898479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f4e5807d070 con 0x7f4e58107df0 2024-06-06T04:19:59.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.666+0000 7f4e4effd700 1 -- 172.21.15.45:0/249898479 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f4e54021d80 con 0x7f4e58107df0 2024-06-06T04:19:59.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.668+0000 7f4e5fa89700 1 -- 172.21.15.45:0/249898479 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f4e440390b0 msgr2=0x7f4e4403b570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T04:19:59.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.668+0000 7f4e5fa89700 1 --2- 172.21.15.45:0/249898479 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f4e440390b0 0x7f4e4403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:59.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.668+0000 7f4e5fa89700 1 -- 172.21.15.45:0/249898479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e58107df0 msgr2=0x7f4e5807db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:19:59.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.668+0000 7f4e5fa89700 1 --2- 172.21.15.45:0/249898479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e58107df0 0x7f4e5807db60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e54006ba0 tx=0x7f4e54011ad0 comp rx=0 tx=0).stop 2024-06-06T04:19:59.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.668+0000 7f4e5fa89700 1 -- 172.21.15.45:0/249898479 shutdown_connections 2024-06-06T04:19:59.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.668+0000 7f4e5fa89700 1 --2- 172.21.15.45:0/249898479 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f4e440390b0 0x7f4e4403b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:59.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.668+0000 7f4e5fa89700 1 --2- 172.21.15.45:0/249898479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e58107df0 0x7f4e5807db60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:19:59.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.668+0000 7f4e5fa89700 1 -- 172.21.15.45:0/249898479 >> 172.21.15.45:0/249898479 conn(0x7f4e581033d0 msgr2=0x7f4e58103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:19:59.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.668+0000 7f4e5fa89700 1 -- 172.21.15.45:0/249898479 shutdown_connections 2024-06-06T04:19:59.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:58.668+0000 7f4e5fa89700 1 -- 172.21.15.45:0/249898479 wait complete. 2024-06-06T04:19:59.351 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-06-06T04:19:59.351 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 9... 2024-06-06T04:19:59.575 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:59 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:59.264+0000 7f12a8d19200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T04:19:59.575 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:59 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:59.445+0000 7f12a8d19200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T04:19:59.937 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:59 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:59.628+0000 7f12a8d19200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T04:19:59.937 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:19:59 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:19:59.936+0000 7f12a8d19200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T04:20:00.226 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:20:00 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:20:00.034+0000 7f12a8d19200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T04:20:00.727 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:20:00 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:20:00.245+0000 7f12a8d19200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T04:20:00.727 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:20:00 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:20:00.447+0000 7f12a8d19200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T04:20:01.476 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:20:01 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:20:01.078+0000 7f12a8d19200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T04:20:01.832 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:20:01 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:20:01.519+0000 7f12a8d19200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T04:20:01.832 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:20:01 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:20:01.620+0000 7f12a8d19200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T04:20:01.832 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:20:01 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:20:01.831+0000 7f12a8d19200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T04:20:02.726 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:20:02 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:20:02.376+0000 7f12a8d19200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T04:20:02.727 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:20:02 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:20:02.473+0000 7f12a8d19200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T04:20:03.092 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:20:02 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:20:02.728+0000 7f12a8d19200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T04:20:03.093 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:20:02 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:20:02.868+0000 7f12a8d19200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T04:20:03.093 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:20:02 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:20:02.970+0000 7f12a8d19200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T04:20:03.093 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:20:03 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:20:03.092+0000 7f12a8d19200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T04:20:03.726 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:20:03 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:20:03.403+0000 7f12a8d19200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T04:20:04.318 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:20:04 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:20:04.039+0000 7f12a8d19200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T04:20:04.318 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:20:04 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:20:04.265+0000 7f12a8d19200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T04:20:04.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:04 smithi045 ceph-mon[32299]: Active manager daemon x restarted 2024-06-06T04:20:04.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:04 smithi045 ceph-mon[32299]: Activating manager daemon x 2024-06-06T04:20:04.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:04 smithi045 ceph-mon[32299]: osdmap e3: 0 total, 0 up, 0 in 2024-06-06T04:20:04.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:04 smithi045 ceph-mon[32299]: mgrmap e10: x(active, starting, since 0.00329182s) 2024-06-06T04:20:04.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T04:20:04.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T04:20:04.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T04:20:04.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T04:20:04.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T04:20:04.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:04 smithi045 ceph-mon[32299]: Manager daemon x is now available 2024-06-06T04:20:04.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:20:04.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T04:20:05.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T04:20:05.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:05 smithi045 ceph-mon[32299]: mgrmap e11: x(active, since 1.00727s) 2024-06-06T04:20:06.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-06-06T04:20:06.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-06T04:20:06.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-06T04:20:06.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-06-06T04:20:06.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.786+0000 7f414fff5700 1 Processor -- start 2024-06-06T04:20:06.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.786+0000 7f414fff5700 1 -- start start 2024-06-06T04:20:06.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.786+0000 7f414fff5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41481082f0 0x7f4148108710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:06.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.786+0000 7f414fff5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4148108ce0 con 0x7f41481082f0 2024-06-06T04:20:06.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.787+0000 7f414dd91700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41481082f0 0x7f4148108710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:06.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.787+0000 7f414dd91700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41481082f0 0x7f4148108710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34356/0 (socket says 172.21.15.45:34356) 2024-06-06T04:20:06.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.787+0000 7f414dd91700 1 -- 172.21.15.45:0/3461442452 learned_addr learned my addr 172.21.15.45:0/3461442452 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:06.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.787+0000 7f414dd91700 1 -- 172.21.15.45:0/3461442452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4148106ec0 con 0x7f41481082f0 2024-06-06T04:20:06.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.787+0000 7f414dd91700 1 --2- 172.21.15.45:0/3461442452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41481082f0 0x7f4148108710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f414400bb40 tx=0x7f414400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674100de1c3d8c54 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:06.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.787+0000 7f414cd8f700 1 -- 172.21.15.45:0/3461442452 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f414400dbe0 con 0x7f41481082f0 2024-06-06T04:20:06.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.787+0000 7f414cd8f700 1 -- 172.21.15.45:0/3461442452 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41440044e0 con 0x7f41481082f0 2024-06-06T04:20:06.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.787+0000 7f414cd8f700 1 -- 172.21.15.45:0/3461442452 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4144010470 con 0x7f41481082f0 2024-06-06T04:20:06.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.788+0000 7f414fff5700 1 -- 172.21.15.45:0/3461442452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41481082f0 msgr2=0x7f4148108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:06.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.788+0000 7f414fff5700 1 --2- 172.21.15.45:0/3461442452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41481082f0 0x7f4148108710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f414400bb40 tx=0x7f414400be50 comp rx=0 tx=0).stop 2024-06-06T04:20:06.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.788+0000 7f414fff5700 1 -- 172.21.15.45:0/3461442452 shutdown_connections 2024-06-06T04:20:06.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.788+0000 7f414fff5700 1 --2- 172.21.15.45:0/3461442452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41481082f0 0x7f4148108710 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:06.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.788+0000 7f414fff5700 1 -- 172.21.15.45:0/3461442452 >> 172.21.15.45:0/3461442452 conn(0x7f41481019f0 msgr2=0x7f4148103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:06.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.789+0000 7f414fff5700 1 -- 172.21.15.45:0/3461442452 shutdown_connections 2024-06-06T04:20:06.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.789+0000 7f414fff5700 1 -- 172.21.15.45:0/3461442452 wait complete. 2024-06-06T04:20:06.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.789+0000 7f414fff5700 1 Processor -- start 2024-06-06T04:20:06.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.789+0000 7f414fff5700 1 -- start start 2024-06-06T04:20:06.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.790+0000 7f414fff5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41481082f0 0x7f4148199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:06.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.790+0000 7f414fff5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4148105db0 con 0x7f41481082f0 2024-06-06T04:20:06.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.790+0000 7f414dd91700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41481082f0 0x7f4148199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:06.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.790+0000 7f414dd91700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41481082f0 0x7f4148199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34366/0 (socket says 172.21.15.45:34366) 2024-06-06T04:20:06.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.790+0000 7f414dd91700 1 -- 172.21.15.45:0/1962346036 learned_addr learned my addr 172.21.15.45:0/1962346036 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:06.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.790+0000 7f414dd91700 1 -- 172.21.15.45:0/1962346036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414819a0d0 con 0x7f41481082f0 2024-06-06T04:20:06.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.790+0000 7f414dd91700 1 --2- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41481082f0 0x7f4148199b90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f414400d650 tx=0x7f4144003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:06.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.790+0000 7f413effd700 1 -- 172.21.15.45:0/1962346036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4144027070 con 0x7f41481082f0 2024-06-06T04:20:06.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.790+0000 7f413effd700 1 -- 172.21.15.45:0/1962346036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4144010d50 con 0x7f41481082f0 2024-06-06T04:20:06.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.790+0000 7f414fff5700 1 -- 172.21.15.45:0/1962346036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414819a360 con 0x7f41481082f0 2024-06-06T04:20:06.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.791+0000 7f414fff5700 1 -- 172.21.15.45:0/1962346036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f414819a7c0 con 0x7f41481082f0 2024-06-06T04:20:06.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.791+0000 7f413effd700 1 -- 172.21.15.45:0/1962346036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4144022c00 con 0x7f41481082f0 2024-06-06T04:20:06.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.791+0000 7f413effd700 1 -- 172.21.15.45:0/1962346036 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f414402a070 con 0x7f41481082f0 2024-06-06T04:20:06.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.792+0000 7f413effd700 1 --2- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f41340390b0 0x7f413403b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:06.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.792+0000 7f413effd700 1 -- 172.21.15.45:0/1962346036 --> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f413403bc80 con 0x7f41340390b0 2024-06-06T04:20:06.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.792+0000 7f414d590700 1 -- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f41340390b0 msgr2=0x7f413403b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3465796147 2024-06-06T04:20:06.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.792+0000 7f413effd700 1 -- 172.21.15.45:0/1962346036 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f414401f070 con 0x7f41481082f0 2024-06-06T04:20:06.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.792+0000 7f414d590700 1 --2- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f41340390b0 0x7f413403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T04:20:06.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.993+0000 7f414d590700 1 -- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f41340390b0 msgr2=0x7f413403b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3465796147 2024-06-06T04:20:06.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:19:59.993+0000 7f414d590700 1 --2- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f41340390b0 0x7f413403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T04:20:06.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:00.394+0000 7f414d590700 1 -- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f41340390b0 msgr2=0x7f413403b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3465796147 2024-06-06T04:20:06.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:00.394+0000 7f414d590700 1 --2- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f41340390b0 0x7f413403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-06T04:20:06.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:01.195+0000 7f414d590700 1 -- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f41340390b0 msgr2=0x7f413403b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3465796147 2024-06-06T04:20:06.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:01.195+0000 7f414d590700 1 --2- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f41340390b0 0x7f413403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-06T04:20:06.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:02.796+0000 7f414d590700 1 -- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f41340390b0 msgr2=0x7f413403b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3465796147 2024-06-06T04:20:06.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:02.796+0000 7f414d590700 1 --2- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f41340390b0 0x7f413403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-06T04:20:06.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:04.268+0000 7f413effd700 1 -- 172.21.15.45:0/1962346036 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f414402cce0 con 0x7f41481082f0 2024-06-06T04:20:06.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:04.268+0000 7f413effd700 1 -- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f41340390b0 msgr2=0x7f413403b570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T04:20:06.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:04.268+0000 7f413effd700 1 --2- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f41340390b0 0x7f413403b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:06.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.270+0000 7f413effd700 1 -- 172.21.15.45:0/1962346036 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f4144051560 con 0x7f41481082f0 2024-06-06T04:20:06.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.270+0000 7f413effd700 1 --2- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f413403cb70 0x7f413403ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:06.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.270+0000 7f413effd700 1 -- 172.21.15.45:0/1962346036 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f413403bc80 con 0x7f413403cb70 2024-06-06T04:20:06.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.271+0000 7f414d590700 1 --2- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f413403cb70 0x7f413403ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:06.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.272+0000 7f414d590700 1 --2- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f413403cb70 0x7f413403ef60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4138003a10 tx=0x7f4138009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:06.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.272+0000 7f413effd700 1 -- 172.21.15.45:0/1962346036 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f413403bc80 con 0x7f413403cb70 2024-06-06T04:20:06.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.280+0000 7f414fff5700 1 -- 172.21.15.45:0/1962346036 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f4148063af0 con 0x7f413403cb70 2024-06-06T04:20:06.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.280+0000 7f413effd700 1 -- 172.21.15.45:0/1962346036 <== mgr.14152 v2:172.21.15.45:6800/1718508550 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f4148063af0 con 0x7f413403cb70 2024-06-06T04:20:06.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.280+0000 7f414fff5700 1 -- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f413403cb70 msgr2=0x7f413403ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:06.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.280+0000 7f414fff5700 1 --2- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f413403cb70 0x7f413403ef60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4138003a10 tx=0x7f4138009270 comp rx=0 tx=0).stop 2024-06-06T04:20:06.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.280+0000 7f414fff5700 1 -- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41481082f0 msgr2=0x7f4148199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:06.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.280+0000 7f414fff5700 1 --2- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41481082f0 0x7f4148199b90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f414400d650 tx=0x7f4144003fb0 comp rx=0 tx=0).stop 2024-06-06T04:20:06.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.280+0000 7f414fff5700 1 -- 172.21.15.45:0/1962346036 shutdown_connections 2024-06-06T04:20:06.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.280+0000 7f414fff5700 1 --2- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f413403cb70 0x7f413403ef60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:06.212 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.280+0000 7f414fff5700 1 --2- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41481082f0 0x7f4148199b90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:06.212 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.280+0000 7f414fff5700 1 --2- 172.21.15.45:0/1962346036 >> [v2:172.21.15.45:6800/3465796147,v1:172.21.15.45:6801/3465796147] conn(0x7f41340390b0 0x7f413403b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:06.212 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.280+0000 7f414fff5700 1 -- 172.21.15.45:0/1962346036 >> 172.21.15.45:0/1962346036 conn(0x7f41481019f0 msgr2=0x7f4148190250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:06.212 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.280+0000 7f414fff5700 1 -- 172.21.15.45:0/1962346036 shutdown_connections 2024-06-06T04:20:06.212 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:05.281+0000 7f414fff5700 1 -- 172.21.15.45:0/1962346036 wait complete. 2024-06-06T04:20:06.212 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 9 is available 2024-06-06T04:20:06.212 INFO:teuthology.orchestra.run.smithi045.stdout:Generating a dashboard self-signed certificate... 2024-06-06T04:20:06.874 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:06 smithi045 ceph-mon[32299]: [06/Jun/2024:04:20:05] ENGINE Bus STARTING 2024-06-06T04:20:06.874 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:06 smithi045 ceph-mon[32299]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-06T04:20:06.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:06 smithi045 ceph-mon[32299]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-06T04:20:06.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:06 smithi045 ceph-mon[32299]: [06/Jun/2024:04:20:05] ENGINE Serving on https://172.21.15.45:7150 2024-06-06T04:20:06.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:06 smithi045 ceph-mon[32299]: [06/Jun/2024:04:20:05] ENGINE Serving on http://172.21.15.45:8765 2024-06-06T04:20:06.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:06 smithi045 ceph-mon[32299]: [06/Jun/2024:04:20:05] ENGINE Bus STARTED 2024-06-06T04:20:06.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:06 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-06T04:20:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.639+0000 7ffa7e25c700 1 Processor -- start 2024-06-06T04:20:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.639+0000 7ffa7e25c700 1 -- start start 2024-06-06T04:20:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.640+0000 7ffa7e25c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa78106610 0x7ffa78106a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.640+0000 7ffa7e25c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7807be40 con 0x7ffa78106610 2024-06-06T04:20:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.640+0000 7ffa777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa78106610 0x7ffa78106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.640+0000 7ffa777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa78106610 0x7ffa78106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34478/0 (socket says 172.21.15.45:34478) 2024-06-06T04:20:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.640+0000 7ffa777fe700 1 -- 172.21.15.45:0/1742073556 learned_addr learned my addr 172.21.15.45:0/1742073556 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.640+0000 7ffa777fe700 1 -- 172.21.15.45:0/1742073556 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa78106f70 con 0x7ffa78106610 2024-06-06T04:20:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.640+0000 7ffa777fe700 1 --2- 172.21.15.45:0/1742073556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa78106610 0x7ffa78106a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffa680098a0 tx=0x7ffa68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf39144b7beab45 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.640+0000 7ffa767fc700 1 -- 172.21.15.45:0/1742073556 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa68004160 con 0x7ffa78106610 2024-06-06T04:20:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.641+0000 7ffa767fc700 1 -- 172.21.15.45:0/1742073556 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa68004300 con 0x7ffa78106610 2024-06-06T04:20:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.641+0000 7ffa767fc700 1 -- 172.21.15.45:0/1742073556 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa68011a50 con 0x7ffa78106610 2024-06-06T04:20:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.641+0000 7ffa7e25c700 1 -- 172.21.15.45:0/1742073556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa78106610 msgr2=0x7ffa78106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.641+0000 7ffa7e25c700 1 --2- 172.21.15.45:0/1742073556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa78106610 0x7ffa78106a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffa680098a0 tx=0x7ffa68009bb0 comp rx=0 tx=0).stop 2024-06-06T04:20:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.641+0000 7ffa7e25c700 1 -- 172.21.15.45:0/1742073556 shutdown_connections 2024-06-06T04:20:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.641+0000 7ffa7e25c700 1 --2- 172.21.15.45:0/1742073556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa78106610 0x7ffa78106a30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.641+0000 7ffa7e25c700 1 -- 172.21.15.45:0/1742073556 >> 172.21.15.45:0/1742073556 conn(0x7ffa781021f0 msgr2=0x7ffa78104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.641+0000 7ffa7e25c700 1 -- 172.21.15.45:0/1742073556 shutdown_connections 2024-06-06T04:20:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.641+0000 7ffa7e25c700 1 -- 172.21.15.45:0/1742073556 wait complete. 2024-06-06T04:20:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.642+0000 7ffa7e25c700 1 Processor -- start 2024-06-06T04:20:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.642+0000 7ffa7e25c700 1 -- start start 2024-06-06T04:20:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.642+0000 7ffa7e25c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa78106610 0x7ffa7819de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.642+0000 7ffa7e25c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78107fe0 con 0x7ffa78106610 2024-06-06T04:20:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.642+0000 7ffa777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa78106610 0x7ffa7819de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.643+0000 7ffa777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa78106610 0x7ffa7819de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34482/0 (socket says 172.21.15.45:34482) 2024-06-06T04:20:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.643+0000 7ffa777fe700 1 -- 172.21.15.45:0/27432038 learned_addr learned my addr 172.21.15.45:0/27432038 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.643+0000 7ffa777fe700 1 -- 172.21.15.45:0/27432038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7819e370 con 0x7ffa78106610 2024-06-06T04:20:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.643+0000 7ffa777fe700 1 --2- 172.21.15.45:0/27432038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa78106610 0x7ffa7819de30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa68011a20 tx=0x7ffa68011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.643+0000 7ffa74ff9700 1 -- 172.21.15.45:0/27432038 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa68003b70 con 0x7ffa78106610 2024-06-06T04:20:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.643+0000 7ffa74ff9700 1 -- 172.21.15.45:0/27432038 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa6801a8e0 con 0x7ffa78106610 2024-06-06T04:20:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.643+0000 7ffa7e25c700 1 -- 172.21.15.45:0/27432038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7819e600 con 0x7ffa78106610 2024-06-06T04:20:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.643+0000 7ffa7e25c700 1 -- 172.21.15.45:0/27432038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa781a12f0 con 0x7ffa78106610 2024-06-06T04:20:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.643+0000 7ffa74ff9700 1 -- 172.21.15.45:0/27432038 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa68024990 con 0x7ffa78106610 2024-06-06T04:20:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.644+0000 7ffa74ff9700 1 -- 172.21.15.45:0/27432038 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7ffa68022020 con 0x7ffa78106610 2024-06-06T04:20:07.523 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.644+0000 7ffa7e25c700 1 -- 172.21.15.45:0/27432038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa78063af0 con 0x7ffa78106610 2024-06-06T04:20:07.523 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.644+0000 7ffa74ff9700 1 --2- 172.21.15.45:0/27432038 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ffa64038f90 0x7ffa6403b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:07.523 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.645+0000 7ffa74ff9700 1 -- 172.21.15.45:0/27432038 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffa6801a430 con 0x7ffa78106610 2024-06-06T04:20:07.523 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.645+0000 7ffa76ffd700 1 --2- 172.21.15.45:0/27432038 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ffa64038f90 0x7ffa6403b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:07.523 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.645+0000 7ffa76ffd700 1 --2- 172.21.15.45:0/27432038 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ffa64038f90 0x7ffa6403b450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffa60006fd0 tx=0x7ffa60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:07.523 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.650+0000 7ffa74ff9700 1 -- 172.21.15.45:0/27432038 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffa68020c10 con 0x7ffa78106610 2024-06-06T04:20:07.523 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.676+0000 7ffa74ff9700 1 -- 172.21.15.45:0/27432038 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7ffa6802dc90 con 0x7ffa78106610 2024-06-06T04:20:07.523 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.873+0000 7ffa7e25c700 1 -- 172.21.15.45:0/27432038 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7ffa7802e050 con 0x7ffa64038f90 2024-06-06T04:20:07.523 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.945+0000 7ffa74ff9700 1 -- 172.21.15.45:0/27432038 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7ffa7802e050 con 0x7ffa64038f90 2024-06-06T04:20:07.523 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.947+0000 7ffa7e25c700 1 -- 172.21.15.45:0/27432038 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ffa64038f90 msgr2=0x7ffa6403b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:07.523 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.947+0000 7ffa7e25c700 1 --2- 172.21.15.45:0/27432038 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ffa64038f90 0x7ffa6403b450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffa60006fd0 tx=0x7ffa60006e40 comp rx=0 tx=0).stop 2024-06-06T04:20:07.523 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.947+0000 7ffa7e25c700 1 -- 172.21.15.45:0/27432038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa78106610 msgr2=0x7ffa7819de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:07.523 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.947+0000 7ffa7e25c700 1 --2- 172.21.15.45:0/27432038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa78106610 0x7ffa7819de30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa68011a20 tx=0x7ffa68011b00 comp rx=0 tx=0).stop 2024-06-06T04:20:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.947+0000 7ffa7e25c700 1 -- 172.21.15.45:0/27432038 shutdown_connections 2024-06-06T04:20:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.947+0000 7ffa7e25c700 1 --2- 172.21.15.45:0/27432038 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ffa64038f90 0x7ffa6403b450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.947+0000 7ffa7e25c700 1 --2- 172.21.15.45:0/27432038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa78106610 0x7ffa7819de30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.947+0000 7ffa7e25c700 1 -- 172.21.15.45:0/27432038 >> 172.21.15.45:0/27432038 conn(0x7ffa781021f0 msgr2=0x7ffa78194ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.948+0000 7ffa7e25c700 1 -- 172.21.15.45:0/27432038 shutdown_connections 2024-06-06T04:20:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:06.948+0000 7ffa7e25c700 1 -- 172.21.15.45:0/27432038 wait complete. 2024-06-06T04:20:07.524 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial admin user... 2024-06-06T04:20:07.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:07 smithi045 ceph-mon[32299]: mgrmap e12: x(active, since 2s) 2024-06-06T04:20:07.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:07 smithi045 ceph-mon[32299]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:20:07.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:07.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:07.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:09.056 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$o4Hu/X71vdUQ0ebtum417O6tUKaEK0RY3euOPpK.LZM.B9A5Ja4Ay", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1717647608, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-06T04:20:09.056 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.941+0000 7f780f834700 1 Processor -- start 2024-06-06T04:20:09.056 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.942+0000 7f780f834700 1 -- start start 2024-06-06T04:20:09.056 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.942+0000 7f780f834700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7808108880 0x7f780810ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:09.056 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.942+0000 7f780f834700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f780807be40 con 0x7f7808108880 2024-06-06T04:20:09.056 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.942+0000 7f780d5d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7808108880 0x7f780810ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:09.056 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.942+0000 7f780d5d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7808108880 0x7f780810ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54332/0 (socket says 172.21.15.45:54332) 2024-06-06T04:20:09.056 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.942+0000 7f780d5d0700 1 -- 172.21.15.45:0/1077252300 learned_addr learned my addr 172.21.15.45:0/1077252300 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:09.056 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.942+0000 7f780d5d0700 1 -- 172.21.15.45:0/1077252300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780810b1b0 con 0x7f7808108880 2024-06-06T04:20:09.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.942+0000 7f780d5d0700 1 --2- 172.21.15.45:0/1077252300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7808108880 0x7f780810ac70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f78040098a0 tx=0x7f7804009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a58f50503519cb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:09.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.943+0000 7f77fffff700 1 -- 172.21.15.45:0/1077252300 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7804004160 con 0x7f7808108880 2024-06-06T04:20:09.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.943+0000 7f77fffff700 1 -- 172.21.15.45:0/1077252300 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7804004300 con 0x7f7808108880 2024-06-06T04:20:09.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.943+0000 7f77fffff700 1 -- 172.21.15.45:0/1077252300 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7804011a50 con 0x7f7808108880 2024-06-06T04:20:09.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.943+0000 7f780f834700 1 -- 172.21.15.45:0/1077252300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7808108880 msgr2=0x7f780810ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:09.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.943+0000 7f780f834700 1 --2- 172.21.15.45:0/1077252300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7808108880 0x7f780810ac70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f78040098a0 tx=0x7f7804009bb0 comp rx=0 tx=0).stop 2024-06-06T04:20:09.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.943+0000 7f780f834700 1 -- 172.21.15.45:0/1077252300 shutdown_connections 2024-06-06T04:20:09.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.943+0000 7f780f834700 1 --2- 172.21.15.45:0/1077252300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7808108880 0x7f780810ac70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:09.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.943+0000 7f780f834700 1 -- 172.21.15.45:0/1077252300 >> 172.21.15.45:0/1077252300 conn(0x7f7808102270 msgr2=0x7f7808104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:09.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.944+0000 7f780f834700 1 -- 172.21.15.45:0/1077252300 shutdown_connections 2024-06-06T04:20:09.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.944+0000 7f780f834700 1 -- 172.21.15.45:0/1077252300 wait complete. 2024-06-06T04:20:09.057 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.944+0000 7f780f834700 1 Processor -- start 2024-06-06T04:20:09.058 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.944+0000 7f780f834700 1 -- start start 2024-06-06T04:20:09.058 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.944+0000 7f780f834700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7808108880 0x7f780819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:09.058 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.944+0000 7f780f834700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f780810c160 con 0x7f7808108880 2024-06-06T04:20:09.058 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.944+0000 7f780d5d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7808108880 0x7f780819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:09.058 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.944+0000 7f780d5d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7808108880 0x7f780819bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54346/0 (socket says 172.21.15.45:54346) 2024-06-06T04:20:09.058 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.945+0000 7f780d5d0700 1 -- 172.21.15.45:0/3111304343 learned_addr learned my addr 172.21.15.45:0/3111304343 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:09.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.945+0000 7f780d5d0700 1 -- 172.21.15.45:0/3111304343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780819c150 con 0x7f7808108880 2024-06-06T04:20:09.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.945+0000 7f780d5d0700 1 --2- 172.21.15.45:0/3111304343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7808108880 0x7f780819bc10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7804011a20 tx=0x7f7804011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:09.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.945+0000 7f77fe7fc700 1 -- 172.21.15.45:0/3111304343 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7804003b70 con 0x7f7808108880 2024-06-06T04:20:09.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.945+0000 7f77fe7fc700 1 -- 172.21.15.45:0/3111304343 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f780401a8e0 con 0x7f7808108880 2024-06-06T04:20:09.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.945+0000 7f780f834700 1 -- 172.21.15.45:0/3111304343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780819c3e0 con 0x7f7808108880 2024-06-06T04:20:09.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.945+0000 7f77fe7fc700 1 -- 172.21.15.45:0/3111304343 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7804024990 con 0x7f7808108880 2024-06-06T04:20:09.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.945+0000 7f780f834700 1 -- 172.21.15.45:0/3111304343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780819f0d0 con 0x7f7808108880 2024-06-06T04:20:09.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.946+0000 7f780f834700 1 -- 172.21.15.45:0/3111304343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7808063af0 con 0x7f7808108880 2024-06-06T04:20:09.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.947+0000 7f77fe7fc700 1 -- 172.21.15.45:0/3111304343 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f7804022020 con 0x7f7808108880 2024-06-06T04:20:09.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.947+0000 7f77fe7fc700 1 --2- 172.21.15.45:0/3111304343 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f77f40390b0 0x7f77f403b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:09.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.947+0000 7f77fe7fc700 1 -- 172.21.15.45:0/3111304343 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f780404d900 con 0x7f7808108880 2024-06-06T04:20:09.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.947+0000 7f780cdcf700 1 --2- 172.21.15.45:0/3111304343 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f77f40390b0 0x7f77f403b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:09.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.948+0000 7f780cdcf700 1 --2- 172.21.15.45:0/3111304343 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f77f40390b0 0x7f77f403b570 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77f8006fd0 tx=0x7f77f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:09.059 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:07.955+0000 7f77fe7fc700 1 -- 172.21.15.45:0/3111304343 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f780402d3f0 con 0x7f7808108880 2024-06-06T04:20:09.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:08.181+0000 7f780f834700 1 -- 172.21.15.45:0/3111304343 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f780819f360 con 0x7f77f40390b0 2024-06-06T04:20:09.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:08.428+0000 7f77fe7fc700 1 -- 172.21.15.45:0/3111304343 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f780819f360 con 0x7f77f40390b0 2024-06-06T04:20:09.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:08.430+0000 7f780f834700 1 -- 172.21.15.45:0/3111304343 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f77f40390b0 msgr2=0x7f77f403b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:09.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:08.430+0000 7f780f834700 1 --2- 172.21.15.45:0/3111304343 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f77f40390b0 0x7f77f403b570 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77f8006fd0 tx=0x7f77f8006e40 comp rx=0 tx=0).stop 2024-06-06T04:20:09.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:08.430+0000 7f780f834700 1 -- 172.21.15.45:0/3111304343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7808108880 msgr2=0x7f780819bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:09.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:08.430+0000 7f780f834700 1 --2- 172.21.15.45:0/3111304343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7808108880 0x7f780819bc10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7804011a20 tx=0x7f7804011b00 comp rx=0 tx=0).stop 2024-06-06T04:20:09.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:08.430+0000 7f780f834700 1 -- 172.21.15.45:0/3111304343 shutdown_connections 2024-06-06T04:20:09.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:08.430+0000 7f780f834700 1 --2- 172.21.15.45:0/3111304343 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f77f40390b0 0x7f77f403b570 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:09.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:08.430+0000 7f780f834700 1 --2- 172.21.15.45:0/3111304343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7808108880 0x7f780819bc10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:09.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:08.430+0000 7f780f834700 1 -- 172.21.15.45:0/3111304343 >> 172.21.15.45:0/3111304343 conn(0x7f7808102270 msgr2=0x7f7808102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:09.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:08.430+0000 7f780f834700 1 -- 172.21.15.45:0/3111304343 shutdown_connections 2024-06-06T04:20:09.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:08.430+0000 7f780f834700 1 -- 172.21.15.45:0/3111304343 wait complete. 2024-06-06T04:20:09.060 INFO:teuthology.orchestra.run.smithi045.stdout:Fetching dashboard port number... 2024-06-06T04:20:09.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:09 smithi045 ceph-mon[32299]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:20:09.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:09 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:10.473 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 8443 2024-06-06T04:20:10.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.508+0000 7f3f500b7700 1 Processor -- start 2024-06-06T04:20:10.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.508+0000 7f3f500b7700 1 -- start start 2024-06-06T04:20:10.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.509+0000 7f3f500b7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48108810 0x7f3f4810ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:10.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.509+0000 7f3f500b7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4807be60 con 0x7f3f48108810 2024-06-06T04:20:10.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.509+0000 7f3f4de53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48108810 0x7f3f4810ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:10.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.509+0000 7f3f4de53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48108810 0x7f3f4810ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54354/0 (socket says 172.21.15.45:54354) 2024-06-06T04:20:10.474 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.509+0000 7f3f4de53700 1 -- 172.21.15.45:0/2020523114 learned_addr learned my addr 172.21.15.45:0/2020523114 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:10.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.509+0000 7f3f4de53700 1 -- 172.21.15.45:0/2020523114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4810b140 con 0x7f3f48108810 2024-06-06T04:20:10.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.509+0000 7f3f4de53700 1 --2- 172.21.15.45:0/2020523114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48108810 0x7f3f4810ac00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f440098a0 tx=0x7f3f44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59eea1ee9bd97eef server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:10.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.509+0000 7f3f4ce51700 1 -- 172.21.15.45:0/2020523114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f44004160 con 0x7f3f48108810 2024-06-06T04:20:10.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.509+0000 7f3f4ce51700 1 -- 172.21.15.45:0/2020523114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f44004300 con 0x7f3f48108810 2024-06-06T04:20:10.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.509+0000 7f3f4ce51700 1 -- 172.21.15.45:0/2020523114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f44011a50 con 0x7f3f48108810 2024-06-06T04:20:10.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.510+0000 7f3f500b7700 1 -- 172.21.15.45:0/2020523114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48108810 msgr2=0x7f3f4810ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:10.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.510+0000 7f3f500b7700 1 --2- 172.21.15.45:0/2020523114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48108810 0x7f3f4810ac00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f440098a0 tx=0x7f3f44009bb0 comp rx=0 tx=0).stop 2024-06-06T04:20:10.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.510+0000 7f3f500b7700 1 -- 172.21.15.45:0/2020523114 shutdown_connections 2024-06-06T04:20:10.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.510+0000 7f3f500b7700 1 --2- 172.21.15.45:0/2020523114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48108810 0x7f3f4810ac00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:10.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.510+0000 7f3f500b7700 1 -- 172.21.15.45:0/2020523114 >> 172.21.15.45:0/2020523114 conn(0x7f3f481021e0 msgr2=0x7f3f48104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:10.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.510+0000 7f3f500b7700 1 -- 172.21.15.45:0/2020523114 shutdown_connections 2024-06-06T04:20:10.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.510+0000 7f3f500b7700 1 -- 172.21.15.45:0/2020523114 wait complete. 2024-06-06T04:20:10.475 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.511+0000 7f3f500b7700 1 Processor -- start 2024-06-06T04:20:10.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.511+0000 7f3f500b7700 1 -- start start 2024-06-06T04:20:10.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.511+0000 7f3f500b7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48108810 0x7f3f48108090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:10.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.511+0000 7f3f500b7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4810c140 con 0x7f3f48108810 2024-06-06T04:20:10.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.511+0000 7f3f4de53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48108810 0x7f3f48108090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:10.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.511+0000 7f3f4de53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48108810 0x7f3f48108090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54368/0 (socket says 172.21.15.45:54368) 2024-06-06T04:20:10.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.511+0000 7f3f4de53700 1 -- 172.21.15.45:0/920488 learned_addr learned my addr 172.21.15.45:0/920488 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:10.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.511+0000 7f3f4de53700 1 -- 172.21.15.45:0/920488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f481085d0 con 0x7f3f48108810 2024-06-06T04:20:10.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.512+0000 7f3f4de53700 1 --2- 172.21.15.45:0/920488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48108810 0x7f3f48108090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3f44011a20 tx=0x7f3f44011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:10.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.512+0000 7f3f3effd700 1 -- 172.21.15.45:0/920488 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f44003b70 con 0x7f3f48108810 2024-06-06T04:20:10.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.512+0000 7f3f3effd700 1 -- 172.21.15.45:0/920488 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f4401a8e0 con 0x7f3f48108810 2024-06-06T04:20:10.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.512+0000 7f3f3effd700 1 -- 172.21.15.45:0/920488 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f44024990 con 0x7f3f48108810 2024-06-06T04:20:10.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.512+0000 7f3f500b7700 1 -- 172.21.15.45:0/920488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f481067c0 con 0x7f3f48108810 2024-06-06T04:20:10.476 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.512+0000 7f3f500b7700 1 -- 172.21.15.45:0/920488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f48106d00 con 0x7f3f48108810 2024-06-06T04:20:10.477 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.513+0000 7f3f3effd700 1 -- 172.21.15.45:0/920488 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f3f44022020 con 0x7f3f48108810 2024-06-06T04:20:10.477 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.513+0000 7f3f3effd700 1 --2- 172.21.15.45:0/920488 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3f34039050 0x7f3f3403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:10.477 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.513+0000 7f3f3effd700 1 -- 172.21.15.45:0/920488 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3f4404d570 con 0x7f3f48108810 2024-06-06T04:20:10.477 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.513+0000 7f3f500b7700 1 -- 172.21.15.45:0/920488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f48063af0 con 0x7f3f48108810 2024-06-06T04:20:10.477 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.513+0000 7f3f4d652700 1 --2- 172.21.15.45:0/920488 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3f34039050 0x7f3f3403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:10.477 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.514+0000 7f3f4d652700 1 --2- 172.21.15.45:0/920488 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3f34039050 0x7f3f3403b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3f38006fd0 tx=0x7f3f38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:10.477 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.518+0000 7f3f3effd700 1 -- 172.21.15.45:0/920488 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3f44021d80 con 0x7f3f48108810 2024-06-06T04:20:10.477 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.730+0000 7f3f500b7700 1 -- 172.21.15.45:0/920488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f3f48107380 con 0x7f3f48108810 2024-06-06T04:20:10.478 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.731+0000 7f3f3effd700 1 -- 172.21.15.45:0/920488 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f3f44032020 con 0x7f3f48108810 2024-06-06T04:20:10.478 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.733+0000 7f3f500b7700 1 -- 172.21.15.45:0/920488 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3f34039050 msgr2=0x7f3f3403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:10.478 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.733+0000 7f3f500b7700 1 --2- 172.21.15.45:0/920488 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3f34039050 0x7f3f3403b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3f38006fd0 tx=0x7f3f38006e40 comp rx=0 tx=0).stop 2024-06-06T04:20:10.478 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.733+0000 7f3f500b7700 1 -- 172.21.15.45:0/920488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48108810 msgr2=0x7f3f48108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:10.478 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.733+0000 7f3f500b7700 1 --2- 172.21.15.45:0/920488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48108810 0x7f3f48108090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3f44011a20 tx=0x7f3f44011b00 comp rx=0 tx=0).stop 2024-06-06T04:20:10.478 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.733+0000 7f3f500b7700 1 -- 172.21.15.45:0/920488 shutdown_connections 2024-06-06T04:20:10.478 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.733+0000 7f3f500b7700 1 --2- 172.21.15.45:0/920488 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3f34039050 0x7f3f3403b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:10.478 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.733+0000 7f3f500b7700 1 --2- 172.21.15.45:0/920488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48108810 0x7f3f48108090 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:10.478 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.733+0000 7f3f500b7700 1 -- 172.21.15.45:0/920488 >> 172.21.15.45:0/920488 conn(0x7f3f481021e0 msgr2=0x7f3f48102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:10.478 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.733+0000 7f3f500b7700 1 -- 172.21.15.45:0/920488 shutdown_connections 2024-06-06T04:20:10.478 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:09.733+0000 7f3f500b7700 1 -- 172.21.15.45:0/920488 wait complete. 2024-06-06T04:20:10.482 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T04:20:10.482 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-06-06T04:20:10.490 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T04:20:10.491 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-06-06T04:20:10.491 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-06-06T04:20:10.491 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-06T04:20:10.492 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph Dashboard is now available at: 2024-06-06T04:20:10.493 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:20:10.493 INFO:teuthology.orchestra.run.smithi045.stdout: URL: https://smithi045.front.sepia.ceph.com:8443/ 2024-06-06T04:20:10.493 INFO:teuthology.orchestra.run.smithi045.stdout: User: admin 2024-06-06T04:20:10.493 INFO:teuthology.orchestra.run.smithi045.stdout: Password: 15kq2x8zrz 2024-06-06T04:20:10.493 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:20:10.493 INFO:teuthology.orchestra.run.smithi045.stdout:Saving cluster configuration to /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config directory 2024-06-06T04:20:10.493 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling autotune for osd_memory_target 2024-06-06T04:20:10.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:10 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/920488' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-06T04:20:11.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.886+0000 7f98588a9700 1 Processor -- start 2024-06-06T04:20:11.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.887+0000 7f98588a9700 1 -- start start 2024-06-06T04:20:11.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.887+0000 7f98588a9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854108220 0x7f9854108620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:11.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.887+0000 7f98588a9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9854108bf0 con 0x7f9854108220 2024-06-06T04:20:11.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.887+0000 7f985259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854108220 0x7f9854108620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:11.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.887+0000 7f985259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854108220 0x7f9854108620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54374/0 (socket says 172.21.15.45:54374) 2024-06-06T04:20:11.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.887+0000 7f985259c700 1 -- 172.21.15.45:0/1915546809 learned_addr learned my addr 172.21.15.45:0/1915546809 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:11.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.887+0000 7f985259c700 1 -- 172.21.15.45:0/1915546809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9854108d70 con 0x7f9854108220 2024-06-06T04:20:11.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.887+0000 7f985259c700 1 --2- 172.21.15.45:0/1915546809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854108220 0x7f9854108620 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f98440098a0 tx=0x7f9844009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd88035c0312143 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:11.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.888+0000 7f9851d9b700 1 -- 172.21.15.45:0/1915546809 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9844004160 con 0x7f9854108220 2024-06-06T04:20:11.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.888+0000 7f9851d9b700 1 -- 172.21.15.45:0/1915546809 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9844004300 con 0x7f9854108220 2024-06-06T04:20:11.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.888+0000 7f9851d9b700 1 -- 172.21.15.45:0/1915546809 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9844011a50 con 0x7f9854108220 2024-06-06T04:20:11.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.888+0000 7f98588a9700 1 -- 172.21.15.45:0/1915546809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854108220 msgr2=0x7f9854108620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:11.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.888+0000 7f98588a9700 1 --2- 172.21.15.45:0/1915546809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854108220 0x7f9854108620 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f98440098a0 tx=0x7f9844009bb0 comp rx=0 tx=0).stop 2024-06-06T04:20:11.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.888+0000 7f98588a9700 1 -- 172.21.15.45:0/1915546809 shutdown_connections 2024-06-06T04:20:11.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.888+0000 7f98588a9700 1 --2- 172.21.15.45:0/1915546809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854108220 0x7f9854108620 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:11.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.888+0000 7f98588a9700 1 -- 172.21.15.45:0/1915546809 >> 172.21.15.45:0/1915546809 conn(0x7f98541019b0 msgr2=0x7f9854103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:11.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.888+0000 7f98588a9700 1 -- 172.21.15.45:0/1915546809 shutdown_connections 2024-06-06T04:20:11.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.888+0000 7f98588a9700 1 -- 172.21.15.45:0/1915546809 wait complete. 2024-06-06T04:20:11.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.889+0000 7f98588a9700 1 Processor -- start 2024-06-06T04:20:11.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.889+0000 7f98588a9700 1 -- start start 2024-06-06T04:20:11.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.889+0000 7f98588a9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854108220 0x7f985419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:11.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.889+0000 7f98588a9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9854106c30 con 0x7f9854108220 2024-06-06T04:20:11.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.889+0000 7f985259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854108220 0x7f985419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:11.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.889+0000 7f985259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854108220 0x7f985419de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54382/0 (socket says 172.21.15.45:54382) 2024-06-06T04:20:11.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.889+0000 7f985259c700 1 -- 172.21.15.45:0/511229008 learned_addr learned my addr 172.21.15.45:0/511229008 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:11.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.890+0000 7f985259c700 1 -- 172.21.15.45:0/511229008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985419e340 con 0x7f9854108220 2024-06-06T04:20:11.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.890+0000 7f985259c700 1 --2- 172.21.15.45:0/511229008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854108220 0x7f985419de00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9844006b20 tx=0x7f9844011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:11.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.890+0000 7f984b7fe700 1 -- 172.21.15.45:0/511229008 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98440039e0 con 0x7f9854108220 2024-06-06T04:20:11.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.890+0000 7f98588a9700 1 -- 172.21.15.45:0/511229008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f985419e5d0 con 0x7f9854108220 2024-06-06T04:20:11.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.890+0000 7f98588a9700 1 -- 172.21.15.45:0/511229008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98541a12c0 con 0x7f9854108220 2024-06-06T04:20:11.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.890+0000 7f984b7fe700 1 -- 172.21.15.45:0/511229008 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f984401a8e0 con 0x7f9854108220 2024-06-06T04:20:11.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.890+0000 7f984b7fe700 1 -- 172.21.15.45:0/511229008 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98440247e0 con 0x7f9854108220 2024-06-06T04:20:11.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.891+0000 7f984b7fe700 1 -- 172.21.15.45:0/511229008 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9844022020 con 0x7f9854108220 2024-06-06T04:20:11.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.891+0000 7f98588a9700 1 -- 172.21.15.45:0/511229008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9834005360 con 0x7f9854108220 2024-06-06T04:20:11.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.891+0000 7f984b7fe700 1 --2- 172.21.15.45:0/511229008 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9840039190 0x7f984003b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:11.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.891+0000 7f984b7fe700 1 -- 172.21.15.45:0/511229008 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f984404d6d0 con 0x7f9854108220 2024-06-06T04:20:11.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.891+0000 7f984bfff700 1 --2- 172.21.15.45:0/511229008 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9840039190 0x7f984003b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:11.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.892+0000 7f984bfff700 1 --2- 172.21.15.45:0/511229008 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9840039190 0x7f984003b650 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f983c006fd0 tx=0x7f983c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:11.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:10.896+0000 7f984b7fe700 1 -- 172.21.15.45:0/511229008 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f98440172b0 con 0x7f9854108220 2024-06-06T04:20:11.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:11.101+0000 7f98588a9700 1 -- 172.21.15.45:0/511229008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f9834005650 con 0x7f9854108220 2024-06-06T04:20:11.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:11.101+0000 7f984b7fe700 1 -- 172.21.15.45:0/511229008 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f984400fdd0 con 0x7f9854108220 2024-06-06T04:20:11.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:11.104+0000 7f98588a9700 1 -- 172.21.15.45:0/511229008 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9840039190 msgr2=0x7f984003b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:11.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:11.104+0000 7f98588a9700 1 --2- 172.21.15.45:0/511229008 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9840039190 0x7f984003b650 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f983c006fd0 tx=0x7f983c006e40 comp rx=0 tx=0).stop 2024-06-06T04:20:11.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:11.104+0000 7f98588a9700 1 -- 172.21.15.45:0/511229008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854108220 msgr2=0x7f985419de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:11.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:11.104+0000 7f98588a9700 1 --2- 172.21.15.45:0/511229008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854108220 0x7f985419de00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9844006b20 tx=0x7f9844011ad0 comp rx=0 tx=0).stop 2024-06-06T04:20:11.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:11.104+0000 7f98588a9700 1 -- 172.21.15.45:0/511229008 shutdown_connections 2024-06-06T04:20:11.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:11.104+0000 7f98588a9700 1 --2- 172.21.15.45:0/511229008 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9840039190 0x7f984003b650 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:11.625 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:11.104+0000 7f98588a9700 1 --2- 172.21.15.45:0/511229008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854108220 0x7f985419de00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:11.625 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:11.104+0000 7f98588a9700 1 -- 172.21.15.45:0/511229008 >> 172.21.15.45:0/511229008 conn(0x7f98541019b0 msgr2=0x7f9854194440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:11.625 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:11.104+0000 7f98588a9700 1 -- 172.21.15.45:0/511229008 shutdown_connections 2024-06-06T04:20:11.625 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:11.104+0000 7f98588a9700 1 -- 172.21.15.45:0/511229008 wait complete. 2024-06-06T04:20:11.722 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:11 smithi045 ceph-mon[32299]: mgrmap e13: x(active, since 6s) 2024-06-06T04:20:12.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.004+0000 7f04dfa75700 1 Processor -- start 2024-06-06T04:20:12.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.004+0000 7f04dfa75700 1 -- start start 2024-06-06T04:20:12.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.005+0000 7f04dfa75700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d8106c50 0x7f04d8107070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:12.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.005+0000 7f04dfa75700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8107640 con 0x7f04d8106c50 2024-06-06T04:20:12.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.005+0000 7f04dd811700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d8106c50 0x7f04d8107070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:12.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.005+0000 7f04dd811700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d8106c50 0x7f04d8107070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54392/0 (socket says 172.21.15.45:54392) 2024-06-06T04:20:12.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.005+0000 7f04dd811700 1 -- 172.21.15.45:0/4198838453 learned_addr learned my addr 172.21.15.45:0/4198838453 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:12.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.005+0000 7f04dd811700 1 -- 172.21.15.45:0/4198838453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d8107e50 con 0x7f04d8106c50 2024-06-06T04:20:12.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.005+0000 7f04dd811700 1 --2- 172.21.15.45:0/4198838453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d8106c50 0x7f04d8107070 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f04d400d180 tx=0x7f04d400d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dacae8131387f17 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:12.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.005+0000 7f04dc80f700 1 -- 172.21.15.45:0/4198838453 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d4019070 con 0x7f04d8106c50 2024-06-06T04:20:12.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.006+0000 7f04dc80f700 1 -- 172.21.15.45:0/4198838453 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04d400de20 con 0x7f04d8106c50 2024-06-06T04:20:12.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.006+0000 7f04dc80f700 1 -- 172.21.15.45:0/4198838453 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d4009a80 con 0x7f04d8106c50 2024-06-06T04:20:12.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.006+0000 7f04dfa75700 1 -- 172.21.15.45:0/4198838453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d8106c50 msgr2=0x7f04d8107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:12.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.006+0000 7f04dfa75700 1 --2- 172.21.15.45:0/4198838453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d8106c50 0x7f04d8107070 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f04d400d180 tx=0x7f04d400d490 comp rx=0 tx=0).stop 2024-06-06T04:20:12.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.006+0000 7f04dfa75700 1 -- 172.21.15.45:0/4198838453 shutdown_connections 2024-06-06T04:20:12.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.006+0000 7f04dfa75700 1 --2- 172.21.15.45:0/4198838453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d8106c50 0x7f04d8107070 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:12.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.006+0000 7f04dfa75700 1 -- 172.21.15.45:0/4198838453 >> 172.21.15.45:0/4198838453 conn(0x7f04d81021f0 msgr2=0x7f04d8104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:12.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.006+0000 7f04dfa75700 1 -- 172.21.15.45:0/4198838453 shutdown_connections 2024-06-06T04:20:12.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.006+0000 7f04dfa75700 1 -- 172.21.15.45:0/4198838453 wait complete. 2024-06-06T04:20:12.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.007+0000 7f04dfa75700 1 Processor -- start 2024-06-06T04:20:12.872 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.007+0000 7f04dfa75700 1 -- start start 2024-06-06T04:20:12.872 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.007+0000 7f04dfa75700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d8106c50 0x7f04d819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:12.872 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.007+0000 7f04dfa75700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f04d807afc0 con 0x7f04d8106c50 2024-06-06T04:20:12.872 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.007+0000 7f04dd811700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d8106c50 0x7f04d819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:12.872 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.007+0000 7f04dd811700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d8106c50 0x7f04d819de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54400/0 (socket says 172.21.15.45:54400) 2024-06-06T04:20:12.872 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.007+0000 7f04dd811700 1 -- 172.21.15.45:0/1840702552 learned_addr learned my addr 172.21.15.45:0/1840702552 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:12.872 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.007+0000 7f04dd811700 1 -- 172.21.15.45:0/1840702552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d819e3a0 con 0x7f04d8106c50 2024-06-06T04:20:12.872 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.007+0000 7f04dd811700 1 --2- 172.21.15.45:0/1840702552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d8106c50 0x7f04d819de60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f04d400f010 tx=0x7f04d4003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:12.872 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.008+0000 7f04ceffd700 1 -- 172.21.15.45:0/1840702552 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d4019040 con 0x7f04d8106c50 2024-06-06T04:20:12.872 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.008+0000 7f04ceffd700 1 -- 172.21.15.45:0/1840702552 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04d400b440 con 0x7f04d8106c50 2024-06-06T04:20:12.872 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.008+0000 7f04dfa75700 1 -- 172.21.15.45:0/1840702552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d819e630 con 0x7f04d8106c50 2024-06-06T04:20:12.873 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.008+0000 7f04ceffd700 1 -- 172.21.15.45:0/1840702552 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d401f7a0 con 0x7f04d8106c50 2024-06-06T04:20:12.873 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.008+0000 7f04dfa75700 1 -- 172.21.15.45:0/1840702552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d81a12d0 con 0x7f04d8106c50 2024-06-06T04:20:12.873 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.009+0000 7f04ceffd700 1 -- 172.21.15.45:0/1840702552 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f04d401fa20 con 0x7f04d8106c50 2024-06-06T04:20:12.873 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.009+0000 7f04dfa75700 1 -- 172.21.15.45:0/1840702552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d8063af0 con 0x7f04d8106c50 2024-06-06T04:20:12.873 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.009+0000 7f04ceffd700 1 --2- 172.21.15.45:0/1840702552 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f04c40390f0 0x7f04c403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:12.873 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.009+0000 7f04ceffd700 1 -- 172.21.15.45:0/1840702552 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f04d4011050 con 0x7f04d8106c50 2024-06-06T04:20:12.873 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.009+0000 7f04dd010700 1 --2- 172.21.15.45:0/1840702552 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f04c40390f0 0x7f04c403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:12.874 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.009+0000 7f04dd010700 1 --2- 172.21.15.45:0/1840702552 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f04c40390f0 0x7f04c403b5b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f04c8006fd0 tx=0x7f04c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:12.874 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.014+0000 7f04ceffd700 1 -- 172.21.15.45:0/1840702552 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f04d4014950 con 0x7f04d8106c50 2024-06-06T04:20:12.874 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.333+0000 7f04dfa75700 1 -- 172.21.15.45:0/1840702552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f04d81a14a0 con 0x7f04d8106c50 2024-06-06T04:20:12.874 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.336+0000 7f04ceffd700 1 -- 172.21.15.45:0/1840702552 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f04d4014430 con 0x7f04d8106c50 2024-06-06T04:20:12.874 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.338+0000 7f04dfa75700 1 -- 172.21.15.45:0/1840702552 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f04c40390f0 msgr2=0x7f04c403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:12.874 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.338+0000 7f04dfa75700 1 --2- 172.21.15.45:0/1840702552 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f04c40390f0 0x7f04c403b5b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f04c8006fd0 tx=0x7f04c8006e40 comp rx=0 tx=0).stop 2024-06-06T04:20:12.874 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.338+0000 7f04dfa75700 1 -- 172.21.15.45:0/1840702552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d8106c50 msgr2=0x7f04d819de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:12.874 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.338+0000 7f04dfa75700 1 --2- 172.21.15.45:0/1840702552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d8106c50 0x7f04d819de60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f04d400f010 tx=0x7f04d4003d00 comp rx=0 tx=0).stop 2024-06-06T04:20:12.874 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.338+0000 7f04dfa75700 1 -- 172.21.15.45:0/1840702552 shutdown_connections 2024-06-06T04:20:12.874 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.338+0000 7f04dfa75700 1 --2- 172.21.15.45:0/1840702552 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f04c40390f0 0x7f04c403b5b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:12.874 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.338+0000 7f04dfa75700 1 --2- 172.21.15.45:0/1840702552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d8106c50 0x7f04d819de60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:12.874 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.338+0000 7f04dfa75700 1 -- 172.21.15.45:0/1840702552 >> 172.21.15.45:0/1840702552 conn(0x7f04d81021f0 msgr2=0x7f04d8190070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:12.875 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.339+0000 7f04dfa75700 1 -- 172.21.15.45:0/1840702552 shutdown_connections 2024-06-06T04:20:12.875 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-06-06T04:20:12.339+0000 7f04dfa75700 1 -- 172.21.15.45:0/1840702552 wait complete. 2024-06-06T04:20:12.875 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-06T04:20:12.875 INFO:teuthology.orchestra.run.smithi045.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-06T04:20:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:20:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-06T04:20:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:20:12.875 INFO:teuthology.orchestra.run.smithi045.stdout:Or, if you are only running a single cluster on this host: 2024-06-06T04:20:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:20:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-06T04:20:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:20:12.875 INFO:teuthology.orchestra.run.smithi045.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-06T04:20:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:20:12.875 INFO:teuthology.orchestra.run.smithi045.stdout: ceph telemetry on 2024-06-06T04:20:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:20:12.876 INFO:teuthology.orchestra.run.smithi045.stdout:For more information see: 2024-06-06T04:20:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:20:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-06T04:20:12.876 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:20:12.876 INFO:teuthology.orchestra.run.smithi045.stdout:Bootstrap complete. 2024-06-06T04:20:12.911 INFO:tasks.cephadm:Fetching config... 2024-06-06T04:20:12.912 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:20:12.912 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-06T04:20:12.930 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-06T04:20:12.930 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:20:12.930 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-06T04:20:12.986 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-06T04:20:12.986 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:20:12.986 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-06T04:20:13.052 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-06T04:20:13.052 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:20:13.052 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-06T04:20:13.108 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-06T04:20:13.108 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaQ80a1msjimWFH42LBkkRUHyrBG+CxhI6lol3TFACE1Nzq2HjAR5zPbKduXNJkpsaAxWkQeZrTuIDzp2q0xY7yCVt5NBvt3KyjMKdaxZ38WMdu1TUYtkPBf+7RwZ/xY5uFsNlY15sglJoV2OfKAqp8Gra1kzPny1SuFFAlcmCbbBf079j7hYTRW5nO2hIdM4Ar9Tjrzd52WnniuLeX37zAlr1s35VkdznYI81ZZqNJkk01NhgICJnLl2P2Hn/ysoEbnQ2UwXjq4BlCjebWRcXHVTmnQ34t0L3nBJqIVhfidqJ5IB6XV9TUxoDxH/72sFFhqDridkkSb0nzWbc07Wym8JIgRDvaf313FK3Kx/nF8/C7xozl5OuFGwtzRnXyFswoA8Hw1vHwQaRJXeTjyWFTdKKAOpNse+LoZYix2t/0410nBe750lPkvhI5eZ5Votz3P3gzEDR5XUI50HEIfWGGEFXrrVwlLNmPgVeC8qhU8Pu7WCMTxFHLF0alnntCc8= ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-06T04:20:13.188 INFO:teuthology.orchestra.run.smithi045.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaQ80a1msjimWFH42LBkkRUHyrBG+CxhI6lol3TFACE1Nzq2HjAR5zPbKduXNJkpsaAxWkQeZrTuIDzp2q0xY7yCVt5NBvt3KyjMKdaxZ38WMdu1TUYtkPBf+7RwZ/xY5uFsNlY15sglJoV2OfKAqp8Gra1kzPny1SuFFAlcmCbbBf079j7hYTRW5nO2hIdM4Ar9Tjrzd52WnniuLeX37zAlr1s35VkdznYI81ZZqNJkk01NhgICJnLl2P2Hn/ysoEbnQ2UwXjq4BlCjebWRcXHVTmnQ34t0L3nBJqIVhfidqJ5IB6XV9TUxoDxH/72sFFhqDridkkSb0nzWbc07Wym8JIgRDvaf313FK3Kx/nF8/C7xozl5OuFGwtzRnXyFswoA8Hw1vHwQaRJXeTjyWFTdKKAOpNse+LoZYix2t/0410nBe750lPkvhI5eZ5Votz3P3gzEDR5XUI50HEIfWGGEFXrrVwlLNmPgVeC8qhU8Pu7WCMTxFHLF0alnntCc8= ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:20:13.202 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaQ80a1msjimWFH42LBkkRUHyrBG+CxhI6lol3TFACE1Nzq2HjAR5zPbKduXNJkpsaAxWkQeZrTuIDzp2q0xY7yCVt5NBvt3KyjMKdaxZ38WMdu1TUYtkPBf+7RwZ/xY5uFsNlY15sglJoV2OfKAqp8Gra1kzPny1SuFFAlcmCbbBf079j7hYTRW5nO2hIdM4Ar9Tjrzd52WnniuLeX37zAlr1s35VkdznYI81ZZqNJkk01NhgICJnLl2P2Hn/ysoEbnQ2UwXjq4BlCjebWRcXHVTmnQ34t0L3nBJqIVhfidqJ5IB6XV9TUxoDxH/72sFFhqDridkkSb0nzWbc07Wym8JIgRDvaf313FK3Kx/nF8/C7xozl5OuFGwtzRnXyFswoA8Hw1vHwQaRJXeTjyWFTdKKAOpNse+LoZYix2t/0410nBe750lPkvhI5eZ5Votz3P3gzEDR5XUI50HEIfWGGEFXrrVwlLNmPgVeC8qhU8Pu7WCMTxFHLF0alnntCc8= ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-06T04:20:13.244 INFO:teuthology.orchestra.run.smithi086.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaQ80a1msjimWFH42LBkkRUHyrBG+CxhI6lol3TFACE1Nzq2HjAR5zPbKduXNJkpsaAxWkQeZrTuIDzp2q0xY7yCVt5NBvt3KyjMKdaxZ38WMdu1TUYtkPBf+7RwZ/xY5uFsNlY15sglJoV2OfKAqp8Gra1kzPny1SuFFAlcmCbbBf079j7hYTRW5nO2hIdM4Ar9Tjrzd52WnniuLeX37zAlr1s35VkdznYI81ZZqNJkk01NhgICJnLl2P2Hn/ysoEbnQ2UwXjq4BlCjebWRcXHVTmnQ34t0L3nBJqIVhfidqJ5IB6XV9TUxoDxH/72sFFhqDridkkSb0nzWbc07Wym8JIgRDvaf313FK3Kx/nF8/C7xozl5OuFGwtzRnXyFswoA8Hw1vHwQaRJXeTjyWFTdKKAOpNse+LoZYix2t/0410nBe750lPkvhI5eZ5Votz3P3gzEDR5XUI50HEIfWGGEFXrrVwlLNmPgVeC8qhU8Pu7WCMTxFHLF0alnntCc8= ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:20:13.256 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaQ80a1msjimWFH42LBkkRUHyrBG+CxhI6lol3TFACE1Nzq2HjAR5zPbKduXNJkpsaAxWkQeZrTuIDzp2q0xY7yCVt5NBvt3KyjMKdaxZ38WMdu1TUYtkPBf+7RwZ/xY5uFsNlY15sglJoV2OfKAqp8Gra1kzPny1SuFFAlcmCbbBf079j7hYTRW5nO2hIdM4Ar9Tjrzd52WnniuLeX37zAlr1s35VkdznYI81ZZqNJkk01NhgICJnLl2P2Hn/ysoEbnQ2UwXjq4BlCjebWRcXHVTmnQ34t0L3nBJqIVhfidqJ5IB6XV9TUxoDxH/72sFFhqDridkkSb0nzWbc07Wym8JIgRDvaf313FK3Kx/nF8/C7xozl5OuFGwtzRnXyFswoA8Hw1vHwQaRJXeTjyWFTdKKAOpNse+LoZYix2t/0410nBe750lPkvhI5eZ5Votz3P3gzEDR5XUI50HEIfWGGEFXrrVwlLNmPgVeC8qhU8Pu7WCMTxFHLF0alnntCc8= ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-06T04:20:13.298 INFO:teuthology.orchestra.run.smithi154.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaQ80a1msjimWFH42LBkkRUHyrBG+CxhI6lol3TFACE1Nzq2HjAR5zPbKduXNJkpsaAxWkQeZrTuIDzp2q0xY7yCVt5NBvt3KyjMKdaxZ38WMdu1TUYtkPBf+7RwZ/xY5uFsNlY15sglJoV2OfKAqp8Gra1kzPny1SuFFAlcmCbbBf079j7hYTRW5nO2hIdM4Ar9Tjrzd52WnniuLeX37zAlr1s35VkdznYI81ZZqNJkk01NhgICJnLl2P2Hn/ysoEbnQ2UwXjq4BlCjebWRcXHVTmnQ34t0L3nBJqIVhfidqJ5IB6XV9TUxoDxH/72sFFhqDridkkSb0nzWbc07Wym8JIgRDvaf313FK3Kx/nF8/C7xozl5OuFGwtzRnXyFswoA8Hw1vHwQaRJXeTjyWFTdKKAOpNse+LoZYix2t/0410nBe750lPkvhI5eZ5Votz3P3gzEDR5XUI50HEIfWGGEFXrrVwlLNmPgVeC8qhU8Pu7WCMTxFHLF0alnntCc8= ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:20:13.311 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-06T04:20:13.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:13 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1840702552' entity='client.admin' 2024-06-06T04:20:13.561 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:20:15.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.859+0000 7f5ce590f700 1 -- 172.21.15.45:0/603575467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce0100e10 msgr2=0x7f5ce0101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:15.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.859+0000 7f5ce590f700 1 --2- 172.21.15.45:0/603575467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce0100e10 0x7f5ce0101230 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009910 tx=0x7f5cd0009c20 comp rx=0 tx=0).stop 2024-06-06T04:20:15.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.859+0000 7f5ce590f700 1 -- 172.21.15.45:0/603575467 shutdown_connections 2024-06-06T04:20:15.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.860+0000 7f5ce590f700 1 --2- 172.21.15.45:0/603575467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce0100e10 0x7f5ce0101230 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:15.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.860+0000 7f5ce590f700 1 -- 172.21.15.45:0/603575467 >> 172.21.15.45:0/603575467 conn(0x7f5ce00fc3d0 msgr2=0x7f5ce00fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:15.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.860+0000 7f5ce590f700 1 -- 172.21.15.45:0/603575467 shutdown_connections 2024-06-06T04:20:15.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.860+0000 7f5ce590f700 1 -- 172.21.15.45:0/603575467 wait complete. 2024-06-06T04:20:15.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.860+0000 7f5ce590f700 1 Processor -- start 2024-06-06T04:20:15.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.860+0000 7f5ce590f700 1 -- start start 2024-06-06T04:20:15.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.860+0000 7f5ce590f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce0100e10 0x7f5ce0197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:15.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.860+0000 7f5ce590f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0106cd0 con 0x7f5ce0100e10 2024-06-06T04:20:15.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.861+0000 7f5cdeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce0100e10 0x7f5ce0197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:15.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.861+0000 7f5cdeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce0100e10 0x7f5ce0197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54412/0 (socket says 172.21.15.45:54412) 2024-06-06T04:20:15.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.861+0000 7f5cdeffd700 1 -- 172.21.15.45:0/3651456645 learned_addr learned my addr 172.21.15.45:0/3651456645 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:15.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.861+0000 7f5cdeffd700 1 -- 172.21.15.45:0/3651456645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce0197ac0 con 0x7f5ce0100e10 2024-06-06T04:20:15.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.861+0000 7f5cdeffd700 1 --2- 172.21.15.45:0/3651456645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce0100e10 0x7f5ce0197580 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0005b40 tx=0x7f5cd0004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:15.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.861+0000 7f5ce490d700 1 -- 172.21.15.45:0/3651456645 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cd0027070 con 0x7f5ce0100e10 2024-06-06T04:20:15.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.861+0000 7f5ce490d700 1 -- 172.21.15.45:0/3651456645 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cd000be10 con 0x7f5ce0100e10 2024-06-06T04:20:15.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.861+0000 7f5ce590f700 1 -- 172.21.15.45:0/3651456645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce0197d50 con 0x7f5ce0100e10 2024-06-06T04:20:15.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.861+0000 7f5ce490d700 1 -- 172.21.15.45:0/3651456645 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cd0022d70 con 0x7f5ce0100e10 2024-06-06T04:20:15.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.861+0000 7f5ce590f700 1 -- 172.21.15.45:0/3651456645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce019aa40 con 0x7f5ce0100e10 2024-06-06T04:20:15.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.862+0000 7f5ce490d700 1 -- 172.21.15.45:0/3651456645 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5cd002a070 con 0x7f5ce0100e10 2024-06-06T04:20:15.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.862+0000 7f5ce590f700 1 -- 172.21.15.45:0/3651456645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce0107450 con 0x7f5ce0100e10 2024-06-06T04:20:15.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.862+0000 7f5ce490d700 1 --2- 172.21.15.45:0/3651456645 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5ccc039150 0x7f5ccc03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:15.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.862+0000 7f5ce490d700 1 -- 172.21.15.45:0/3651456645 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5cd001e030 con 0x7f5ce0100e10 2024-06-06T04:20:15.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.862+0000 7f5cd7fff700 1 --2- 172.21.15.45:0/3651456645 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5ccc039150 0x7f5ccc03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:15.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.863+0000 7f5cd7fff700 1 --2- 172.21.15.45:0/3651456645 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5ccc039150 0x7f5ccc03b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8006fd0 tx=0x7f5cc8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:15.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:15.867+0000 7f5ce490d700 1 -- 172.21.15.45:0/3651456645 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5cd0013070 con 0x7f5ce0100e10 2024-06-06T04:20:16.041 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:15 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:16.042 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:15 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:16.042 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:15 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:20:16.042 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:15 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:16.042 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:15 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:20:16.042 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:15 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:16.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:16.073+0000 7f5ce590f700 1 -- 172.21.15.45:0/3651456645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f5ce002ddb0 con 0x7f5ce0100e10 2024-06-06T04:20:16.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:16.075+0000 7f5ce490d700 1 -- 172.21.15.45:0/3651456645 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f5cd00310c0 con 0x7f5ce0100e10 2024-06-06T04:20:16.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:16.079+0000 7f5ce590f700 1 -- 172.21.15.45:0/3651456645 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5ccc039150 msgr2=0x7f5ccc03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:16.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:16.079+0000 7f5ce590f700 1 --2- 172.21.15.45:0/3651456645 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5ccc039150 0x7f5ccc03b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8006fd0 tx=0x7f5cc8006e40 comp rx=0 tx=0).stop 2024-06-06T04:20:16.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:16.079+0000 7f5ce590f700 1 -- 172.21.15.45:0/3651456645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce0100e10 msgr2=0x7f5ce0197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:16.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:16.079+0000 7f5ce590f700 1 --2- 172.21.15.45:0/3651456645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce0100e10 0x7f5ce0197580 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0005b40 tx=0x7f5cd0004f80 comp rx=0 tx=0).stop 2024-06-06T04:20:16.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:16.079+0000 7f5ce590f700 1 -- 172.21.15.45:0/3651456645 shutdown_connections 2024-06-06T04:20:16.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:16.079+0000 7f5ce590f700 1 --2- 172.21.15.45:0/3651456645 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5ccc039150 0x7f5ccc03b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:16.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:16.079+0000 7f5ce590f700 1 --2- 172.21.15.45:0/3651456645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ce0100e10 0x7f5ce0197580 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:16.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:16.079+0000 7f5ce590f700 1 -- 172.21.15.45:0/3651456645 >> 172.21.15.45:0/3651456645 conn(0x7f5ce00fc3d0 msgr2=0x7f5ce006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:16.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:16.080+0000 7f5ce590f700 1 -- 172.21.15.45:0/3651456645 shutdown_connections 2024-06-06T04:20:16.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:16.080+0000 7f5ce590f700 1 -- 172.21.15.45:0/3651456645 wait complete. 2024-06-06T04:20:16.756 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-06T04:20:16.756 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-06T04:20:17.347 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:17 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3651456645' entity='client.admin' 2024-06-06T04:20:17.347 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:20:17.348 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:17.448 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:20:18.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.815+0000 7f6a24ac6700 1 -- 172.21.15.45:0/2880573597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20101f30 msgr2=0x7f6a20102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:18.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.815+0000 7f6a24ac6700 1 --2- 172.21.15.45:0/2880573597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20101f30 0x7f6a20102350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009910 tx=0x7f6a10009c20 comp rx=0 tx=0).stop 2024-06-06T04:20:18.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.815+0000 7f6a24ac6700 1 -- 172.21.15.45:0/2880573597 shutdown_connections 2024-06-06T04:20:18.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.815+0000 7f6a24ac6700 1 --2- 172.21.15.45:0/2880573597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20101f30 0x7f6a20102350 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:18.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.815+0000 7f6a24ac6700 1 -- 172.21.15.45:0/2880573597 >> 172.21.15.45:0/2880573597 conn(0x7f6a200fd490 msgr2=0x7f6a200ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:18.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.815+0000 7f6a24ac6700 1 -- 172.21.15.45:0/2880573597 shutdown_connections 2024-06-06T04:20:18.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.815+0000 7f6a24ac6700 1 -- 172.21.15.45:0/2880573597 wait complete. 2024-06-06T04:20:18.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.816+0000 7f6a24ac6700 1 Processor -- start 2024-06-06T04:20:18.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.816+0000 7f6a24ac6700 1 -- start start 2024-06-06T04:20:18.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.816+0000 7f6a24ac6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20101f30 0x7f6a20197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.816+0000 7f6a24ac6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6a20107df0 con 0x7f6a20101f30 2024-06-06T04:20:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.817+0000 7f6a1e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20101f30 0x7f6a20197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.817+0000 7f6a1e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20101f30 0x7f6a20197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45456/0 (socket says 172.21.15.45:45456) 2024-06-06T04:20:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.817+0000 7f6a1e59c700 1 -- 172.21.15.45:0/2148548036 learned_addr learned my addr 172.21.15.45:0/2148548036 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.817+0000 7f6a1e59c700 1 -- 172.21.15.45:0/2148548036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a20197a70 con 0x7f6a20101f30 2024-06-06T04:20:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.817+0000 7f6a1e59c700 1 --2- 172.21.15.45:0/2148548036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20101f30 0x7f6a20197530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6a10005190 tx=0x7f6a10004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.817+0000 7f6a177fe700 1 -- 172.21.15.45:0/2148548036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a10027070 con 0x7f6a20101f30 2024-06-06T04:20:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.817+0000 7f6a24ac6700 1 -- 172.21.15.45:0/2148548036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a20197d00 con 0x7f6a20101f30 2024-06-06T04:20:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.817+0000 7f6a177fe700 1 -- 172.21.15.45:0/2148548036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a10022470 con 0x7f6a20101f30 2024-06-06T04:20:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.817+0000 7f6a177fe700 1 -- 172.21.15.45:0/2148548036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a1000f5f0 con 0x7f6a20101f30 2024-06-06T04:20:18.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.817+0000 7f6a24ac6700 1 -- 172.21.15.45:0/2148548036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2019a9f0 con 0x7f6a20101f30 2024-06-06T04:20:18.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.818+0000 7f6a177fe700 1 -- 172.21.15.45:0/2148548036 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6a1002a070 con 0x7f6a20101f30 2024-06-06T04:20:18.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.818+0000 7f6a24ac6700 1 -- 172.21.15.45:0/2148548036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a20108570 con 0x7f6a20101f30 2024-06-06T04:20:18.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.818+0000 7f6a177fe700 1 --2- 172.21.15.45:0/2148548036 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6a0c0390f0 0x7f6a0c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:18.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.818+0000 7f6a177fe700 1 -- 172.21.15.45:0/2148548036 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6a1001e030 con 0x7f6a20101f30 2024-06-06T04:20:18.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.818+0000 7f6a17fff700 1 --2- 172.21.15.45:0/2148548036 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6a0c0390f0 0x7f6a0c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.819+0000 7f6a17fff700 1 --2- 172.21.15.45:0/2148548036 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6a0c0390f0 0x7f6a0c03b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6a08006fd0 tx=0x7f6a08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:18.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:18.823+0000 7f6a177fe700 1 -- 172.21.15.45:0/2148548036 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6a10013070 con 0x7f6a20101f30 2024-06-06T04:20:19.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:19.034+0000 7f6a24ac6700 1 -- 172.21.15.45:0/2148548036 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f6a2002ddb0 con 0x7f6a0c0390f0 2024-06-06T04:20:19.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:19.037+0000 7f6a177fe700 1 -- 172.21.15.45:0/2148548036 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6a2002ddb0 con 0x7f6a0c0390f0 2024-06-06T04:20:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:19.040+0000 7f6a24ac6700 1 -- 172.21.15.45:0/2148548036 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6a0c0390f0 msgr2=0x7f6a0c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:19.040+0000 7f6a24ac6700 1 --2- 172.21.15.45:0/2148548036 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6a0c0390f0 0x7f6a0c03b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6a08006fd0 tx=0x7f6a08006e40 comp rx=0 tx=0).stop 2024-06-06T04:20:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:19.040+0000 7f6a24ac6700 1 -- 172.21.15.45:0/2148548036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20101f30 msgr2=0x7f6a20197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:19.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:19.040+0000 7f6a24ac6700 1 --2- 172.21.15.45:0/2148548036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20101f30 0x7f6a20197530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6a10005190 tx=0x7f6a10004f80 comp rx=0 tx=0).stop 2024-06-06T04:20:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:19.040+0000 7f6a24ac6700 1 -- 172.21.15.45:0/2148548036 shutdown_connections 2024-06-06T04:20:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:19.040+0000 7f6a24ac6700 1 --2- 172.21.15.45:0/2148548036 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6a0c0390f0 0x7f6a0c03b5b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:19.040+0000 7f6a24ac6700 1 --2- 172.21.15.45:0/2148548036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a20101f30 0x7f6a20197530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:19.040+0000 7f6a24ac6700 1 -- 172.21.15.45:0/2148548036 >> 172.21.15.45:0/2148548036 conn(0x7f6a200fd490 msgr2=0x7f6a200fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:19.040+0000 7f6a24ac6700 1 -- 172.21.15.45:0/2148548036 shutdown_connections 2024-06-06T04:20:19.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:19.040+0000 7f6a24ac6700 1 -- 172.21.15.45:0/2148548036 wait complete. 2024-06-06T04:20:19.712 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi086 2024-06-06T04:20:19.713 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-06-06T04:20:19.713 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/etc/ceph/ceph.conf 2024-06-06T04:20:19.732 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-06-06T04:20:19.732 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T04:20:19.790 INFO:tasks.cephadm:Adding host smithi086 to orchestrator... 2024-06-06T04:20:19.791 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch host add smithi086 2024-06-06T04:20:20.055 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:20:20.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:20 smithi045 ceph-mon[32299]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:20:20.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:20 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:20.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:20 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:20:20.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:20 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:20:20.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:20 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:20:20.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:20 smithi045 ceph-mon[32299]: Updating smithi045:/etc/ceph/ceph.conf 2024-06-06T04:20:21.360 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:21 smithi045 ceph-mon[32299]: Updating smithi045:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T04:20:21.361 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:21 smithi045 ceph-mon[32299]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-06-06T04:20:21.361 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:21 smithi045 ceph-mon[32299]: Updating smithi045:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T04:20:21.361 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:21 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:21.361 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:21 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:21.361 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:21 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.550+0000 7f6114960700 1 -- 172.21.15.45:0/1459349195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6110101040 msgr2=0x7f6110101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.550+0000 7f6114960700 1 --2- 172.21.15.45:0/1459349195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6110101040 0x7f6110101420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6100009910 tx=0x7f6100009c20 comp rx=0 tx=0).stop 2024-06-06T04:20:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.551+0000 7f6114960700 1 -- 172.21.15.45:0/1459349195 shutdown_connections 2024-06-06T04:20:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.551+0000 7f6114960700 1 --2- 172.21.15.45:0/1459349195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6110101040 0x7f6110101420 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.551+0000 7f6114960700 1 -- 172.21.15.45:0/1459349195 >> 172.21.15.45:0/1459349195 conn(0x7f61100fc8f0 msgr2=0x7f61100fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.551+0000 7f6114960700 1 -- 172.21.15.45:0/1459349195 shutdown_connections 2024-06-06T04:20:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.551+0000 7f6114960700 1 -- 172.21.15.45:0/1459349195 wait complete. 2024-06-06T04:20:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.551+0000 7f6114960700 1 Processor -- start 2024-06-06T04:20:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.552+0000 7f6114960700 1 -- start start 2024-06-06T04:20:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.552+0000 7f6114960700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6110101040 0x7f611019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.552+0000 7f6114960700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f611010b2c0 con 0x7f6110101040 2024-06-06T04:20:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.552+0000 7f610e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6110101040 0x7f611019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.552+0000 7f610e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6110101040 0x7f611019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45468/0 (socket says 172.21.15.45:45468) 2024-06-06T04:20:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.552+0000 7f610e59c700 1 -- 172.21.15.45:0/2420180049 learned_addr learned my addr 172.21.15.45:0/2420180049 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.552+0000 7f610e59c700 1 -- 172.21.15.45:0/2420180049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611019dba0 con 0x7f6110101040 2024-06-06T04:20:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.552+0000 7f610e59c700 1 --2- 172.21.15.45:0/2420180049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6110101040 0x7f611019d660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f61000056d0 tx=0x7f61000057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.552+0000 7f60ff7fe700 1 -- 172.21.15.45:0/2420180049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6100026070 con 0x7f6110101040 2024-06-06T04:20:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.553+0000 7f6114960700 1 -- 172.21.15.45:0/2420180049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611019de90 con 0x7f6110101040 2024-06-06T04:20:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.553+0000 7f60ff7fe700 1 -- 172.21.15.45:0/2420180049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f610000bba0 con 0x7f6110101040 2024-06-06T04:20:21.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.553+0000 7f60ff7fe700 1 -- 172.21.15.45:0/2420180049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61000037f0 con 0x7f6110101040 2024-06-06T04:20:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.553+0000 7f6114960700 1 -- 172.21.15.45:0/2420180049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6110197900 con 0x7f6110101040 2024-06-06T04:20:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.553+0000 7f60ff7fe700 1 -- 172.21.15.45:0/2420180049 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6100029070 con 0x7f6110101040 2024-06-06T04:20:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.553+0000 7f6114960700 1 -- 172.21.15.45:0/2420180049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6110102520 con 0x7f6110101040 2024-06-06T04:20:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.554+0000 7f60ff7fe700 1 --2- 172.21.15.45:0/2420180049 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f60f80390f0 0x7f60f803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.554+0000 7f60ff7fe700 1 -- 172.21.15.45:0/2420180049 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f610001e070 con 0x7f6110101040 2024-06-06T04:20:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.554+0000 7f610dd9b700 1 --2- 172.21.15.45:0/2420180049 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f60f80390f0 0x7f60f803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:21.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.554+0000 7f610dd9b700 1 --2- 172.21.15.45:0/2420180049 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f60f80390f0 0x7f60f803b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6104006fd0 tx=0x7f6104006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:21.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.559+0000 7f60ff7fe700 1 -- 172.21.15.45:0/2420180049 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6100013070 con 0x7f6110101040 2024-06-06T04:20:21.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:21.769+0000 7f6114960700 1 -- 172.21.15.45:0/2420180049 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi086", "target": ["mon-mgr", ""]}) -- 0x7f6110198cd0 con 0x7f60f80390f0 2024-06-06T04:20:22.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:22 smithi045 ceph-mon[32299]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi086", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:20:23.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:23 smithi045 ceph-mon[32299]: Deploying cephadm binary to smithi086 2024-06-06T04:20:25.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:25 smithi045 ceph-mon[32299]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:25.985 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi086' with addr '172.21.15.86' 2024-06-06T04:20:25.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:25.984+0000 7f60ff7fe700 1 -- 172.21.15.45:0/2420180049 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f6110198cd0 con 0x7f60f80390f0 2024-06-06T04:20:25.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:25.987+0000 7f6114960700 1 -- 172.21.15.45:0/2420180049 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f60f80390f0 msgr2=0x7f60f803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:25.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:25.987+0000 7f6114960700 1 --2- 172.21.15.45:0/2420180049 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f60f80390f0 0x7f60f803b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6104006fd0 tx=0x7f6104006e40 comp rx=0 tx=0).stop 2024-06-06T04:20:25.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:25.987+0000 7f6114960700 1 -- 172.21.15.45:0/2420180049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6110101040 msgr2=0x7f611019d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:25.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:25.987+0000 7f6114960700 1 --2- 172.21.15.45:0/2420180049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6110101040 0x7f611019d660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f61000056d0 tx=0x7f61000057b0 comp rx=0 tx=0).stop 2024-06-06T04:20:25.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:25.987+0000 7f6114960700 1 -- 172.21.15.45:0/2420180049 shutdown_connections 2024-06-06T04:20:25.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:25.987+0000 7f6114960700 1 --2- 172.21.15.45:0/2420180049 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f60f80390f0 0x7f60f803b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:25.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:25.987+0000 7f6114960700 1 --2- 172.21.15.45:0/2420180049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6110101040 0x7f611019d660 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:25.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:25.987+0000 7f6114960700 1 -- 172.21.15.45:0/2420180049 >> 172.21.15.45:0/2420180049 conn(0x7f61100fc8f0 msgr2=0x7f611006a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:25.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:25.987+0000 7f6114960700 1 -- 172.21.15.45:0/2420180049 shutdown_connections 2024-06-06T04:20:25.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:25.987+0000 7f6114960700 1 -- 172.21.15.45:0/2420180049 wait complete. 2024-06-06T04:20:26.592 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-06T04:20:26.841 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:20:27.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:27.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:26 smithi045 ceph-mon[32299]: Added host smithi086 2024-06-06T04:20:27.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:20:27.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:28.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:28 smithi045 ceph-mon[32299]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:28.153 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:28 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.376+0000 7f0c461cd700 1 -- 172.21.15.45:0/2643923601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c40108070 msgr2=0x7f0c40108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.376+0000 7f0c461cd700 1 --2- 172.21.15.45:0/2643923601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c40108070 0x7f0c40108450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009910 tx=0x7f0c30009c20 comp rx=0 tx=0).stop 2024-06-06T04:20:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.376+0000 7f0c461cd700 1 -- 172.21.15.45:0/2643923601 shutdown_connections 2024-06-06T04:20:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.376+0000 7f0c461cd700 1 --2- 172.21.15.45:0/2643923601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c40108070 0x7f0c40108450 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.376+0000 7f0c461cd700 1 -- 172.21.15.45:0/2643923601 >> 172.21.15.45:0/2643923601 conn(0x7f0c400783c0 msgr2=0x7f0c400787d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.376+0000 7f0c461cd700 1 -- 172.21.15.45:0/2643923601 shutdown_connections 2024-06-06T04:20:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.377+0000 7f0c461cd700 1 -- 172.21.15.45:0/2643923601 wait complete. 2024-06-06T04:20:28.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.377+0000 7f0c461cd700 1 Processor -- start 2024-06-06T04:20:28.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.378+0000 7f0c461cd700 1 -- start start 2024-06-06T04:20:28.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.378+0000 7f0c461cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c40108070 0x7f0c4019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:28.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.378+0000 7f0c461cd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4010c530 con 0x7f0c40108070 2024-06-06T04:20:28.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.378+0000 7f0c3f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c40108070 0x7f0c4019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:28.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.378+0000 7f0c3f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c40108070 0x7f0c4019d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56896/0 (socket says 172.21.15.45:56896) 2024-06-06T04:20:28.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.378+0000 7f0c3f7fe700 1 -- 172.21.15.45:0/3944077610 learned_addr learned my addr 172.21.15.45:0/3944077610 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:28.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.378+0000 7f0c3f7fe700 1 -- 172.21.15.45:0/3944077610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4019db80 con 0x7f0c40108070 2024-06-06T04:20:28.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.378+0000 7f0c3f7fe700 1 --2- 172.21.15.45:0/3944077610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c40108070 0x7f0c4019d640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c300056d0 tx=0x7f0c300057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:28.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.379+0000 7f0c3cff9700 1 -- 172.21.15.45:0/3944077610 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c30026070 con 0x7f0c40108070 2024-06-06T04:20:28.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.379+0000 7f0c3cff9700 1 -- 172.21.15.45:0/3944077610 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c3000bc00 con 0x7f0c40108070 2024-06-06T04:20:28.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.379+0000 7f0c461cd700 1 -- 172.21.15.45:0/3944077610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4019de10 con 0x7f0c40108070 2024-06-06T04:20:28.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.379+0000 7f0c3cff9700 1 -- 172.21.15.45:0/3944077610 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c300039b0 con 0x7f0c40108070 2024-06-06T04:20:28.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.379+0000 7f0c461cd700 1 -- 172.21.15.45:0/3944077610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c40197710 con 0x7f0c40108070 2024-06-06T04:20:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.380+0000 7f0c3cff9700 1 -- 172.21.15.45:0/3944077610 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0c30029070 con 0x7f0c40108070 2024-06-06T04:20:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.380+0000 7f0c461cd700 1 -- 172.21.15.45:0/3944077610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4006a080 con 0x7f0c40108070 2024-06-06T04:20:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.380+0000 7f0c3cff9700 1 --2- 172.21.15.45:0/3944077610 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0c280390f0 0x7f0c2803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.380+0000 7f0c3cff9700 1 -- 172.21.15.45:0/3944077610 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0c3001e070 con 0x7f0c40108070 2024-06-06T04:20:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.380+0000 7f0c3effd700 1 --2- 172.21.15.45:0/3944077610 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0c280390f0 0x7f0c2803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.380+0000 7f0c3effd700 1 --2- 172.21.15.45:0/3944077610 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0c280390f0 0x7f0c2803b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0c34006fd0 tx=0x7f0c34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.385+0000 7f0c3cff9700 1 -- 172.21.15.45:0/3944077610 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0c30013070 con 0x7f0c40108070 2024-06-06T04:20:28.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.595+0000 7f0c461cd700 1 -- 172.21.15.45:0/3944077610 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c40063af0 con 0x7f0c280390f0 2024-06-06T04:20:28.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.597+0000 7f0c3cff9700 1 -- 172.21.15.45:0/3944077610 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f0c40063af0 con 0x7f0c280390f0 2024-06-06T04:20:28.597 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:20:28.597 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}] 2024-06-06T04:20:28.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.599+0000 7f0c461cd700 1 -- 172.21.15.45:0/3944077610 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0c280390f0 msgr2=0x7f0c2803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:28.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.599+0000 7f0c461cd700 1 --2- 172.21.15.45:0/3944077610 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0c280390f0 0x7f0c2803b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0c34006fd0 tx=0x7f0c34006e40 comp rx=0 tx=0).stop 2024-06-06T04:20:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.599+0000 7f0c461cd700 1 -- 172.21.15.45:0/3944077610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c40108070 msgr2=0x7f0c4019d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.599+0000 7f0c461cd700 1 --2- 172.21.15.45:0/3944077610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c40108070 0x7f0c4019d640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c300056d0 tx=0x7f0c300057b0 comp rx=0 tx=0).stop 2024-06-06T04:20:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.599+0000 7f0c461cd700 1 -- 172.21.15.45:0/3944077610 shutdown_connections 2024-06-06T04:20:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.599+0000 7f0c461cd700 1 --2- 172.21.15.45:0/3944077610 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0c280390f0 0x7f0c2803b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.599+0000 7f0c461cd700 1 --2- 172.21.15.45:0/3944077610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c40108070 0x7f0c4019d640 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.599+0000 7f0c461cd700 1 -- 172.21.15.45:0/3944077610 >> 172.21.15.45:0/3944077610 conn(0x7f0c400783c0 msgr2=0x7f0c400fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.599+0000 7f0c461cd700 1 -- 172.21.15.45:0/3944077610 shutdown_connections 2024-06-06T04:20:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:28.599+0000 7f0c461cd700 1 -- 172.21.15.45:0/3944077610 wait complete. 2024-06-06T04:20:29.194 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi154 2024-06-06T04:20:29.195 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-06-06T04:20:29.195 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/etc/ceph/ceph.conf 2024-06-06T04:20:29.215 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-06-06T04:20:29.215 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T04:20:29.273 INFO:tasks.cephadm:Adding host smithi154 to orchestrator... 2024-06-06T04:20:29.273 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch host add smithi154 2024-06-06T04:20:29.519 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:20:29.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:29 smithi045 ceph-mon[32299]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:29.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:29 smithi045 ceph-mon[32299]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:20:29.876 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:29 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:31.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.059+0000 7ff98a2c4700 1 -- 172.21.15.45:0/1935914116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff984103120 msgr2=0x7ff984103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:31.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.059+0000 7ff98a2c4700 1 --2- 172.21.15.45:0/1935914116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff984103120 0x7ff984103500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff974009910 tx=0x7ff974009c20 comp rx=0 tx=0).stop 2024-06-06T04:20:31.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.059+0000 7ff98a2c4700 1 -- 172.21.15.45:0/1935914116 shutdown_connections 2024-06-06T04:20:31.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.059+0000 7ff98a2c4700 1 --2- 172.21.15.45:0/1935914116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff984103120 0x7ff984103500 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:31.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.059+0000 7ff98a2c4700 1 -- 172.21.15.45:0/1935914116 >> 172.21.15.45:0/1935914116 conn(0x7ff9840fc890 msgr2=0x7ff9840fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:31.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.059+0000 7ff98a2c4700 1 -- 172.21.15.45:0/1935914116 shutdown_connections 2024-06-06T04:20:31.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.059+0000 7ff98a2c4700 1 -- 172.21.15.45:0/1935914116 wait complete. 2024-06-06T04:20:31.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.060+0000 7ff98a2c4700 1 Processor -- start 2024-06-06T04:20:31.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.060+0000 7ff98a2c4700 1 -- start start 2024-06-06T04:20:31.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.060+0000 7ff98a2c4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff984103120 0x7ff984100d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:31.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.060+0000 7ff98a2c4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff98410c3f0 con 0x7ff984103120 2024-06-06T04:20:31.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.060+0000 7ff983fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff984103120 0x7ff984100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:31.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.060+0000 7ff983fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff984103120 0x7ff984100d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56906/0 (socket says 172.21.15.45:56906) 2024-06-06T04:20:31.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.060+0000 7ff983fff700 1 -- 172.21.15.45:0/2854346106 learned_addr learned my addr 172.21.15.45:0/2854346106 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:31.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.061+0000 7ff983fff700 1 -- 172.21.15.45:0/2854346106 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff984102c70 con 0x7ff984103120 2024-06-06T04:20:31.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.061+0000 7ff983fff700 1 --2- 172.21.15.45:0/2854346106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff984103120 0x7ff984100d80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff9740056d0 tx=0x7ff9740057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:31.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.061+0000 7ff981ffb700 1 -- 172.21.15.45:0/2854346106 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff974026070 con 0x7ff984103120 2024-06-06T04:20:31.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.061+0000 7ff981ffb700 1 -- 172.21.15.45:0/2854346106 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff974004370 con 0x7ff984103120 2024-06-06T04:20:31.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.061+0000 7ff98a2c4700 1 -- 172.21.15.45:0/2854346106 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9841012c0 con 0x7ff984103120 2024-06-06T04:20:31.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.061+0000 7ff981ffb700 1 -- 172.21.15.45:0/2854346106 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9740039b0 con 0x7ff984103120 2024-06-06T04:20:31.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.061+0000 7ff98a2c4700 1 -- 172.21.15.45:0/2854346106 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff984101780 con 0x7ff984103120 2024-06-06T04:20:31.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.062+0000 7ff981ffb700 1 -- 172.21.15.45:0/2854346106 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff974029070 con 0x7ff984103120 2024-06-06T04:20:31.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.062+0000 7ff98a2c4700 1 -- 172.21.15.45:0/2854346106 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff964005330 con 0x7ff984103120 2024-06-06T04:20:31.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.062+0000 7ff981ffb700 1 --2- 172.21.15.45:0/2854346106 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff9700390f0 0x7ff97003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:31.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.062+0000 7ff981ffb700 1 -- 172.21.15.45:0/2854346106 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff97401e070 con 0x7ff984103120 2024-06-06T04:20:31.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.062+0000 7ff97bfff700 1 --2- 172.21.15.45:0/2854346106 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff9700390f0 0x7ff97003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:31.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.063+0000 7ff97bfff700 1 --2- 172.21.15.45:0/2854346106 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff9700390f0 0x7ff97003b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff96c006fd0 tx=0x7ff96c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:31.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.067+0000 7ff981ffb700 1 -- 172.21.15.45:0/2854346106 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff974013070 con 0x7ff984103120 2024-06-06T04:20:31.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:31.280+0000 7ff98a2c4700 1 -- 172.21.15.45:0/2854346106 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi154", "target": ["mon-mgr", ""]}) -- 0x7ff964000bc0 con 0x7ff9700390f0 2024-06-06T04:20:31.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:31 smithi045 ceph-mon[32299]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:33.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:32 smithi045 ceph-mon[32299]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi154", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:20:33.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:32 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:33.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:32 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:33.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:32 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:33.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:32 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:33.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:32 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:20:33.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:32 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:20:33.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:32 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:20:33.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:32 smithi045 ceph-mon[32299]: Updating smithi086:/etc/ceph/ceph.conf 2024-06-06T04:20:33.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:32 smithi045 ceph-mon[32299]: Deploying cephadm binary to smithi154 2024-06-06T04:20:33.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:32 smithi045 ceph-mon[32299]: Updating smithi086:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T04:20:34.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:33 smithi045 ceph-mon[32299]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:34.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:33 smithi045 ceph-mon[32299]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-06-06T04:20:35.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:35.582+0000 7ff981ffb700 1 -- 172.21.15.45:0/2854346106 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff964000bc0 con 0x7ff9700390f0 2024-06-06T04:20:35.583 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi154' with addr '172.21.15.154' 2024-06-06T04:20:35.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:35.584+0000 7ff98a2c4700 1 -- 172.21.15.45:0/2854346106 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff9700390f0 msgr2=0x7ff97003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:35.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:35.584+0000 7ff98a2c4700 1 --2- 172.21.15.45:0/2854346106 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff9700390f0 0x7ff97003b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff96c006fd0 tx=0x7ff96c006e40 comp rx=0 tx=0).stop 2024-06-06T04:20:35.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:35.584+0000 7ff98a2c4700 1 -- 172.21.15.45:0/2854346106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff984103120 msgr2=0x7ff984100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:35.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:35.584+0000 7ff98a2c4700 1 --2- 172.21.15.45:0/2854346106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff984103120 0x7ff984100d80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff9740056d0 tx=0x7ff9740057b0 comp rx=0 tx=0).stop 2024-06-06T04:20:35.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:35.584+0000 7ff98a2c4700 1 -- 172.21.15.45:0/2854346106 shutdown_connections 2024-06-06T04:20:35.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:35.584+0000 7ff98a2c4700 1 --2- 172.21.15.45:0/2854346106 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff9700390f0 0x7ff97003b5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:35.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:35.584+0000 7ff98a2c4700 1 --2- 172.21.15.45:0/2854346106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff984103120 0x7ff984100d80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:35.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:35.584+0000 7ff98a2c4700 1 -- 172.21.15.45:0/2854346106 >> 172.21.15.45:0/2854346106 conn(0x7ff9840fc890 msgr2=0x7ff984103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:35.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:35.585+0000 7ff98a2c4700 1 -- 172.21.15.45:0/2854346106 shutdown_connections 2024-06-06T04:20:35.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:35.585+0000 7ff98a2c4700 1 -- 172.21.15.45:0/2854346106 wait complete. 2024-06-06T04:20:36.350 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:36 smithi045 ceph-mon[32299]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:36.351 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:36 smithi045 ceph-mon[32299]: Updating smithi086:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T04:20:36.351 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:36 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:36.351 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:36 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:36.351 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:36 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:36.351 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:36 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:36.351 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:36 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:20:36.351 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:36 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:36.352 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-06T04:20:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:20:37.382 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:37 smithi045 ceph-mon[32299]: Added host smithi154 2024-06-06T04:20:37.382 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:37 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.160+0000 7f3b78053700 1 -- 172.21.15.45:0/2508085312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b70101000 msgr2=0x7f3b701013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.160+0000 7f3b78053700 1 --2- 172.21.15.45:0/2508085312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b70101000 0x7f3b701013e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009910 tx=0x7f3b6c009c20 comp rx=0 tx=0).stop 2024-06-06T04:20:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.160+0000 7f3b78053700 1 -- 172.21.15.45:0/2508085312 shutdown_connections 2024-06-06T04:20:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.160+0000 7f3b78053700 1 --2- 172.21.15.45:0/2508085312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b70101000 0x7f3b701013e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.160+0000 7f3b78053700 1 -- 172.21.15.45:0/2508085312 >> 172.21.15.45:0/2508085312 conn(0x7f3b700fc890 msgr2=0x7f3b700fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.160+0000 7f3b78053700 1 -- 172.21.15.45:0/2508085312 shutdown_connections 2024-06-06T04:20:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.160+0000 7f3b78053700 1 -- 172.21.15.45:0/2508085312 wait complete. 2024-06-06T04:20:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.161+0000 7f3b78053700 1 Processor -- start 2024-06-06T04:20:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.161+0000 7f3b78053700 1 -- start start 2024-06-06T04:20:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.161+0000 7f3b78053700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b70101000 0x7f3b7019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.161+0000 7f3b78053700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7010b280 con 0x7f3b70101000 2024-06-06T04:20:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.161+0000 7f3b75def700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b70101000 0x7f3b7019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.161+0000 7f3b75def700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b70101000 0x7f3b7019c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52768/0 (socket says 172.21.15.45:52768) 2024-06-06T04:20:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.161+0000 7f3b75def700 1 -- 172.21.15.45:0/4197754788 learned_addr learned my addr 172.21.15.45:0/4197754788 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.162+0000 7f3b75def700 1 -- 172.21.15.45:0/4197754788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7019c960 con 0x7f3b70101000 2024-06-06T04:20:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.162+0000 7f3b75def700 1 --2- 172.21.15.45:0/4197754788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b70101000 0x7f3b7019c420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0056d0 tx=0x7f3b6c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.162+0000 7f3b66ffd700 1 -- 172.21.15.45:0/4197754788 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b6c026070 con 0x7f3b70101000 2024-06-06T04:20:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.162+0000 7f3b66ffd700 1 -- 172.21.15.45:0/4197754788 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b6c00bba0 con 0x7f3b70101000 2024-06-06T04:20:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.162+0000 7f3b78053700 1 -- 172.21.15.45:0/4197754788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b7019cc50 con 0x7f3b70101000 2024-06-06T04:20:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.162+0000 7f3b66ffd700 1 -- 172.21.15.45:0/4197754788 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b6c0037f0 con 0x7f3b70101000 2024-06-06T04:20:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.163+0000 7f3b78053700 1 -- 172.21.15.45:0/4197754788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b701965e0 con 0x7f3b70101000 2024-06-06T04:20:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.164+0000 7f3b78053700 1 -- 172.21.15.45:0/4197754788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b701024e0 con 0x7f3b70101000 2024-06-06T04:20:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.165+0000 7f3b66ffd700 1 -- 172.21.15.45:0/4197754788 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3b6c029070 con 0x7f3b70101000 2024-06-06T04:20:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.165+0000 7f3b66ffd700 1 --2- 172.21.15.45:0/4197754788 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b5c039150 0x7f3b5c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.165+0000 7f3b66ffd700 1 -- 172.21.15.45:0/4197754788 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3b6c01e070 con 0x7f3b70101000 2024-06-06T04:20:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.166+0000 7f3b755ee700 1 --2- 172.21.15.45:0/4197754788 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b5c039150 0x7f3b5c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.166+0000 7f3b755ee700 1 --2- 172.21.15.45:0/4197754788 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b5c039150 0x7f3b5c03b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3b60006fd0 tx=0x7f3b60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.170+0000 7f3b66ffd700 1 -- 172.21.15.45:0/4197754788 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3b6c030020 con 0x7f3b70101000 2024-06-06T04:20:38.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:38 smithi045 ceph-mon[32299]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:38.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.384+0000 7f3b78053700 1 -- 172.21.15.45:0/4197754788 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b70197b10 con 0x7f3b5c039150 2024-06-06T04:20:38.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.385+0000 7f3b66ffd700 1 -- 172.21.15.45:0/4197754788 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f3b70197b10 con 0x7f3b5c039150 2024-06-06T04:20:38.386 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:20:38.386 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.154", "hostname": "smithi154", "labels": [], "status": ""}] 2024-06-06T04:20:38.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.388+0000 7f3b78053700 1 -- 172.21.15.45:0/4197754788 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b5c039150 msgr2=0x7f3b5c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:38.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.388+0000 7f3b78053700 1 --2- 172.21.15.45:0/4197754788 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b5c039150 0x7f3b5c03b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3b60006fd0 tx=0x7f3b60006e40 comp rx=0 tx=0).stop 2024-06-06T04:20:38.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.388+0000 7f3b78053700 1 -- 172.21.15.45:0/4197754788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b70101000 msgr2=0x7f3b7019c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:38.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.388+0000 7f3b78053700 1 --2- 172.21.15.45:0/4197754788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b70101000 0x7f3b7019c420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0056d0 tx=0x7f3b6c0057b0 comp rx=0 tx=0).stop 2024-06-06T04:20:38.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.388+0000 7f3b78053700 1 -- 172.21.15.45:0/4197754788 shutdown_connections 2024-06-06T04:20:38.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.388+0000 7f3b78053700 1 --2- 172.21.15.45:0/4197754788 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b5c039150 0x7f3b5c03b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:38.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.388+0000 7f3b78053700 1 --2- 172.21.15.45:0/4197754788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b70101000 0x7f3b7019c420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:38.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.388+0000 7f3b78053700 1 -- 172.21.15.45:0/4197754788 >> 172.21.15.45:0/4197754788 conn(0x7f3b700fc890 msgr2=0x7f3b70102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:38.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.388+0000 7f3b78053700 1 -- 172.21.15.45:0/4197754788 shutdown_connections 2024-06-06T04:20:38.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:38.388+0000 7f3b78053700 1 -- 172.21.15.45:0/4197754788 wait complete. 2024-06-06T04:20:38.935 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-06T04:20:38.935 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-06T04:20:39.176 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:20:39.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:39 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:39.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:39 smithi045 ceph-mon[32299]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:39.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:39 smithi045 ceph-mon[32299]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:20:40.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.729+0000 7f2675732700 1 -- 172.21.15.45:0/3306281033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2670107f30 msgr2=0x7f2670108310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:40.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.729+0000 7f2675732700 1 --2- 172.21.15.45:0/3306281033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2670107f30 0x7f2670108310 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f266000b0a0 tx=0x7f266000b3b0 comp rx=0 tx=0).stop 2024-06-06T04:20:40.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.729+0000 7f2675732700 1 -- 172.21.15.45:0/3306281033 shutdown_connections 2024-06-06T04:20:40.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.729+0000 7f2675732700 1 --2- 172.21.15.45:0/3306281033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2670107f30 0x7f2670108310 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:40.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.729+0000 7f2675732700 1 -- 172.21.15.45:0/3306281033 >> 172.21.15.45:0/3306281033 conn(0x7f26700768f0 msgr2=0x7f2670076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.729+0000 7f2675732700 1 -- 172.21.15.45:0/3306281033 shutdown_connections 2024-06-06T04:20:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.729+0000 7f2675732700 1 -- 172.21.15.45:0/3306281033 wait complete. 2024-06-06T04:20:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.730+0000 7f2675732700 1 Processor -- start 2024-06-06T04:20:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.730+0000 7f2675732700 1 -- start start 2024-06-06T04:20:40.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.730+0000 7f2675732700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2670107f30 0x7f267019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:40.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.730+0000 7f2675732700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f267010c370 con 0x7f2670107f30 2024-06-06T04:20:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.730+0000 7f266effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2670107f30 0x7f267019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.730+0000 7f266effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2670107f30 0x7f267019d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52796/0 (socket says 172.21.15.45:52796) 2024-06-06T04:20:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.730+0000 7f266effd700 1 -- 172.21.15.45:0/4070842472 learned_addr learned my addr 172.21.15.45:0/4070842472 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:20:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.731+0000 7f266effd700 1 -- 172.21.15.45:0/4070842472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267019db10 con 0x7f2670107f30 2024-06-06T04:20:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.731+0000 7f266effd700 1 --2- 172.21.15.45:0/4070842472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2670107f30 0x7f267019d5d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f266000d010 tx=0x7f2660009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.731+0000 7f2657fff700 1 -- 172.21.15.45:0/4070842472 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2660017040 con 0x7f2670107f30 2024-06-06T04:20:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.731+0000 7f2675732700 1 -- 172.21.15.45:0/4070842472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267019dda0 con 0x7f2670107f30 2024-06-06T04:20:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.731+0000 7f2657fff700 1 -- 172.21.15.45:0/4070842472 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2660022070 con 0x7f2670107f30 2024-06-06T04:20:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.731+0000 7f2657fff700 1 -- 172.21.15.45:0/4070842472 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f266001d5e0 con 0x7f2670107f30 2024-06-06T04:20:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.731+0000 7f2675732700 1 -- 172.21.15.45:0/4070842472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26701976a0 con 0x7f2670107f30 2024-06-06T04:20:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.732+0000 7f2657fff700 1 -- 172.21.15.45:0/4070842472 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f266001d8e0 con 0x7f2670107f30 2024-06-06T04:20:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.732+0000 7f2657fff700 1 --2- 172.21.15.45:0/4070842472 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2658039140 0x7f265803b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.732+0000 7f2657fff700 1 -- 172.21.15.45:0/4070842472 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f266000f050 con 0x7f2670107f30 2024-06-06T04:20:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.732+0000 7f2675732700 1 -- 172.21.15.45:0/4070842472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26701035e0 con 0x7f2670107f30 2024-06-06T04:20:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.732+0000 7f266e7fc700 1 --2- 172.21.15.45:0/4070842472 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2658039140 0x7f265803b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:40.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.733+0000 7f266e7fc700 1 --2- 172.21.15.45:0/4070842472 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2658039140 0x7f265803b600 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2664006fd0 tx=0x7f2664006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:40.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.737+0000 7f2657fff700 1 -- 172.21.15.45:0/4070842472 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2660025c20 con 0x7f2670107f30 2024-06-06T04:20:40.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:40.948+0000 7f2675732700 1 -- 172.21.15.45:0/4070842472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f2670063af0 con 0x7f2670107f30 2024-06-06T04:20:41.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:41.329+0000 7f2657fff700 1 -- 172.21.15.45:0/4070842472 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f266002d250 con 0x7f2670107f30 2024-06-06T04:20:41.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:41.332+0000 7f2675732700 1 -- 172.21.15.45:0/4070842472 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2658039140 msgr2=0x7f265803b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:41.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:41.332+0000 7f2675732700 1 --2- 172.21.15.45:0/4070842472 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2658039140 0x7f265803b600 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2664006fd0 tx=0x7f2664006e40 comp rx=0 tx=0).stop 2024-06-06T04:20:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:41.332+0000 7f2675732700 1 -- 172.21.15.45:0/4070842472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2670107f30 msgr2=0x7f267019d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:41.332+0000 7f2675732700 1 --2- 172.21.15.45:0/4070842472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2670107f30 0x7f267019d5d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f266000d010 tx=0x7f2660009790 comp rx=0 tx=0).stop 2024-06-06T04:20:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:41.332+0000 7f2675732700 1 -- 172.21.15.45:0/4070842472 shutdown_connections 2024-06-06T04:20:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:41.332+0000 7f2675732700 1 --2- 172.21.15.45:0/4070842472 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2658039140 0x7f265803b600 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:41.332+0000 7f2675732700 1 --2- 172.21.15.45:0/4070842472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2670107f30 0x7f267019d5d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:41.332+0000 7f2675732700 1 -- 172.21.15.45:0/4070842472 >> 172.21.15.45:0/4070842472 conn(0x7f26700768f0 msgr2=0x7f26700fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:41.332+0000 7f2675732700 1 -- 172.21.15.45:0/4070842472 shutdown_connections 2024-06-06T04:20:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:20:41.332+0000 7f2675732700 1 -- 172.21.15.45:0/4070842472 wait complete. 2024-06-06T04:20:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2024-06-06T04:20:41.655 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:41 smithi045 ceph-mon[32299]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:41.655 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:41 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/4070842472' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-06T04:20:41.656 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:41.656 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:41.656 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:41.656 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:41.656 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi154", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:20:41.656 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:20:41.656 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:20:41.984 INFO:tasks.cephadm:Adding mon.a on smithi045 2024-06-06T04:20:41.985 INFO:tasks.cephadm:Adding mon.b on smithi086 2024-06-06T04:20:41.985 INFO:tasks.cephadm:Adding mon.c on smithi154 2024-06-06T04:20:41.985 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi045:172.21.15.45=a;smithi086:172.21.15.86=b;smithi154:172.21.15.154=c' 2024-06-06T04:20:42.219 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T04:20:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:42 smithi045 ceph-mon[32299]: Updating smithi154:/etc/ceph/ceph.conf 2024-06-06T04:20:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:42 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/4070842472' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-06T04:20:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:42 smithi045 ceph-mon[32299]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T04:20:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:42 smithi045 ceph-mon[32299]: Updating smithi154:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T04:20:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:42 smithi045 ceph-mon[32299]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-06-06T04:20:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:42 smithi045 ceph-mon[32299]: Updating smithi154:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T04:20:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:42 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:42 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:42 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:43.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.583+0000 7f95b28eb700 1 -- 172.21.15.154:0/3789704349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95ac102100 msgr2=0x7f95ac1024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:43.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.583+0000 7f95b28eb700 1 --2- 172.21.15.154:0/3789704349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95ac102100 0x7f95ac1024e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f959c009910 tx=0x7f959c009c20 comp rx=0 tx=0).stop 2024-06-06T04:20:43.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.583+0000 7f95b28eb700 1 -- 172.21.15.154:0/3789704349 shutdown_connections 2024-06-06T04:20:43.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.583+0000 7f95b28eb700 1 --2- 172.21.15.154:0/3789704349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95ac102100 0x7f95ac1024e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:43.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.583+0000 7f95b28eb700 1 -- 172.21.15.154:0/3789704349 >> 172.21.15.154:0/3789704349 conn(0x7f95ac0fd9d0 msgr2=0x7f95ac0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:43.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.583+0000 7f95b28eb700 1 -- 172.21.15.154:0/3789704349 shutdown_connections 2024-06-06T04:20:43.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.583+0000 7f95b28eb700 1 -- 172.21.15.154:0/3789704349 wait complete. 2024-06-06T04:20:43.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.584+0000 7f95b28eb700 1 Processor -- start 2024-06-06T04:20:43.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.584+0000 7f95b28eb700 1 -- start start 2024-06-06T04:20:43.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.584+0000 7f95b28eb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95ac102100 0x7f95ac074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:43.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.584+0000 7f95b28eb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac10c380 con 0x7f95ac102100 2024-06-06T04:20:43.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.585+0000 7f95abfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95ac102100 0x7f95ac074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:43.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.585+0000 7f95abfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95ac102100 0x7f95ac074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.154:58508/0 (socket says 172.21.15.154:58508) 2024-06-06T04:20:43.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.585+0000 7f95abfff700 1 -- 172.21.15.154:0/2301220158 learned_addr learned my addr 172.21.15.154:0/2301220158 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-06-06T04:20:43.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.585+0000 7f95abfff700 1 -- 172.21.15.154:0/2301220158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95ac076680 con 0x7f95ac102100 2024-06-06T04:20:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.585+0000 7f95abfff700 1 --2- 172.21.15.154:0/2301220158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95ac102100 0x7f95ac074790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f959c0056d0 tx=0x7f959c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.586+0000 7f95a97fa700 1 -- 172.21.15.154:0/2301220158 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f959c026070 con 0x7f95ac102100 2024-06-06T04:20:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.586+0000 7f95a97fa700 1 -- 172.21.15.154:0/2301220158 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f959c00bc00 con 0x7f95ac102100 2024-06-06T04:20:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.586+0000 7f95b28eb700 1 -- 172.21.15.154:0/2301220158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95ac074d30 con 0x7f95ac102100 2024-06-06T04:20:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.586+0000 7f95a97fa700 1 -- 172.21.15.154:0/2301220158 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f959c0039b0 con 0x7f95ac102100 2024-06-06T04:20:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.586+0000 7f95b28eb700 1 -- 172.21.15.154:0/2301220158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95ac0751f0 con 0x7f95ac102100 2024-06-06T04:20:43.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.587+0000 7f95b28eb700 1 -- 172.21.15.154:0/2301220158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95ac1035e0 con 0x7f95ac102100 2024-06-06T04:20:43.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.587+0000 7f95a97fa700 1 -- 172.21.15.154:0/2301220158 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f959c029070 con 0x7f95ac102100 2024-06-06T04:20:43.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.587+0000 7f95a97fa700 1 --2- 172.21.15.154:0/2301220158 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f95940390f0 0x7f959403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:43.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.587+0000 7f95a97fa700 1 -- 172.21.15.154:0/2301220158 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f959c01e070 con 0x7f95ac102100 2024-06-06T04:20:43.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.587+0000 7f95ab7fe700 1 --2- 172.21.15.154:0/2301220158 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f95940390f0 0x7f959403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:43.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.588+0000 7f95ab7fe700 1 --2- 172.21.15.154:0/2301220158 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f95940390f0 0x7f959403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f95a0006fd0 tx=0x7f95a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:43.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.593+0000 7f95a97fa700 1 -- 172.21.15.154:0/2301220158 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f959c013070 con 0x7f95ac102100 2024-06-06T04:20:43.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.800+0000 7f95b28eb700 1 -- 172.21.15.154:0/2301220158 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi086:172.21.15.86=b;smithi154:172.21.15.154=c", "target": ["mon-mgr", ""]}) -- 0x7f95ac075480 con 0x7f95940390f0 2024-06-06T04:20:43.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.806+0000 7f95a97fa700 1 -- 172.21.15.154:0/2301220158 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f95ac075480 con 0x7f95940390f0 2024-06-06T04:20:43.807 INFO:teuthology.orchestra.run.smithi154.stdout:Scheduled mon update... 2024-06-06T04:20:43.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.808+0000 7f95b28eb700 1 -- 172.21.15.154:0/2301220158 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f95940390f0 msgr2=0x7f959403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:43.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.808+0000 7f95b28eb700 1 --2- 172.21.15.154:0/2301220158 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f95940390f0 0x7f959403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f95a0006fd0 tx=0x7f95a0006e40 comp rx=0 tx=0).stop 2024-06-06T04:20:43.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.808+0000 7f95b28eb700 1 -- 172.21.15.154:0/2301220158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95ac102100 msgr2=0x7f95ac074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:43.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.808+0000 7f95b28eb700 1 --2- 172.21.15.154:0/2301220158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95ac102100 0x7f95ac074790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f959c0056d0 tx=0x7f959c0057b0 comp rx=0 tx=0).stop 2024-06-06T04:20:43.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.808+0000 7f95b28eb700 1 -- 172.21.15.154:0/2301220158 shutdown_connections 2024-06-06T04:20:43.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.808+0000 7f95b28eb700 1 --2- 172.21.15.154:0/2301220158 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f95940390f0 0x7f959403b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:43.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.808+0000 7f95b28eb700 1 --2- 172.21.15.154:0/2301220158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95ac102100 0x7f95ac074790 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:43.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.808+0000 7f95b28eb700 1 -- 172.21.15.154:0/2301220158 >> 172.21.15.154:0/2301220158 conn(0x7f95ac0fd9d0 msgr2=0x7f95ac103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:43.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.808+0000 7f95b28eb700 1 -- 172.21.15.154:0/2301220158 shutdown_connections 2024-06-06T04:20:43.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:43.808+0000 7f95b28eb700 1 -- 172.21.15.154:0/2301220158 wait complete. 2024-06-06T04:20:44.385 DEBUG:teuthology.orchestra.run.smithi086:mon.b> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mon.b.service 2024-06-06T04:20:44.388 DEBUG:teuthology.orchestra.run.smithi154:mon.c> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mon.c.service 2024-06-06T04:20:44.390 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-06T04:20:44.390 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-06T04:20:44.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:44 smithi045 ceph-mon[32299]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:44.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:44 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:44.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:44 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:20:44.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:44 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:20:44.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:44 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:20:44.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:44 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:44.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:44 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T04:20:44.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:44 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:20:45.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:45 smithi045 ceph-mon[32299]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi086:172.21.15.86=b;smithi154:172.21.15.154=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:20:45.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:45 smithi045 ceph-mon[32299]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi086:172.21.15.86=b;smithi154:172.21.15.154=c;count:3 2024-06-06T04:20:45.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:45 smithi045 ceph-mon[32299]: Deploying daemon mon.c on smithi154 2024-06-06T04:20:45.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:45 smithi045 ceph-mon[32299]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:45.485 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T04:20:47.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:47 smithi045 ceph-mon[32299]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:48.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.101+0000 7ff7c571e700 1 -- 172.21.15.154:0/1270822762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c0102130 msgr2=0x7ff7c0102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:48.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.101+0000 7ff7c571e700 1 --2- 172.21.15.154:0/1270822762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c0102130 0x7ff7c0102510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009910 tx=0x7ff7b0009c20 comp rx=0 tx=0).stop 2024-06-06T04:20:48.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.101+0000 7ff7c571e700 1 -- 172.21.15.154:0/1270822762 shutdown_connections 2024-06-06T04:20:48.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.101+0000 7ff7c571e700 1 --2- 172.21.15.154:0/1270822762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c0102130 0x7ff7c0102510 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:48.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.101+0000 7ff7c571e700 1 -- 172.21.15.154:0/1270822762 >> 172.21.15.154:0/1270822762 conn(0x7ff7c00fd9e0 msgr2=0x7ff7c00ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:48.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.101+0000 7ff7c571e700 1 -- 172.21.15.154:0/1270822762 shutdown_connections 2024-06-06T04:20:48.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.101+0000 7ff7c571e700 1 -- 172.21.15.154:0/1270822762 wait complete. 2024-06-06T04:20:48.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.102+0000 7ff7c571e700 1 Processor -- start 2024-06-06T04:20:48.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.102+0000 7ff7c571e700 1 -- start start 2024-06-06T04:20:48.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.102+0000 7ff7c571e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c0102130 0x7ff7c0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:48.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.102+0000 7ff7c571e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c010c3b0 con 0x7ff7c0102130 2024-06-06T04:20:48.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.102+0000 7ff7beffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c0102130 0x7ff7c0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:48.106 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.102+0000 7ff7beffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c0102130 0x7ff7c0074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.154:58534/0 (socket says 172.21.15.154:58534) 2024-06-06T04:20:48.106 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.102+0000 7ff7beffd700 1 -- 172.21.15.154:0/3674178827 learned_addr learned my addr 172.21.15.154:0/3674178827 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-06-06T04:20:48.106 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.103+0000 7ff7beffd700 1 -- 172.21.15.154:0/3674178827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c0076680 con 0x7ff7c0102130 2024-06-06T04:20:48.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.103+0000 7ff7beffd700 1 --2- 172.21.15.154:0/3674178827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c0102130 0x7ff7c0074790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00056d0 tx=0x7ff7b00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:48.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.103+0000 7ff7a7fff700 1 -- 172.21.15.154:0/3674178827 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7b0026070 con 0x7ff7c0102130 2024-06-06T04:20:48.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.103+0000 7ff7a7fff700 1 -- 172.21.15.154:0/3674178827 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7b000bc00 con 0x7ff7c0102130 2024-06-06T04:20:48.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.103+0000 7ff7a7fff700 1 -- 172.21.15.154:0/3674178827 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7b00039b0 con 0x7ff7c0102130 2024-06-06T04:20:48.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.103+0000 7ff7c571e700 1 -- 172.21.15.154:0/3674178827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c0074d30 con 0x7ff7c0102130 2024-06-06T04:20:48.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.103+0000 7ff7c571e700 1 -- 172.21.15.154:0/3674178827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c00751f0 con 0x7ff7c0102130 2024-06-06T04:20:48.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.104+0000 7ff7c571e700 1 -- 172.21.15.154:0/3674178827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c0103610 con 0x7ff7c0102130 2024-06-06T04:20:48.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.104+0000 7ff7a7fff700 1 -- 172.21.15.154:0/3674178827 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff7b0029070 con 0x7ff7c0102130 2024-06-06T04:20:48.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.105+0000 7ff7a7fff700 1 --2- 172.21.15.154:0/3674178827 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff7a80390f0 0x7ff7a803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:48.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.105+0000 7ff7a7fff700 1 -- 172.21.15.154:0/3674178827 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff7b001e070 con 0x7ff7c0102130 2024-06-06T04:20:48.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.105+0000 7ff7be7fc700 1 --2- 172.21.15.154:0/3674178827 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff7a80390f0 0x7ff7a803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:48.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.105+0000 7ff7be7fc700 1 --2- 172.21.15.154:0/3674178827 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff7a80390f0 0x7ff7a803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4006fd0 tx=0x7ff7b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:48.112 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.110+0000 7ff7a7fff700 1 -- 172.21.15.154:0/3674178827 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff7b0013070 con 0x7ff7c0102130 2024-06-06T04:20:48.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.409+0000 7ff7c571e700 1 -- 172.21.15.154:0/3674178827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff7c0075d60 con 0x7ff7c0102130 2024-06-06T04:20:48.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.410+0000 7ff7a7fff700 1 -- 172.21.15.154:0/3674178827 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7ff7b0030020 con 0x7ff7c0102130 2024-06-06T04:20:48.410 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:20:48.410 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":1,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","modified":"2024-06-06T04:18:59.496605Z","created":"2024-06-06T04:18:59.496605Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-06T04:20:48.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.412+0000 7ff7c571e700 1 -- 172.21.15.154:0/3674178827 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff7a80390f0 msgr2=0x7ff7a803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:48.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.412+0000 7ff7c571e700 1 --2- 172.21.15.154:0/3674178827 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff7a80390f0 0x7ff7a803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4006fd0 tx=0x7ff7b4006e40 comp rx=0 tx=0).stop 2024-06-06T04:20:48.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.412+0000 7ff7c571e700 1 -- 172.21.15.154:0/3674178827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c0102130 msgr2=0x7ff7c0074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:48.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.412+0000 7ff7c571e700 1 --2- 172.21.15.154:0/3674178827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c0102130 0x7ff7c0074790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00056d0 tx=0x7ff7b00057b0 comp rx=0 tx=0).stop 2024-06-06T04:20:48.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.412+0000 7ff7c571e700 1 -- 172.21.15.154:0/3674178827 shutdown_connections 2024-06-06T04:20:48.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.412+0000 7ff7c571e700 1 --2- 172.21.15.154:0/3674178827 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff7a80390f0 0x7ff7a803b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:48.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.412+0000 7ff7c571e700 1 --2- 172.21.15.154:0/3674178827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c0102130 0x7ff7c0074790 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:48.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.412+0000 7ff7c571e700 1 -- 172.21.15.154:0/3674178827 >> 172.21.15.154:0/3674178827 conn(0x7ff7c00fd9e0 msgr2=0x7ff7c0103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:48.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.412+0000 7ff7c571e700 1 -- 172.21.15.154:0/3674178827 shutdown_connections 2024-06-06T04:20:48.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:48.412+0000 7ff7c571e700 1 -- 172.21.15.154:0/3674178827 wait complete. 2024-06-06T04:20:48.414 INFO:teuthology.orchestra.run.smithi154.stderr:dumped monmap epoch 1 2024-06-06T04:20:50.282 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-06T04:20:50.283 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-06T04:20:50.530 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T04:20:53.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:20:53 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:20:53.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:53.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T04:20:53.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:20:53.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: mon.a calling monitor election 2024-06-06T04:20:53.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:20:53.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:53.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:20:53.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: mon.c calling monitor election 2024-06-06T04:20:53.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:20:53.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:53.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:20:53.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:20:53.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-06T04:20:53.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: monmap epoch 2 2024-06-06T04:20:53.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:20:53.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: last_changed 2024-06-06T04:20:48.600821+0000 2024-06-06T04:20:53.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: created 2024-06-06T04:18:59.496605+0000 2024-06-06T04:20:53.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: min_mon_release 19 (squid) 2024-06-06T04:20:53.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: election_strategy: 1 2024-06-06T04:20:53.981 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-06-06T04:20:53.981 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-06-06T04:20:53.981 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: fsmap 2024-06-06T04:20:53.981 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T04:20:53.981 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: mgrmap e13: x(active, since 49s) 2024-06-06T04:20:53.981 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: overall HEALTH_OK 2024-06-06T04:20:53.982 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:53.982 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:53.982 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:20:53.982 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T04:20:53.982 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:20:54.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:20:54 smithi154 ceph-mon[36428]: Deploying daemon mon.b on smithi086 2024-06-06T04:20:54.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:20:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:20:54.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.957+0000 7f25ee3d7700 1 -- 172.21.15.154:0/590472656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25d00035c0 msgr2=0x7f25d0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:54.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.957+0000 7f25ee3d7700 1 --2- 172.21.15.154:0/590472656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25d00035c0 0x7f25d0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f25e80775d0 tx=0x7f25d800feb0 comp rx=0 tx=0).stop 2024-06-06T04:20:54.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.958+0000 7f25ee3d7700 1 -- 172.21.15.154:0/590472656 shutdown_connections 2024-06-06T04:20:54.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.958+0000 7f25ee3d7700 1 --2- 172.21.15.154:0/590472656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25d00035c0 0x7f25d0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:54.959 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.958+0000 7f25ee3d7700 1 -- 172.21.15.154:0/590472656 >> 172.21.15.154:0/590472656 conn(0x7f25e80761f0 msgr2=0x7f25e8076600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:54.959 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.958+0000 7f25ee3d7700 1 -- 172.21.15.154:0/590472656 shutdown_connections 2024-06-06T04:20:54.959 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.958+0000 7f25ee3d7700 1 -- 172.21.15.154:0/590472656 wait complete. 2024-06-06T04:20:54.959 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.958+0000 7f25ee3d7700 1 Processor -- start 2024-06-06T04:20:54.959 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.959+0000 7f25ee3d7700 1 -- start start 2024-06-06T04:20:54.959 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.959+0000 7f25ee3d7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25d00035c0 0x7f25e8102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:54.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.959+0000 7f25ee3d7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e81030e0 0x7f25e8103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:54.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.959+0000 7f25ee3d7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f25e8108eb0 con 0x7f25d00035c0 2024-06-06T04:20:54.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.959+0000 7f25ee3d7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f25e8108d30 con 0x7f25e81030e0 2024-06-06T04:20:54.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.959+0000 7f25e77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e81030e0 0x7f25e8103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:54.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.959+0000 7f25e7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25d00035c0 0x7f25e8102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:54.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.959+0000 7f25e77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e81030e0 0x7f25e8103560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:42354/0 (socket says 172.21.15.154:42354) 2024-06-06T04:20:54.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.959+0000 7f25e7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25d00035c0 0x7f25e8102ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.154:55340/0 (socket says 172.21.15.154:55340) 2024-06-06T04:20:54.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.959+0000 7f25e77fe700 1 -- 172.21.15.154:0/1945110186 learned_addr learned my addr 172.21.15.154:0/1945110186 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-06-06T04:20:54.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.960+0000 7f25e77fe700 1 -- 172.21.15.154:0/1945110186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e81030e0 msgr2=0x7f25e8103560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-06T04:20:54.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.960+0000 7f25e77fe700 1 -- 172.21.15.154:0/1945110186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e81030e0 msgr2=0x7f25e8103560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-06T04:20:54.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.960+0000 7f25e77fe700 1 --2- 172.21.15.154:0/1945110186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e81030e0 0x7f25e8103560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-06T04:20:54.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.960+0000 7f25e77fe700 1 --2- 172.21.15.154:0/1945110186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e81030e0 0x7f25e8103560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T04:20:54.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.960+0000 7f25e7fff700 1 -- 172.21.15.154:0/1945110186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e81030e0 msgr2=0x7f25e8103560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T04:20:54.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.960+0000 7f25e7fff700 1 --2- 172.21.15.154:0/1945110186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e81030e0 0x7f25e8103560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:54.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.960+0000 7f25e7fff700 1 -- 172.21.15.154:0/1945110186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e8104ac0 con 0x7f25d00035c0 2024-06-06T04:20:54.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.960+0000 7f25e7fff700 1 --2- 172.21.15.154:0/1945110186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25d00035c0 0x7f25e8102ba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000c00 tx=0x7f25d80050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:54.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.960+0000 7f25e57fa700 1 -- 172.21.15.154:0/1945110186 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f25d8004aa0 con 0x7f25d00035c0 2024-06-06T04:20:54.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.960+0000 7f25e57fa700 1 -- 172.21.15.154:0/1945110186 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25d8004030 con 0x7f25d00035c0 2024-06-06T04:20:54.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.960+0000 7f25e57fa700 1 -- 172.21.15.154:0/1945110186 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f25d8011bb0 con 0x7f25d00035c0 2024-06-06T04:20:54.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.960+0000 7f25ee3d7700 1 -- 172.21.15.154:0/1945110186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e81a3d30 con 0x7f25d00035c0 2024-06-06T04:20:54.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.960+0000 7f25ee3d7700 1 -- 172.21.15.154:0/1945110186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e81a4210 con 0x7f25d00035c0 2024-06-06T04:20:54.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.961+0000 7f25e57fa700 1 -- 172.21.15.154:0/1945110186 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f25d8023070 con 0x7f25d00035c0 2024-06-06T04:20:54.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.962+0000 7f25ee3d7700 1 -- 172.21.15.154:0/1945110186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e806a490 con 0x7f25d00035c0 2024-06-06T04:20:54.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.962+0000 7f25e57fa700 1 --2- 172.21.15.154:0/1945110186 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f25d4038ee0 0x7f25d403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:20:54.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.962+0000 7f25e57fa700 1 -- 172.21.15.154:0/1945110186 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f25d8020050 con 0x7f25d00035c0 2024-06-06T04:20:54.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.962+0000 7f25e77fe700 1 --2- 172.21.15.154:0/1945110186 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f25d4038ee0 0x7f25d403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:20:54.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.962+0000 7f25e77fe700 1 --2- 172.21.15.154:0/1945110186 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f25d4038ee0 0x7f25d403b3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009180 tx=0x7f25dc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:20:54.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:54.967+0000 7f25e57fa700 1 -- 172.21.15.154:0/1945110186 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f25d8028020 con 0x7f25d00035c0 2024-06-06T04:20:54.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:54 smithi045 ceph-mon[32299]: Deploying daemon mon.b on smithi086 2024-06-06T04:20:54.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:20:55.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:55.247+0000 7f25ee3d7700 1 -- 172.21.15.154:0/1945110186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f25e81a44c0 con 0x7f25d00035c0 2024-06-06T04:20:55.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:55.247+0000 7f25e57fa700 1 -- 172.21.15.154:0/1945110186 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f25d801a390 con 0x7f25d00035c0 2024-06-06T04:20:55.248 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:20:55.248 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":2,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","modified":"2024-06-06T04:20:48.600821Z","created":"2024-06-06T04:18:59.496605Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-06T04:20:55.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:55.250+0000 7f25ee3d7700 1 -- 172.21.15.154:0/1945110186 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f25d4038ee0 msgr2=0x7f25d403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:55.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:55.250+0000 7f25ee3d7700 1 --2- 172.21.15.154:0/1945110186 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f25d4038ee0 0x7f25d403b3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009180 tx=0x7f25dc008040 comp rx=0 tx=0).stop 2024-06-06T04:20:55.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:55.250+0000 7f25ee3d7700 1 -- 172.21.15.154:0/1945110186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25d00035c0 msgr2=0x7f25e8102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:20:55.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:55.250+0000 7f25ee3d7700 1 --2- 172.21.15.154:0/1945110186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25d00035c0 0x7f25e8102ba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000c00 tx=0x7f25d80050d0 comp rx=0 tx=0).stop 2024-06-06T04:20:55.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:55.250+0000 7f25ee3d7700 1 -- 172.21.15.154:0/1945110186 shutdown_connections 2024-06-06T04:20:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:55.250+0000 7f25ee3d7700 1 --2- 172.21.15.154:0/1945110186 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f25d4038ee0 0x7f25d403b3a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:55.250+0000 7f25ee3d7700 1 --2- 172.21.15.154:0/1945110186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25d00035c0 0x7f25e8102ba0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:55.250+0000 7f25ee3d7700 1 --2- 172.21.15.154:0/1945110186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e81030e0 0x7f25e8103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:20:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:55.250+0000 7f25ee3d7700 1 -- 172.21.15.154:0/1945110186 >> 172.21.15.154:0/1945110186 conn(0x7f25e80761f0 msgr2=0x7f25e81001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:20:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:55.250+0000 7f25ee3d7700 1 -- 172.21.15.154:0/1945110186 shutdown_connections 2024-06-06T04:20:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:20:55.250+0000 7f25ee3d7700 1 -- 172.21.15.154:0/1945110186 wait complete. 2024-06-06T04:20:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:dumped monmap epoch 2 2024-06-06T04:20:55.841 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:20:55 smithi154 ceph-mon[36428]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:55.841 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:20:55 smithi154 ceph-mon[36428]: from='client.? 172.21.15.154:0/1945110186' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T04:20:55.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:55 smithi045 ceph-mon[32299]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:20:55.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:20:55 smithi045 ceph-mon[32299]: from='client.? 172.21.15.154:0/1945110186' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T04:20:56.843 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-06T04:20:56.843 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-06T04:20:57.082 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T04:20:57.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:20:57 smithi086 ceph-mon[36053]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-06T04:20:57.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:20:57 smithi086 bash[36035]: b0595627ff8975d6e7b8bd62f43545bc1db08327eef53419625b8ec9e3be3cf5 2024-06-06T04:20:57.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:20:57 smithi086 systemd[1]: Started Ceph mon.b for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:21:02.814 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.813+0000 7ff3a7821700 1 -- 172.21.15.154:0/3891867695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff38c0057d0 msgr2=0x7ff38c007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:02.814 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.813+0000 7ff3a7821700 1 --2- 172.21.15.154:0/3891867695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff38c0057d0 0x7ff38c007c60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0100db0 tx=0x7ff39000fe60 comp rx=0 tx=0).stop 2024-06-06T04:21:02.814 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.813+0000 7ff3a7821700 1 -- 172.21.15.154:0/3891867695 shutdown_connections 2024-06-06T04:21:02.814 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.813+0000 7ff3a7821700 1 --2- 172.21.15.154:0/3891867695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff38c0057d0 0x7ff38c007c60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:02.815 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.813+0000 7ff3a7821700 1 -- 172.21.15.154:0/3891867695 >> 172.21.15.154:0/3891867695 conn(0x7ff3a00fc970 msgr2=0x7ff3a00fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:21:02.815 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.813+0000 7ff3a7821700 1 -- 172.21.15.154:0/3891867695 shutdown_connections 2024-06-06T04:21:02.815 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.813+0000 7ff3a7821700 1 -- 172.21.15.154:0/3891867695 wait complete. 2024-06-06T04:21:02.815 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.814+0000 7ff3a7821700 1 Processor -- start 2024-06-06T04:21:02.815 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.814+0000 7ff3a7821700 1 -- start start 2024-06-06T04:21:02.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T04:21:02.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: mon.a calling monitor election 2024-06-06T04:21:02.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: mon.c calling monitor election 2024-06-06T04:21:02.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:21:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: mon.b calling monitor election 2024-06-06T04:21:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-06T04:21:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: monmap epoch 3 2024-06-06T04:21:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:21:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: last_changed 2024-06-06T04:20:57.463151+0000 2024-06-06T04:21:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: created 2024-06-06T04:18:59.496605+0000 2024-06-06T04:21:02.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: min_mon_release 19 (squid) 2024-06-06T04:21:02.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: election_strategy: 1 2024-06-06T04:21:02.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-06-06T04:21:02.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-06-06T04:21:02.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: 2: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-06-06T04:21:02.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: fsmap 2024-06-06T04:21:02.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T04:21:02.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: mgrmap e13: x(active, since 58s) 2024-06-06T04:21:02.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: overall HEALTH_OK 2024-06-06T04:21:02.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:02.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:02.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:02.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:02.819 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.814+0000 7ff3a7821700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff38c0057d0 0x7ff3a019b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:02.819 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.814+0000 7ff3a7821700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a019bba0 0x7ff3a01957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:02.819 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.814+0000 7ff3a7821700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a0195d20 0x7ff3a01961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:02.819 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.814+0000 7ff3a7821700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a0107d60 con 0x7ff3a0195d20 2024-06-06T04:21:02.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.814+0000 7ff3a7821700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a006ac50 con 0x7ff38c0057d0 2024-06-06T04:21:02.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.814+0000 7ff3a7821700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a0107ee0 con 0x7ff3a019bba0 2024-06-06T04:21:02.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.814+0000 7ff3a4dbc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a019bba0 0x7ff3a01957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:02.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.814+0000 7ff3a4dbc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a019bba0 0x7ff3a01957e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40712/0 (socket says 172.21.15.154:40712) 2024-06-06T04:21:02.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.814+0000 7ff3a4dbc700 1 -- 172.21.15.154:0/231386727 learned_addr learned my addr 172.21.15.154:0/231386727 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-06-06T04:21:02.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.814+0000 7ff3a55bd700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff38c0057d0 0x7ff3a019b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:02.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.814+0000 7ff3a5dbe700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a0195d20 0x7ff3a01961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:02.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.814+0000 7ff3a4dbc700 1 -- 172.21.15.154:0/231386727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a019bba0 msgr2=0x7ff3a01957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-06T04:21:02.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3a4dbc700 1 -- 172.21.15.154:0/231386727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a019bba0 msgr2=0x7ff3a01957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-06T04:21:02.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3a4dbc700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a019bba0 0x7ff3a01957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-06T04:21:02.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3a4dbc700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a019bba0 0x7ff3a01957e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T04:21:02.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3a55bd700 1 -- 172.21.15.154:0/231386727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff38c0057d0 msgr2=0x7ff3a019b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-06T04:21:02.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3a55bd700 1 -- 172.21.15.154:0/231386727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff38c0057d0 msgr2=0x7ff3a019b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-06T04:21:02.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3a55bd700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff38c0057d0 0x7ff3a019b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-06T04:21:02.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3a55bd700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff38c0057d0 0x7ff3a019b660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T04:21:02.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3a5dbe700 1 -- 172.21.15.154:0/231386727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a019bba0 msgr2=0x7ff3a01957e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T04:21:02.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3a5dbe700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a019bba0 0x7ff3a01957e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:02.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3a5dbe700 1 -- 172.21.15.154:0/231386727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff38c0057d0 msgr2=0x7ff3a019b660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T04:21:02.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3a5dbe700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff38c0057d0 0x7ff3a019b660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:02.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3a5dbe700 1 -- 172.21.15.154:0/231386727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a01a24c0 con 0x7ff3a0195d20 2024-06-06T04:21:02.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3a5dbe700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a0195d20 0x7ff3a01961d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff39800bd50 tx=0x7ff39800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:21:02.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3967fc700 1 -- 172.21.15.154:0/231386727 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff398019070 con 0x7ff3a0195d20 2024-06-06T04:21:02.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3a7821700 1 -- 172.21.15.154:0/231386727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a01a2750 con 0x7ff3a0195d20 2024-06-06T04:21:02.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3967fc700 1 -- 172.21.15.154:0/231386727 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff39800cd10 con 0x7ff3a0195d20 2024-06-06T04:21:02.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.815+0000 7ff3967fc700 1 -- 172.21.15.154:0/231386727 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff398012700 con 0x7ff3a0195d20 2024-06-06T04:21:02.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.816+0000 7ff3a7821700 1 -- 172.21.15.154:0/231386727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a01a2c60 con 0x7ff3a0195d20 2024-06-06T04:21:02.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.817+0000 7ff3967fc700 1 -- 172.21.15.154:0/231386727 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff39801c070 con 0x7ff3a0195d20 2024-06-06T04:21:02.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.817+0000 7ff3967fc700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff384039580 0x7ff38403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:02.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.817+0000 7ff3967fc700 1 -- 172.21.15.154:0/231386727 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff39804ba90 con 0x7ff3a0195d20 2024-06-06T04:21:02.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.817+0000 7ff3a7821700 1 -- 172.21.15.154:0/231386727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a01025c0 con 0x7ff3a0195d20 2024-06-06T04:21:02.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.817+0000 7ff3a55bd700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff384039580 0x7ff38403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:02.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.818+0000 7ff3a55bd700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff384039580 0x7ff38403ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff39000fa10 tx=0x7ff390005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:21:02.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:02.823+0000 7ff3967fc700 1 -- 172.21.15.154:0/231386727 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff39800e0a0 con 0x7ff3a0195d20 2024-06-06T04:21:02.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T04:21:02.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: mon.a calling monitor election 2024-06-06T04:21:02.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: mon.c calling monitor election 2024-06-06T04:21:02.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:21:02.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:02.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: mon.b calling monitor election 2024-06-06T04:21:02.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:02.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-06T04:21:02.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: monmap epoch 3 2024-06-06T04:21:02.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:21:02.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: last_changed 2024-06-06T04:20:57.463151+0000 2024-06-06T04:21:02.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: created 2024-06-06T04:18:59.496605+0000 2024-06-06T04:21:02.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: min_mon_release 19 (squid) 2024-06-06T04:21:02.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: election_strategy: 1 2024-06-06T04:21:02.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-06-06T04:21:02.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-06-06T04:21:02.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: 2: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-06-06T04:21:02.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: fsmap 2024-06-06T04:21:02.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T04:21:02.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: mgrmap e13: x(active, since 58s) 2024-06-06T04:21:02.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: overall HEALTH_OK 2024-06-06T04:21:02.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:02.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:02.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:02.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:02.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T04:21:02.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: mon.a calling monitor election 2024-06-06T04:21:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: mon.c calling monitor election 2024-06-06T04:21:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:21:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: mon.b calling monitor election 2024-06-06T04:21:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-06T04:21:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: monmap epoch 3 2024-06-06T04:21:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:21:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: last_changed 2024-06-06T04:20:57.463151+0000 2024-06-06T04:21:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: created 2024-06-06T04:18:59.496605+0000 2024-06-06T04:21:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: min_mon_release 19 (squid) 2024-06-06T04:21:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: election_strategy: 1 2024-06-06T04:21:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-06-06T04:21:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-06-06T04:21:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: 2: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-06-06T04:21:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: fsmap 2024-06-06T04:21:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T04:21:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: mgrmap e13: x(active, since 58s) 2024-06-06T04:21:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: overall HEALTH_OK 2024-06-06T04:21:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:02.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:02.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:02.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:03.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:03.110+0000 7ff3a7821700 1 -- 172.21.15.154:0/231386727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff3a0196d10 con 0x7ff3a0195d20 2024-06-06T04:21:03.111 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:03.111+0000 7ff3967fc700 1 -- 172.21.15.154:0/231386727 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7ff398007290 con 0x7ff3a0195d20 2024-06-06T04:21:03.112 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-06-06T04:21:03.112 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":3,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","modified":"2024-06-06T04:20:57.463151Z","created":"2024-06-06T04:18:59.496605Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-06-06T04:21:03.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:03.113+0000 7ff3a7821700 1 -- 172.21.15.154:0/231386727 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff384039580 msgr2=0x7ff38403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:03.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:03.113+0000 7ff3a7821700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff384039580 0x7ff38403ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff39000fa10 tx=0x7ff390005a90 comp rx=0 tx=0).stop 2024-06-06T04:21:03.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:03.113+0000 7ff3a7821700 1 -- 172.21.15.154:0/231386727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a0195d20 msgr2=0x7ff3a01961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:03.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:03.113+0000 7ff3a7821700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a0195d20 0x7ff3a01961d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff39800bd50 tx=0x7ff39800be30 comp rx=0 tx=0).stop 2024-06-06T04:21:03.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:03.113+0000 7ff3a7821700 1 -- 172.21.15.154:0/231386727 shutdown_connections 2024-06-06T04:21:03.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:03.113+0000 7ff3a7821700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a0195d20 0x7ff3a01961d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:03.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:03.113+0000 7ff3a7821700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff384039580 0x7ff38403ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:03.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:03.113+0000 7ff3a7821700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff38c0057d0 0x7ff3a019b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:03.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:03.113+0000 7ff3a7821700 1 --2- 172.21.15.154:0/231386727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a019bba0 0x7ff3a01957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:03.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:03.113+0000 7ff3a7821700 1 -- 172.21.15.154:0/231386727 >> 172.21.15.154:0/231386727 conn(0x7ff3a00fc970 msgr2=0x7ff3a00fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:21:03.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:03.113+0000 7ff3a7821700 1 -- 172.21.15.154:0/231386727 shutdown_connections 2024-06-06T04:21:03.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:03.113+0000 7ff3a7821700 1 -- 172.21.15.154:0/231386727 wait complete. 2024-06-06T04:21:03.115 INFO:teuthology.orchestra.run.smithi154.stderr:dumped monmap epoch 3 2024-06-06T04:21:03.678 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:03 smithi154 ceph-mon[36428]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:03.679 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:03.679 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:03 smithi154 ceph-mon[36428]: from='client.? 172.21.15.154:0/231386727' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T04:21:03.679 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:03.705 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-06T04:21:03.706 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-06T04:21:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:03 smithi086 ceph-mon[36053]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:03 smithi086 ceph-mon[36053]: from='client.? 172.21.15.154:0/231386727' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T04:21:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:03.961 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:21:03.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:03 smithi045 ceph-mon[32299]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:03.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:03.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:03 smithi045 ceph-mon[32299]: from='client.? 172.21.15.154:0/231386727' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T04:21:03.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:21:04.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:05.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:05.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:05.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:05.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:05.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:05.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:05.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:05.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.556+0000 7f49e042b700 1 -- 172.21.15.45:0/997781228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d8100fe0 msgr2=0x7f49d81013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:05.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.556+0000 7f49e042b700 1 --2- 172.21.15.45:0/997781228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d8100fe0 0x7f49d81013c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009910 tx=0x7f49cc009c20 comp rx=0 tx=0).stop 2024-06-06T04:21:05.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.556+0000 7f49e042b700 1 -- 172.21.15.45:0/997781228 shutdown_connections 2024-06-06T04:21:05.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.556+0000 7f49e042b700 1 --2- 172.21.15.45:0/997781228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d8100fe0 0x7f49d81013c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:05.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.556+0000 7f49e042b700 1 -- 172.21.15.45:0/997781228 >> 172.21.15.45:0/997781228 conn(0x7f49d80fc890 msgr2=0x7f49d80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:21:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.556+0000 7f49e042b700 1 -- 172.21.15.45:0/997781228 shutdown_connections 2024-06-06T04:21:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.556+0000 7f49e042b700 1 -- 172.21.15.45:0/997781228 wait complete. 2024-06-06T04:21:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.557+0000 7f49e042b700 1 Processor -- start 2024-06-06T04:21:05.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.557+0000 7f49e042b700 1 -- start start 2024-06-06T04:21:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.557+0000 7f49e042b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d8100fe0 0x7f49d819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.557+0000 7f49e042b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49d819dcd0 0x7f49d8197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.557+0000 7f49e042b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d8197e90 0x7f49d8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.557+0000 7f49e042b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f49d810b260 con 0x7f49d8197e90 2024-06-06T04:21:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.557+0000 7f49e042b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49d810b0e0 con 0x7f49d8100fe0 2024-06-06T04:21:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.557+0000 7f49e042b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f49d810b3e0 con 0x7f49d819dcd0 2024-06-06T04:21:05.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.557+0000 7f49de9c8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d8197e90 0x7f49d8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:05.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.557+0000 7f49de9c8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d8197e90 0x7f49d8198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50270/0 (socket says 172.21.15.45:50270) 2024-06-06T04:21:05.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.557+0000 7f49de9c8700 1 -- 172.21.15.45:0/2139494895 learned_addr learned my addr 172.21.15.45:0/2139494895 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:21:05.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.557+0000 7f49dd9c6700 1 --2- 172.21.15.45:0/2139494895 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49d819dcd0 0x7f49d8197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:05.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.557+0000 7f49de1c7700 1 --2- 172.21.15.45:0/2139494895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d8100fe0 0x7f49d819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:05.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.558+0000 7f49de9c8700 1 -- 172.21.15.45:0/2139494895 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49d819dcd0 msgr2=0x7f49d8197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:05.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.558+0000 7f49de9c8700 1 --2- 172.21.15.45:0/2139494895 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49d819dcd0 0x7f49d8197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:05.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.558+0000 7f49de9c8700 1 -- 172.21.15.45:0/2139494895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d8100fe0 msgr2=0x7f49d819d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:05.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.558+0000 7f49de9c8700 1 --2- 172.21.15.45:0/2139494895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d8100fe0 0x7f49d819d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:05.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.558+0000 7f49de9c8700 1 -- 172.21.15.45:0/2139494895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d81a4660 con 0x7f49d8197e90 2024-06-06T04:21:05.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.558+0000 7f49de9c8700 1 --2- 172.21.15.45:0/2139494895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d8197e90 0x7f49d8198340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f49d000ba50 tx=0x7f49d000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:21:05.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.558+0000 7f49cb7fe700 1 -- 172.21.15.45:0/2139494895 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0019070 con 0x7f49d8197e90 2024-06-06T04:21:05.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.558+0000 7f49cb7fe700 1 -- 172.21.15.45:0/2139494895 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49d000cde0 con 0x7f49d8197e90 2024-06-06T04:21:05.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.558+0000 7f49e042b700 1 -- 172.21.15.45:0/2139494895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d81a4890 con 0x7f49d8197e90 2024-06-06T04:21:05.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.558+0000 7f49cb7fe700 1 -- 172.21.15.45:0/2139494895 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0007350 con 0x7f49d8197e90 2024-06-06T04:21:05.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.558+0000 7f49e042b700 1 -- 172.21.15.45:0/2139494895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d81a4dd0 con 0x7f49d8197e90 2024-06-06T04:21:05.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.559+0000 7f49cb7fe700 1 -- 172.21.15.45:0/2139494895 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f49d001c070 con 0x7f49d8197e90 2024-06-06T04:21:05.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.559+0000 7f49e042b700 1 -- 172.21.15.45:0/2139494895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d8063af0 con 0x7f49d8197e90 2024-06-06T04:21:05.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.560+0000 7f49cb7fe700 1 --2- 172.21.15.45:0/2139494895 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f49c4039580 0x7f49c403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:05.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.560+0000 7f49cb7fe700 1 -- 172.21.15.45:0/2139494895 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f49d004b720 con 0x7f49d8197e90 2024-06-06T04:21:05.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.560+0000 7f49de1c7700 1 --2- 172.21.15.45:0/2139494895 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f49c4039580 0x7f49c403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:05.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.560+0000 7f49de1c7700 1 --2- 172.21.15.45:0/2139494895 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f49c4039580 0x7f49c403ba40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49cc006010 tx=0x7f49cc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:21:05.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.566+0000 7f49cb7fe700 1 -- 172.21.15.45:0/2139494895 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f49d0014440 con 0x7f49d8197e90 2024-06-06T04:21:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.774+0000 7f49e042b700 1 -- 172.21.15.45:0/2139494895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f49d81a50e0 con 0x7f49d8197e90 2024-06-06T04:21:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.775+0000 7f49cb7fe700 1 -- 172.21.15.45:0/2139494895 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f49d0012c60 con 0x7f49d8197e90 2024-06-06T04:21:05.775 INFO:teuthology.orchestra.run.smithi045.stdout:# minimal ceph.conf for c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:21:05.775 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-06-06T04:21:05.776 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:21:05.776 INFO:teuthology.orchestra.run.smithi045.stdout: mon_host = [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-06-06T04:21:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.778+0000 7f49e042b700 1 -- 172.21.15.45:0/2139494895 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f49c4039580 msgr2=0x7f49c403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.778+0000 7f49e042b700 1 --2- 172.21.15.45:0/2139494895 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f49c4039580 0x7f49c403ba40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49cc006010 tx=0x7f49cc01a040 comp rx=0 tx=0).stop 2024-06-06T04:21:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.778+0000 7f49e042b700 1 -- 172.21.15.45:0/2139494895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d8197e90 msgr2=0x7f49d8198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.778+0000 7f49e042b700 1 --2- 172.21.15.45:0/2139494895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d8197e90 0x7f49d8198340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f49d000ba50 tx=0x7f49d000bd60 comp rx=0 tx=0).stop 2024-06-06T04:21:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.778+0000 7f49e042b700 1 -- 172.21.15.45:0/2139494895 shutdown_connections 2024-06-06T04:21:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.778+0000 7f49e042b700 1 --2- 172.21.15.45:0/2139494895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49d8197e90 0x7f49d8198340 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.778+0000 7f49e042b700 1 --2- 172.21.15.45:0/2139494895 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f49c4039580 0x7f49c403ba40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.778+0000 7f49e042b700 1 --2- 172.21.15.45:0/2139494895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d8100fe0 0x7f49d819d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.778+0000 7f49e042b700 1 --2- 172.21.15.45:0/2139494895 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49d819dcd0 0x7f49d8197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.778+0000 7f49e042b700 1 -- 172.21.15.45:0/2139494895 >> 172.21.15.45:0/2139494895 conn(0x7f49d80fc890 msgr2=0x7f49d8108b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:21:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.778+0000 7f49e042b700 1 -- 172.21.15.45:0/2139494895 shutdown_connections 2024-06-06T04:21:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:05.778+0000 7f49e042b700 1 -- 172.21.15.45:0/2139494895 wait complete. 2024-06-06T04:21:06.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:06.261 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: Updating smithi045:/etc/ceph/ceph.conf 2024-06-06T04:21:06.262 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: Updating smithi086:/etc/ceph/ceph.conf 2024-06-06T04:21:06.262 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: Updating smithi154:/etc/ceph/ceph.conf 2024-06-06T04:21:06.262 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: Updating smithi045:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T04:21:06.262 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: Updating smithi154:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T04:21:06.262 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: Updating smithi086:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T04:21:06.262 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.262 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.262 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.262 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.262 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.263 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.263 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.263 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.263 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.263 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.263 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.263 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T04:21:06.263 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T04:21:06.263 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:06.263 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:05 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2139494895' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:06.288 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:06.288 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: Updating smithi045:/etc/ceph/ceph.conf 2024-06-06T04:21:06.288 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: Updating smithi086:/etc/ceph/ceph.conf 2024-06-06T04:21:06.289 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: Updating smithi154:/etc/ceph/ceph.conf 2024-06-06T04:21:06.289 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: Updating smithi045:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T04:21:06.289 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: Updating smithi154:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T04:21:06.289 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: Updating smithi086:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T04:21:06.289 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.289 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.290 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.290 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.290 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.290 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.290 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.290 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.291 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.291 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.291 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.291 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T04:21:06.291 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T04:21:06.291 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:06.291 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:05 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2139494895' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:06.403 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-06T04:21:06.403 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:21:06.403 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.conf 2024-06-06T04:21:06.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:06.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: Updating smithi045:/etc/ceph/ceph.conf 2024-06-06T04:21:06.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: Updating smithi086:/etc/ceph/ceph.conf 2024-06-06T04:21:06.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: Updating smithi154:/etc/ceph/ceph.conf 2024-06-06T04:21:06.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: Updating smithi045:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T04:21:06.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: Updating smithi154:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T04:21:06.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: Updating smithi086:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T04:21:06.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.432 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.432 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.432 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:06.432 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T04:21:06.432 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T04:21:06.432 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:06.432 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:05 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2139494895' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:06.433 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:21:06.433 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T04:21:06.499 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-06-06T04:21:06.499 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.conf 2024-06-06T04:21:06.528 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-06-06T04:21:06.528 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T04:21:06.594 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-06-06T04:21:06.594 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.conf 2024-06-06T04:21:06.627 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-06-06T04:21:06.627 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T04:21:06.696 INFO:tasks.cephadm:Adding mgr.x on smithi045 2024-06-06T04:21:06.696 INFO:tasks.cephadm:Adding mgr.y on smithi086 2024-06-06T04:21:06.696 INFO:tasks.cephadm:Adding mgr.z on smithi154 2024-06-06T04:21:06.696 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi045=x;smithi086=y;smithi154=z' 2024-06-06T04:21:06.971 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T04:21:07.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:06 smithi045 ceph-mon[32299]: Reconfiguring mon.a (unknown last config time)... 2024-06-06T04:21:07.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:06 smithi045 ceph-mon[32299]: Reconfiguring daemon mon.a on smithi045 2024-06-06T04:21:07.237 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:06 smithi154 ceph-mon[36428]: Reconfiguring mon.a (unknown last config time)... 2024-06-06T04:21:07.237 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:06 smithi154 ceph-mon[36428]: Reconfiguring daemon mon.a on smithi045 2024-06-06T04:21:07.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:06 smithi086 ceph-mon[36053]: Reconfiguring mon.a (unknown last config time)... 2024-06-06T04:21:07.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:06 smithi086 ceph-mon[36053]: Reconfiguring daemon mon.a on smithi045 2024-06-06T04:21:08.118 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:07 smithi154 ceph-mon[36428]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:08.119 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:07 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:08.119 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:07 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:08.119 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:07 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T04:21:08.119 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:07 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T04:21:08.119 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:07 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:08.224 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:07 smithi086 ceph-mon[36053]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:08.224 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:07 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:08.224 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:07 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:08.224 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:07 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T04:21:08.224 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:07 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T04:21:08.225 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:07 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:08.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:07 smithi045 ceph-mon[32299]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:08.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:08.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:08.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T04:21:08.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T04:21:08.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:08.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.338+0000 7fc2aeda1700 1 -- 172.21.15.154:0/917670188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2a8100f10 msgr2=0x7fc2a8101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:08.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.338+0000 7fc2aeda1700 1 --2- 172.21.15.154:0/917670188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2a8100f10 0x7fc2a8101330 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009910 tx=0x7fc29c009c20 comp rx=0 tx=0).stop 2024-06-06T04:21:08.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.338+0000 7fc2aeda1700 1 -- 172.21.15.154:0/917670188 shutdown_connections 2024-06-06T04:21:08.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.338+0000 7fc2aeda1700 1 --2- 172.21.15.154:0/917670188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2a8100f10 0x7fc2a8101330 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:08.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.338+0000 7fc2aeda1700 1 -- 172.21.15.154:0/917670188 >> 172.21.15.154:0/917670188 conn(0x7fc2a80fc4d0 msgr2=0x7fc2a80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:21:08.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.338+0000 7fc2aeda1700 1 -- 172.21.15.154:0/917670188 shutdown_connections 2024-06-06T04:21:08.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.338+0000 7fc2aeda1700 1 -- 172.21.15.154:0/917670188 wait complete. 2024-06-06T04:21:08.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.338+0000 7fc2aeda1700 1 Processor -- start 2024-06-06T04:21:08.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.338+0000 7fc2aeda1700 1 -- start start 2024-06-06T04:21:08.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2aeda1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a8100f10 0x7fc2a806b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:08.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2aeda1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2a819a430 0x7fc2a819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:08.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2aeda1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a819ce00 0x7fc2a819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:08.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2aeda1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a8106dd0 con 0x7fc2a819a430 2024-06-06T04:21:08.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2aeda1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a8106c50 con 0x7fc2a819ce00 2024-06-06T04:21:08.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2aeda1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a8106f50 con 0x7fc2a8100f10 2024-06-06T04:21:08.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2acb3d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a8100f10 0x7fc2a806b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:08.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2acb3d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a8100f10 0x7fc2a806b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40728/0 (socket says 172.21.15.154:40728) 2024-06-06T04:21:08.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2acb3d700 1 -- 172.21.15.154:0/584263345 learned_addr learned my addr 172.21.15.154:0/584263345 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-06-06T04:21:08.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2a7fff700 1 --2- 172.21.15.154:0/584263345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2a819a430 0x7fc2a819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:08.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2acb3d700 1 -- 172.21.15.154:0/584263345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a819ce00 msgr2=0x7fc2a819f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:08.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2ad33e700 1 --2- 172.21.15.154:0/584263345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a819ce00 0x7fc2a819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:08.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2acb3d700 1 --2- 172.21.15.154:0/584263345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a819ce00 0x7fc2a819f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:08.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2acb3d700 1 -- 172.21.15.154:0/584263345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2a819a430 msgr2=0x7fc2a819c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:08.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2acb3d700 1 --2- 172.21.15.154:0/584263345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2a819a430 0x7fc2a819c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:08.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2acb3d700 1 -- 172.21.15.154:0/584263345 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a819fb60 con 0x7fc2a8100f10 2024-06-06T04:21:08.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2ad33e700 1 --2- 172.21.15.154:0/584263345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a819ce00 0x7fc2a819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:21:08.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.339+0000 7fc2acb3d700 1 --2- 172.21.15.154:0/584263345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a8100f10 0x7fc2a806b260 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc29c005e00 tx=0x7fc29c004fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:21:08.347 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.340+0000 7fc2a5ffb700 1 -- 172.21.15.154:0/584263345 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c027070 con 0x7fc2a8100f10 2024-06-06T04:21:08.347 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.340+0000 7fc2a5ffb700 1 -- 172.21.15.154:0/584263345 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc29c00be10 con 0x7fc2a8100f10 2024-06-06T04:21:08.347 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.340+0000 7fc2aeda1700 1 -- 172.21.15.154:0/584263345 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a819fdf0 con 0x7fc2a8100f10 2024-06-06T04:21:08.347 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.340+0000 7fc2a5ffb700 1 -- 172.21.15.154:0/584263345 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c00f460 con 0x7fc2a8100f10 2024-06-06T04:21:08.347 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.340+0000 7fc2aeda1700 1 -- 172.21.15.154:0/584263345 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a81a0290 con 0x7fc2a8100f10 2024-06-06T04:21:08.347 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.341+0000 7fc2a5ffb700 1 -- 172.21.15.154:0/584263345 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc29c02a070 con 0x7fc2a8100f10 2024-06-06T04:21:08.347 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.341+0000 7fc2a5ffb700 1 --2- 172.21.15.154:0/584263345 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc298039580 0x7fc29803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:08.347 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.341+0000 7fc2a5ffb700 1 -- 172.21.15.154:0/584263345 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc29c04e2e0 con 0x7fc2a8100f10 2024-06-06T04:21:08.347 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.341+0000 7fc2aeda1700 1 -- 172.21.15.154:0/584263345 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a8063af0 con 0x7fc2a8100f10 2024-06-06T04:21:08.347 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.341+0000 7fc2a7fff700 1 --2- 172.21.15.154:0/584263345 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc298039580 0x7fc29803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:08.348 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.341+0000 7fc2a7fff700 1 --2- 172.21.15.154:0/584263345 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc298039580 0x7fc29803ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc294009ad0 tx=0x7fc294009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:21:08.348 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.346+0000 7fc2a5ffb700 1 -- 172.21.15.154:0/584263345 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc29c031070 con 0x7fc2a8100f10 2024-06-06T04:21:08.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.554+0000 7fc2aeda1700 1 -- 172.21.15.154:0/584263345 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi086=y;smithi154=z", "target": ["mon-mgr", ""]}) -- 0x7fc2a8062960 con 0x7fc298039580 2024-06-06T04:21:08.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.561+0000 7fc2a5ffb700 1 -- 172.21.15.154:0/584263345 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc2a8062960 con 0x7fc298039580 2024-06-06T04:21:08.561 INFO:teuthology.orchestra.run.smithi154.stdout:Scheduled mgr update... 2024-06-06T04:21:08.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.563+0000 7fc2aeda1700 1 -- 172.21.15.154:0/584263345 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc298039580 msgr2=0x7fc29803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:08.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.563+0000 7fc2aeda1700 1 --2- 172.21.15.154:0/584263345 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc298039580 0x7fc29803ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc294009ad0 tx=0x7fc294009210 comp rx=0 tx=0).stop 2024-06-06T04:21:08.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.563+0000 7fc2aeda1700 1 -- 172.21.15.154:0/584263345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a8100f10 msgr2=0x7fc2a806b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:08.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.563+0000 7fc2aeda1700 1 --2- 172.21.15.154:0/584263345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a8100f10 0x7fc2a806b260 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc29c005e00 tx=0x7fc29c004fa0 comp rx=0 tx=0).stop 2024-06-06T04:21:08.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.563+0000 7fc2aeda1700 1 -- 172.21.15.154:0/584263345 shutdown_connections 2024-06-06T04:21:08.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.563+0000 7fc2aeda1700 1 --2- 172.21.15.154:0/584263345 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc298039580 0x7fc29803ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:08.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.563+0000 7fc2aeda1700 1 --2- 172.21.15.154:0/584263345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a819ce00 0x7fc2a819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:08.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.563+0000 7fc2aeda1700 1 --2- 172.21.15.154:0/584263345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a8100f10 0x7fc2a806b260 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:08.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.563+0000 7fc2aeda1700 1 --2- 172.21.15.154:0/584263345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2a819a430 0x7fc2a819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:08.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.563+0000 7fc2aeda1700 1 -- 172.21.15.154:0/584263345 >> 172.21.15.154:0/584263345 conn(0x7fc2a80fc4d0 msgr2=0x7fc2a80fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:21:08.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.563+0000 7fc2aeda1700 1 -- 172.21.15.154:0/584263345 shutdown_connections 2024-06-06T04:21:08.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:21:08.563+0000 7fc2aeda1700 1 -- 172.21.15.154:0/584263345 wait complete. 2024-06-06T04:21:09.132 DEBUG:teuthology.orchestra.run.smithi086:mgr.y> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mgr.y.service 2024-06-06T04:21:09.134 DEBUG:teuthology.orchestra.run.smithi154:mgr.z> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mgr.z.service 2024-06-06T04:21:09.137 INFO:tasks.cephadm:Deploying OSDs... 2024-06-06T04:21:09.137 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:21:09.137 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-06-06T04:21:09.158 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-06T04:21:09.158 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-06-06T04:21:09.218 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-06T04:21:09.218 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T04:21:09.218 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-06-06T04:21:09.218 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T04:21:09.218 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T04:21:09.218 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-06-06 04:20:16.459020859 +0000 2024-06-06T04:21:09.218 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-06-06 03:50:15.610039015 +0000 2024-06-06T04:21:09.218 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-06-06 03:50:15.610039015 +0000 2024-06-06T04:21:09.218 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-06-06 03:50:15.610039015 +0000 2024-06-06T04:21:09.219 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-06T04:21:09.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:08 smithi045 ceph-mon[32299]: Reconfiguring mon.b (monmap changed)... 2024-06-06T04:21:09.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:08 smithi045 ceph-mon[32299]: Reconfiguring daemon mon.b on smithi086 2024-06-06T04:21:09.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:08 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:09.256 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-06-06T04:21:09.257 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-06-06T04:21:09.257 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000111595 s, 4.6 MB/s 2024-06-06T04:21:09.258 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-06T04:21:09.318 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-06-06T04:21:09.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:08 smithi086 ceph-mon[36053]: Reconfiguring mon.b (monmap changed)... 2024-06-06T04:21:09.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:08 smithi086 ceph-mon[36053]: Reconfiguring daemon mon.b on smithi086 2024-06-06T04:21:09.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:08 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:09.376 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-06T04:21:09.377 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T04:21:09.377 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-06-06T04:21:09.377 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T04:21:09.377 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T04:21:09.377 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-06-06 04:20:16.494020323 +0000 2024-06-06T04:21:09.377 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-06-06 03:50:15.977033485 +0000 2024-06-06T04:21:09.377 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-06-06 03:50:15.977033485 +0000 2024-06-06T04:21:09.377 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-06-06 03:50:15.977033485 +0000 2024-06-06T04:21:09.377 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-06T04:21:09.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:08 smithi154 ceph-mon[36428]: Reconfiguring mon.b (monmap changed)... 2024-06-06T04:21:09.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:08 smithi154 ceph-mon[36428]: Reconfiguring daemon mon.b on smithi086 2024-06-06T04:21:09.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:08 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:09.445 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-06-06T04:21:09.445 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-06-06T04:21:09.445 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000120277 s, 4.3 MB/s 2024-06-06T04:21:09.447 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-06T04:21:09.506 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-06-06T04:21:09.563 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-06T04:21:09.563 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T04:21:09.563 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-06-06T04:21:09.563 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T04:21:09.563 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T04:21:09.563 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-06-06 04:20:16.529019786 +0000 2024-06-06T04:21:09.563 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-06-06 03:50:16.308028499 +0000 2024-06-06T04:21:09.563 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-06-06 03:50:16.308028499 +0000 2024-06-06T04:21:09.563 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-06-06 03:50:16.308028499 +0000 2024-06-06T04:21:09.564 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-06T04:21:09.629 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-06-06T04:21:09.629 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-06-06T04:21:09.630 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 9.5e-05 s, 5.4 MB/s 2024-06-06T04:21:09.631 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-06T04:21:09.689 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-06-06T04:21:09.746 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-06T04:21:09.746 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T04:21:09.746 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-06T04:21:09.746 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T04:21:09.746 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T04:21:09.746 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-06-06 04:20:16.563019265 +0000 2024-06-06T04:21:09.746 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-06-06 03:50:16.638023526 +0000 2024-06-06T04:21:09.746 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-06-06 03:50:16.638023526 +0000 2024-06-06T04:21:09.747 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-06-06 03:50:16.638023526 +0000 2024-06-06T04:21:09.747 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-06T04:21:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-06-06T04:21:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-06-06T04:21:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 9.7111e-05 s, 5.3 MB/s 2024-06-06T04:21:09.812 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-06T04:21:09.869 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-06-06T04:21:09.869 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-06-06T04:21:09.888 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-06T04:21:09.888 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-06-06T04:21:09.946 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-06T04:21:09.946 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T04:21:09.946 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-06-06T04:21:09.947 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T04:21:09.947 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T04:21:09.947 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-06-06 04:20:56.908234780 +0000 2024-06-06T04:21:09.947 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-06-06 03:51:17.168813720 +0000 2024-06-06T04:21:09.947 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-06-06 03:51:17.168813720 +0000 2024-06-06T04:21:09.947 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-06-06 03:51:17.168813720 +0000 2024-06-06T04:21:09.947 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-06T04:21:10.014 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-06-06T04:21:10.014 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-06-06T04:21:10.014 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.00025859 s, 2.0 MB/s 2024-06-06T04:21:10.015 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-06T04:21:10.073 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-06-06T04:21:10.130 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-06T04:21:10.130 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T04:21:10.130 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-06-06T04:21:10.130 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T04:21:10.130 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T04:21:10.130 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-06-06 04:20:56.908234780 +0000 2024-06-06T04:21:10.130 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-06-06 03:51:17.488807021 +0000 2024-06-06T04:21:10.130 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-06-06 03:51:17.488807021 +0000 2024-06-06T04:21:10.130 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-06-06 03:51:17.488807021 +0000 2024-06-06T04:21:10.130 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-06T04:21:10.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:09 smithi154 ceph-mon[36428]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:10.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:09 smithi154 ceph-mon[36428]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi086=y;smithi154=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:21:10.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:09 smithi154 ceph-mon[36428]: Saving service mgr spec with placement smithi045=x;smithi086=y;smithi154=z;count:3 2024-06-06T04:21:10.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:09 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:10.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:09 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:10.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:09 smithi154 ceph-mon[36428]: Reconfiguring mon.c (monmap changed)... 2024-06-06T04:21:10.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:09 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T04:21:10.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:09 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T04:21:10.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:09 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:10.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:09 smithi154 ceph-mon[36428]: Reconfiguring daemon mon.c on smithi154 2024-06-06T04:21:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-06-06T04:21:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-06-06T04:21:10.195 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000174924 s, 2.9 MB/s 2024-06-06T04:21:10.196 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-06T04:21:10.253 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-06-06T04:21:10.309 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-06T04:21:10.309 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T04:21:10.309 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-06-06T04:21:10.309 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T04:21:10.309 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T04:21:10.309 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-06-06 04:20:56.908234780 +0000 2024-06-06T04:21:10.309 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-06-06 03:51:17.799800511 +0000 2024-06-06T04:21:10.309 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-06-06 03:51:17.799800511 +0000 2024-06-06T04:21:10.310 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-06-06 03:51:17.799800511 +0000 2024-06-06T04:21:10.310 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-06T04:21:10.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:09 smithi086 ceph-mon[36053]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:10.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:09 smithi086 ceph-mon[36053]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi086=y;smithi154=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:21:10.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:09 smithi086 ceph-mon[36053]: Saving service mgr spec with placement smithi045=x;smithi086=y;smithi154=z;count:3 2024-06-06T04:21:10.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:09 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:10.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:09 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:10.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:09 smithi086 ceph-mon[36053]: Reconfiguring mon.c (monmap changed)... 2024-06-06T04:21:10.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:09 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T04:21:10.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:09 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T04:21:10.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:09 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:10.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:09 smithi086 ceph-mon[36053]: Reconfiguring daemon mon.c on smithi154 2024-06-06T04:21:10.373 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-06-06T04:21:10.373 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-06-06T04:21:10.373 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000194919 s, 2.6 MB/s 2024-06-06T04:21:10.374 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-06T04:21:10.430 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-06-06T04:21:10.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:09 smithi045 ceph-mon[32299]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:09 smithi045 ceph-mon[32299]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi086=y;smithi154=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:21:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:09 smithi045 ceph-mon[32299]: Saving service mgr spec with placement smithi045=x;smithi086=y;smithi154=z;count:3 2024-06-06T04:21:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:09 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:09 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:09 smithi045 ceph-mon[32299]: Reconfiguring mon.c (monmap changed)... 2024-06-06T04:21:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:09 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T04:21:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:09 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T04:21:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:09 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:09 smithi045 ceph-mon[32299]: Reconfiguring daemon mon.c on smithi154 2024-06-06T04:21:10.485 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-06T04:21:10.485 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T04:21:10.485 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-06-06T04:21:10.485 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T04:21:10.485 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T04:21:10.485 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-06-06 04:20:56.908234780 +0000 2024-06-06T04:21:10.485 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-06-06 03:51:18.124793707 +0000 2024-06-06T04:21:10.485 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-06-06 03:51:18.124793707 +0000 2024-06-06T04:21:10.485 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-06-06 03:51:18.124793707 +0000 2024-06-06T04:21:10.486 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-06T04:21:10.549 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-06-06T04:21:10.549 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-06-06T04:21:10.549 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000215778 s, 2.4 MB/s 2024-06-06T04:21:10.550 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-06T04:21:10.606 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-06-06T04:21:10.606 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2024-06-06T04:21:10.623 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-06T04:21:10.623 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2024-06-06T04:21:10.679 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-06T04:21:10.679 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T04:21:10.679 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-06-06T04:21:10.679 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T04:21:10.679 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T04:21:10.679 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-06-06 04:20:47.807999297 +0000 2024-06-06T04:21:10.679 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-06-06 03:48:19.738720602 +0000 2024-06-06T04:21:10.679 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-06-06 03:48:19.738720602 +0000 2024-06-06T04:21:10.679 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-06-06 03:48:19.738720602 +0000 2024-06-06T04:21:10.679 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-06T04:21:10.743 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-06-06T04:21:10.744 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-06-06T04:21:10.744 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000215325 s, 2.4 MB/s 2024-06-06T04:21:10.745 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-06T04:21:10.801 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2024-06-06T04:21:10.857 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-06T04:21:10.857 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T04:21:10.857 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-06-06T04:21:10.857 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T04:21:10.857 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T04:21:10.857 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-06-06 04:20:47.808999269 +0000 2024-06-06T04:21:10.857 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-06-06 03:48:20.081711271 +0000 2024-06-06T04:21:10.858 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-06-06 03:48:20.081711271 +0000 2024-06-06T04:21:10.858 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-06-06 03:48:20.081711271 +0000 2024-06-06T04:21:10.858 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-06T04:21:10.920 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-06-06T04:21:10.920 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-06-06T04:21:10.921 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000202764 s, 2.5 MB/s 2024-06-06T04:21:10.922 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-06T04:21:10.978 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2024-06-06T04:21:11.033 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-06T04:21:11.034 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T04:21:11.034 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-06-06T04:21:11.034 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T04:21:11.034 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T04:21:11.034 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-06-06 04:20:47.808999269 +0000 2024-06-06T04:21:11.034 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-06-06 03:48:20.405702457 +0000 2024-06-06T04:21:11.034 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-06-06 03:48:20.405702457 +0000 2024-06-06T04:21:11.034 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-06-06 03:48:20.405702457 +0000 2024-06-06T04:21:11.034 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-06T04:21:11.097 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-06-06T04:21:11.097 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-06-06T04:21:11.098 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000198893 s, 2.6 MB/s 2024-06-06T04:21:11.099 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-06T04:21:11.155 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2024-06-06T04:21:11.210 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-06T04:21:11.210 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T04:21:11.210 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-06T04:21:11.210 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T04:21:11.210 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T04:21:11.210 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-06-06 04:20:47.809999241 +0000 2024-06-06T04:21:11.210 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-06-06 03:48:20.756692908 +0000 2024-06-06T04:21:11.211 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-06-06 03:48:20.756692908 +0000 2024-06-06T04:21:11.211 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-06-06 03:48:20.756692908 +0000 2024-06-06T04:21:11.211 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-06T04:21:11.275 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-06-06T04:21:11.275 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-06-06T04:21:11.275 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000175163 s, 2.9 MB/s 2024-06-06T04:21:11.276 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-06T04:21:11.332 INFO:tasks.cephadm:Deploying osd.0 on smithi045 with /dev/vg_nvme/lv_4... 2024-06-06T04:21:11.332 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-06T04:21:11.572 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:21:11.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:11 smithi045 ceph-mon[32299]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:11.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:11.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:11.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:11.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:11.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:11.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:11.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T04:21:11.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T04:21:11.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T04:21:11.695 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:11.696 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:11 smithi045 ceph-mon[32299]: Deploying daemon mgr.z on smithi154 2024-06-06T04:21:11.764 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:11 smithi154 ceph-mon[36428]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:11.765 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:11.765 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:11.765 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:11.765 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:11.765 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:11.765 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:11.765 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T04:21:11.765 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T04:21:11.765 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T04:21:11.765 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:11.766 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:11 smithi154 ceph-mon[36428]: Deploying daemon mgr.z on smithi154 2024-06-06T04:21:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:11 smithi086 ceph-mon[36053]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:11.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T04:21:11.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T04:21:11.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T04:21:11.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:11.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:11 smithi086 ceph-mon[36053]: Deploying daemon mgr.z on smithi154 2024-06-06T04:21:13.181 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:13 smithi154 systemd[1]: Starting Ceph mgr.z for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:21:13.438 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-06T04:21:13.438 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-06T04:21:13.438 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-06-06T04:21:13.438 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-06-06T04:21:13.438 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00835451 s, 1.3 GB/s 2024-06-06T04:21:13.439 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-06-06T04:21:13.527 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:13 smithi154 ceph-mon[36428]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:13.528 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:13 smithi154 podman[38262]: 2024-06-06 04:21:13.179980424 +0000 UTC m=+0.016890995 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:21:13.528 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:13 smithi154 podman[38262]: 2024-06-06 04:21:13.279801699 +0000 UTC m=+0.116712252 container create c34767ed5a646046d08c5709b65c1c7d5d4da759a27fadbe3e625b04625b3781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T04:21:13.528 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:13 smithi154 podman[38262]: 2024-06-06 04:21:13.499579164 +0000 UTC m=+0.336489726 container init c34767ed5a646046d08c5709b65c1c7d5d4da759a27fadbe3e625b04625b3781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-06T04:21:13.528 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:13 smithi154 podman[38262]: 2024-06-06 04:21:13.502903213 +0000 UTC m=+0.339813769 container start c34767ed5a646046d08c5709b65c1c7d5d4da759a27fadbe3e625b04625b3781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528) 2024-06-06T04:21:13.528 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:13 smithi154 ceph-mgr[38281]: -- 172.21.15.154:0/3088393938 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55afda2dc5a0 con 0x55afda2b1000 2024-06-06T04:21:13.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:13 smithi045 ceph-mon[32299]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:13.856 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:13 smithi154 bash[38262]: c34767ed5a646046d08c5709b65c1c7d5d4da759a27fadbe3e625b04625b3781 2024-06-06T04:21:13.856 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:13 smithi154 systemd[1]: Started Ceph mgr.z for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:21:13.856 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:13 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:13.654+0000 7fa9ffcbc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T04:21:13.858 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:13 smithi086 ceph-mon[36053]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:14.040 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_4 2024-06-06T04:21:14.180 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:13 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:13.871+0000 7fa9ffcbc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T04:21:14.180 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:13 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:13.975+0000 7fa9ffcbc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T04:21:14.283 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:21:14.680 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:14 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:14.285+0000 7fa9ffcbc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T04:21:14.680 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:14 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:14.370+0000 7fa9ffcbc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T04:21:14.866 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:14.866 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:14.866 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:14.866 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T04:21:14.866 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T04:21:14.866 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T04:21:14.866 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:14.867 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:14 smithi086 ceph-mon[36053]: Deploying daemon mgr.y on smithi086 2024-06-06T04:21:14.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:14.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:14.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:14.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T04:21:14.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T04:21:14.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T04:21:14.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:14.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:14 smithi045 ceph-mon[32299]: Deploying daemon mgr.y on smithi086 2024-06-06T04:21:15.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:15.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:15.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:15.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T04:21:15.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T04:21:15.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T04:21:15.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:15.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:14 smithi154 ceph-mon[36428]: Deploying daemon mgr.y on smithi086 2024-06-06T04:21:15.701 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:15 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:15.571+0000 7fa9ffcbc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T04:21:15.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.792+0000 7f3e95ebe700 1 -- 172.21.15.45:0/4203894140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e90104090 msgr2=0x7f3e9010a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:15.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.792+0000 7f3e95ebe700 1 --2- 172.21.15.45:0/4203894140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e90104090 0x7f3e9010a940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800b0d0 tx=0x7f3e8800b3e0 comp rx=0 tx=0).stop 2024-06-06T04:21:15.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.792+0000 7f3e95ebe700 1 -- 172.21.15.45:0/4203894140 shutdown_connections 2024-06-06T04:21:15.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.792+0000 7f3e95ebe700 1 --2- 172.21.15.45:0/4203894140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e90104090 0x7f3e9010a940 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:15.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.792+0000 7f3e95ebe700 1 --2- 172.21.15.45:0/4203894140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e901024f0 0x7f3e90102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:15.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.792+0000 7f3e95ebe700 1 --2- 172.21.15.45:0/4203894140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e901036f0 0x7f3e90103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:15.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.792+0000 7f3e95ebe700 1 -- 172.21.15.45:0/4203894140 >> 172.21.15.45:0/4203894140 conn(0x7f3e900fda90 msgr2=0x7f3e900ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:21:15.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.792+0000 7f3e95ebe700 1 -- 172.21.15.45:0/4203894140 shutdown_connections 2024-06-06T04:21:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.792+0000 7f3e95ebe700 1 -- 172.21.15.45:0/4203894140 wait complete. 2024-06-06T04:21:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.792+0000 7f3e95ebe700 1 Processor -- start 2024-06-06T04:21:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.792+0000 7f3e95ebe700 1 -- start start 2024-06-06T04:21:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.792+0000 7f3e95ebe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e901024f0 0x7f3e90076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.792+0000 7f3e95ebe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e901036f0 0x7f3e90074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e95ebe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e90104090 0x7f3e90074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e95ebe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9010df70 con 0x7f3e90104090 2024-06-06T04:21:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e95ebe700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9010ddf0 con 0x7f3e901036f0 2024-06-06T04:21:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e95ebe700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9010e0f0 con 0x7f3e901024f0 2024-06-06T04:21:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e8ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e90104090 0x7f3e90074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e8ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e90104090 0x7f3e90074dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51690/0 (socket says 172.21.15.45:51690) 2024-06-06T04:21:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e87fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e901036f0 0x7f3e90074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e8ffff700 1 -- 172.21.15.45:0/200392215 learned_addr learned my addr 172.21.15.45:0/200392215 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:21:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e8f7fe700 1 --2- 172.21.15.45:0/200392215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e901024f0 0x7f3e90076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e8ffff700 1 -- 172.21.15.45:0/200392215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e901024f0 msgr2=0x7f3e90076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e8ffff700 1 --2- 172.21.15.45:0/200392215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e901024f0 0x7f3e90076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e8ffff700 1 -- 172.21.15.45:0/200392215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e901036f0 msgr2=0x7f3e90074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e8ffff700 1 --2- 172.21.15.45:0/200392215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e901036f0 0x7f3e90074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e8ffff700 1 -- 172.21.15.45:0/200392215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e90075670 con 0x7f3e90104090 2024-06-06T04:21:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e87fff700 1 --2- 172.21.15.45:0/200392215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e901036f0 0x7f3e90074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:21:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e8ffff700 1 --2- 172.21.15.45:0/200392215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e90104090 0x7f3e90074dc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009cd0 tx=0x7f3e88004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:21:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e8d7fa700 1 -- 172.21.15.45:0/200392215 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8801f030 con 0x7f3e90104090 2024-06-06T04:21:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e95ebe700 1 -- 172.21.15.45:0/200392215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e901a26b0 con 0x7f3e90104090 2024-06-06T04:21:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e8d7fa700 1 -- 172.21.15.45:0/200392215 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e88005040 con 0x7f3e90104090 2024-06-06T04:21:15.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e8d7fa700 1 -- 172.21.15.45:0/200392215 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8801b670 con 0x7f3e90104090 2024-06-06T04:21:15.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.793+0000 7f3e95ebe700 1 -- 172.21.15.45:0/200392215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e901a2b10 con 0x7f3e90104090 2024-06-06T04:21:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.795+0000 7f3e8d7fa700 1 -- 172.21.15.45:0/200392215 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3e88022030 con 0x7f3e90104090 2024-06-06T04:21:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.795+0000 7f3e95ebe700 1 -- 172.21.15.45:0/200392215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e90069180 con 0x7f3e90104090 2024-06-06T04:21:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.795+0000 7f3e8d7fa700 1 --2- 172.21.15.45:0/200392215 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3e7c039330 0x7f3e7c03b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.795+0000 7f3e8d7fa700 1 -- 172.21.15.45:0/200392215 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3e8800f020 con 0x7f3e90104090 2024-06-06T04:21:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.795+0000 7f3e8f7fe700 1 --2- 172.21.15.45:0/200392215 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3e7c039330 0x7f3e7c03b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.795+0000 7f3e8f7fe700 1 --2- 172.21.15.45:0/200392215 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3e7c039330 0x7f3e7c03b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3e90103550 tx=0x7f3e8000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:21:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:15.800+0000 7f3e8d7fa700 1 -- 172.21.15.45:0/200392215 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3e88029020 con 0x7f3e90104090 2024-06-06T04:21:15.839 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:15 smithi086 ceph-mon[36053]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:15.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:15 smithi045 ceph-mon[32299]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:16.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:16.015+0000 7f3e95ebe700 1 -- 172.21.15.45:0/200392215 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3e900628d0 con 0x7f3e7c039330 2024-06-06T04:21:16.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:15 smithi154 ceph-mon[36428]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:16.180 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:15 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:15.746+0000 7fa9ffcbc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T04:21:16.180 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:15 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:15.926+0000 7fa9ffcbc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T04:21:16.482 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:16 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:16.293+0000 7f61e08f2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T04:21:16.482 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:16 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:16.481+0000 7f61e08f2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T04:21:16.524 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:16 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:16.221+0000 7fa9ffcbc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T04:21:16.524 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:16 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:16.316+0000 7fa9ffcbc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T04:21:16.524 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:16 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:16.523+0000 7fa9ffcbc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T04:21:16.736 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:16 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:16.588+0000 7f61e08f2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T04:21:16.930 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:16 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:16.741+0000 7fa9ffcbc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T04:21:16.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:16 smithi154 ceph-mon[36428]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:21:16.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:16 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:21:16.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:16 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:21:16.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:16 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:16.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:16 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:16.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:16 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:16.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:16 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:16.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:16 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:16.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:16 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:16.931 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:16 smithi045 ceph-mon[32299]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:21:16.931 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:16 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:21:16.931 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:16 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:21:16.932 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:16 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:16.932 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:16 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:16.932 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:16 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:16.932 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:16 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:16.932 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:16 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:16.932 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:16 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:17.001 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:16 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:16.908+0000 7f61e08f2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T04:21:17.001 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:17 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:17.000+0000 7f61e08f2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T04:21:17.001 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:16 smithi086 ceph-mon[36053]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:21:17.001 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:16 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:21:17.001 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:16 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:21:17.001 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:16 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:17.001 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:16 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:17.002 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:16 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:17.002 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:16 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:17.002 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:16 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:17.002 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:16 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:17.680 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:17 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:17.378+0000 7fa9ffcbc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T04:21:17.850 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:17 smithi045 ceph-mon[32299]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:18.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:17 smithi086 ceph-mon[36053]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:18.115 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:17 smithi154 ceph-mon[36428]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:18.116 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:17 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:17.817+0000 7fa9ffcbc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T04:21:18.116 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:17 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:17.912+0000 7fa9ffcbc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T04:21:18.116 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:18.115+0000 7fa9ffcbc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T04:21:18.585 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:18.252+0000 7f61e08f2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T04:21:18.585 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:18.418+0000 7f61e08f2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T04:21:18.586 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:18.585+0000 7f61e08f2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T04:21:18.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:18.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:18.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:18.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:18.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:18.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:18.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:18.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-mon[36053]: Reconfiguring mgr.x (unknown last config time)... 2024-06-06T04:21:18.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T04:21:18.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T04:21:18.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:18.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-mon[36053]: Reconfiguring daemon mgr.x on smithi045 2024-06-06T04:21:18.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3041321289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3cee4a-5025-45b3-b217-082af37e79db"}]: dispatch 2024-06-06T04:21:18.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3041321289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d3cee4a-5025-45b3-b217-082af37e79db"}]': finished 2024-06-06T04:21:18.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-mon[36053]: osdmap e5: 1 total, 0 up, 1 in 2024-06-06T04:21:18.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:18.930 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:18.634+0000 7fa9ffcbc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T04:21:18.930 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:18.724+0000 7fa9ffcbc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T04:21:18.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:18.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:18.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:18.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:18.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:18.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:18.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:18.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-mon[36428]: Reconfiguring mgr.x (unknown last config time)... 2024-06-06T04:21:18.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T04:21:18.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T04:21:18.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:18.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-mon[36428]: Reconfiguring daemon mgr.x on smithi045 2024-06-06T04:21:18.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3041321289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3cee4a-5025-45b3-b217-082af37e79db"}]: dispatch 2024-06-06T04:21:18.932 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3041321289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d3cee4a-5025-45b3-b217-082af37e79db"}]': finished 2024-06-06T04:21:18.932 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-mon[36428]: osdmap e5: 1 total, 0 up, 1 in 2024-06-06T04:21:18.932 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:19.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:19.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:19.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:19.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:19.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:19.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:19.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:19.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:18 smithi045 ceph-mon[32299]: Reconfiguring mgr.x (unknown last config time)... 2024-06-06T04:21:19.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T04:21:19.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T04:21:19.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:19.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:18 smithi045 ceph-mon[32299]: Reconfiguring daemon mgr.x on smithi045 2024-06-06T04:21:19.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:18 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3041321289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3cee4a-5025-45b3-b217-082af37e79db"}]: dispatch 2024-06-06T04:21:19.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:18 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3041321289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d3cee4a-5025-45b3-b217-082af37e79db"}]': finished 2024-06-06T04:21:19.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:18 smithi045 ceph-mon[32299]: osdmap e5: 1 total, 0 up, 1 in 2024-06-06T04:21:19.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:19.162 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:18.871+0000 7f61e08f2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T04:21:19.162 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:18 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:18.962+0000 7f61e08f2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T04:21:19.162 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:19 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:19.162+0000 7f61e08f2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T04:21:19.303 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:18 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:18.965+0000 7fa9ffcbc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T04:21:19.304 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:19 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:19.097+0000 7fa9ffcbc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T04:21:19.304 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:19 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:19.189+0000 7fa9ffcbc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T04:21:19.304 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:19 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:19.303+0000 7fa9ffcbc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T04:21:19.609 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:19 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:19.350+0000 7f61e08f2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T04:21:19.930 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:19 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:19.604+0000 7fa9ffcbc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T04:21:19.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:19 smithi154 ceph-mon[36428]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:19.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:19.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:19 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3490484933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:21:20.109 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:19 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:19.948+0000 7f61e08f2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T04:21:20.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:19 smithi086 ceph-mon[36053]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:20.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:20.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:19 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3490484933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:21:20.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:19 smithi045 ceph-mon[32299]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:20.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:20.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:19 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3490484933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:21:20.660 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:20 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:20.364+0000 7f61e08f2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T04:21:20.661 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:20 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:20.460+0000 7f61e08f2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T04:21:20.661 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:20 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:20.660+0000 7f61e08f2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T04:21:20.680 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:20 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:20.196+0000 7fa9ffcbc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T04:21:20.680 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:21:20 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z[38277]: 2024-06-06T04:21:20.408+0000 7fa9ffcbc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T04:21:20.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:20.806+0000 7f3e8d7fa700 1 -- 172.21.15.45:0/200392215 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f3e8801ac60 con 0x7f3e90104090 2024-06-06T04:21:21.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:20 smithi086 ceph-mon[36053]: Standby manager daemon z started 2024-06-06T04:21:21.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:20 smithi086 ceph-mon[36053]: from='mgr.? 172.21.15.154:0/640895496' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T04:21:21.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:20 smithi086 ceph-mon[36053]: from='mgr.? 172.21.15.154:0/640895496' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T04:21:21.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:20 smithi086 ceph-mon[36053]: from='mgr.? 172.21.15.154:0/640895496' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T04:21:21.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:20 smithi086 ceph-mon[36053]: from='mgr.? 172.21.15.154:0/640895496' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T04:21:21.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:20 smithi154 ceph-mon[36428]: Standby manager daemon z started 2024-06-06T04:21:21.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:20 smithi154 ceph-mon[36428]: from='mgr.? 172.21.15.154:0/640895496' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T04:21:21.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:20 smithi154 ceph-mon[36428]: from='mgr.? 172.21.15.154:0/640895496' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T04:21:21.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:20 smithi154 ceph-mon[36428]: from='mgr.? 172.21.15.154:0/640895496' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T04:21:21.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:20 smithi154 ceph-mon[36428]: from='mgr.? 172.21.15.154:0/640895496' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T04:21:21.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:20 smithi045 ceph-mon[32299]: Standby manager daemon z started 2024-06-06T04:21:21.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:20 smithi045 ceph-mon[32299]: from='mgr.? 172.21.15.154:0/640895496' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T04:21:21.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:20 smithi045 ceph-mon[32299]: from='mgr.? 172.21.15.154:0/640895496' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T04:21:21.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:20 smithi045 ceph-mon[32299]: from='mgr.? 172.21.15.154:0/640895496' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T04:21:21.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:20 smithi045 ceph-mon[32299]: from='mgr.? 172.21.15.154:0/640895496' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T04:21:21.519 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:21 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:21.186+0000 7f61e08f2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T04:21:21.519 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:21 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:21.279+0000 7f61e08f2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T04:21:21.805 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:21 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:21.519+0000 7f61e08f2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T04:21:21.805 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:21 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:21.650+0000 7f61e08f2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T04:21:21.805 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:21 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:21.746+0000 7f61e08f2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T04:21:22.109 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:21 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:21.861+0000 7f61e08f2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T04:21:22.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:21 smithi086 ceph-mon[36053]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:22.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:21 smithi086 ceph-mon[36053]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-06T04:21:22.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:21 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T04:21:22.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:21 smithi154 ceph-mon[36428]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:22.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:21 smithi154 ceph-mon[36428]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-06T04:21:22.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:21 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T04:21:22.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:21 smithi045 ceph-mon[32299]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:22.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:21 smithi045 ceph-mon[32299]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-06T04:21:22.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:21 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T04:21:22.609 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:22 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:22.156+0000 7f61e08f2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T04:21:23.109 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:22 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:22.748+0000 7f61e08f2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T04:21:23.109 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:21:22 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y[37425]: 2024-06-06T04:21:22.961+0000 7f61e08f2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T04:21:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:23.815+0000 7f3e8d7fa700 1 -- 172.21.15.45:0/200392215 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f3e88024410 con 0x7f3e90104090 2024-06-06T04:21:24.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:23 smithi086 ceph-mon[36053]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:24.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:23 smithi086 ceph-mon[36053]: Standby manager daemon y started 2024-06-06T04:21:24.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:23 smithi086 ceph-mon[36053]: from='mgr.? 172.21.15.86:0/1726073063' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T04:21:24.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:23 smithi086 ceph-mon[36053]: from='mgr.? 172.21.15.86:0/1726073063' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T04:21:24.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:23 smithi086 ceph-mon[36053]: from='mgr.? 172.21.15.86:0/1726073063' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T04:21:24.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:23 smithi086 ceph-mon[36053]: from='mgr.? 172.21.15.86:0/1726073063' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T04:21:24.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:23 smithi154 ceph-mon[36428]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:24.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:23 smithi154 ceph-mon[36428]: Standby manager daemon y started 2024-06-06T04:21:24.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:23 smithi154 ceph-mon[36428]: from='mgr.? 172.21.15.86:0/1726073063' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T04:21:24.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:23 smithi154 ceph-mon[36428]: from='mgr.? 172.21.15.86:0/1726073063' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T04:21:24.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:23 smithi154 ceph-mon[36428]: from='mgr.? 172.21.15.86:0/1726073063' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T04:21:24.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:23 smithi154 ceph-mon[36428]: from='mgr.? 172.21.15.86:0/1726073063' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T04:21:24.206 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:23 smithi045 ceph-mon[32299]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:24.206 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:23 smithi045 ceph-mon[32299]: Standby manager daemon y started 2024-06-06T04:21:24.206 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:23 smithi045 ceph-mon[32299]: from='mgr.? 172.21.15.86:0/1726073063' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T04:21:24.206 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:23 smithi045 ceph-mon[32299]: from='mgr.? 172.21.15.86:0/1726073063' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T04:21:24.206 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:23 smithi045 ceph-mon[32299]: from='mgr.? 172.21.15.86:0/1726073063' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T04:21:24.206 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:23 smithi045 ceph-mon[32299]: from='mgr.? 172.21.15.86:0/1726073063' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T04:21:24.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:24 smithi045 ceph-mon[32299]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-06-06T04:21:24.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:24 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T04:21:24.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:24 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:24.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:24 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:24.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:24 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:24.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:24 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:24.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:24 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:24.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:24 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:25.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:24 smithi086 ceph-mon[36053]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-06-06T04:21:25.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:24 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T04:21:25.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:24 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:25.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:24 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:25.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:24 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:25.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:24 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:25.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:24 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:25.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:24 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:25.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:24 smithi154 ceph-mon[36428]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-06-06T04:21:25.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:24 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T04:21:25.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:24 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:25.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:24 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:25.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:24 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:25.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:24 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:25.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:24 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:25.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:24 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:26.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:25 smithi086 ceph-mon[36053]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:26.148 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:25 smithi154 ceph-mon[36428]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:26.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:25 smithi045 ceph-mon[32299]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:28.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:27 smithi086 ceph-mon[36053]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:28.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:27 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-06T04:21:28.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:27 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:28.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:27 smithi086 ceph-mon[36053]: Deploying daemon osd.0 on smithi045 2024-06-06T04:21:28.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:27 smithi154 ceph-mon[36428]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:28.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:27 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-06T04:21:28.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:27 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:28.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:27 smithi154 ceph-mon[36428]: Deploying daemon osd.0 on smithi045 2024-06-06T04:21:28.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:27 smithi045 ceph-mon[32299]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:28.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:27 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-06T04:21:28.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:27 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:28.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:27 smithi045 ceph-mon[32299]: Deploying daemon osd.0 on smithi045 2024-06-06T04:21:30.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:29 smithi086 ceph-mon[36053]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:30.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:29 smithi154 ceph-mon[36428]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:30.183 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:29 smithi045 ceph-mon[32299]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:32.088 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:31 smithi045 ceph-mon[32299]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:32.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:31 smithi086 ceph-mon[36053]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:32.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:31 smithi154 ceph-mon[36428]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:33.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:33 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:33.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:33 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:33.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:33 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:33.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:33 smithi045 ceph-mon[32299]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:33.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:33 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:33.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:33 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:33.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:33 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:33.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:33 smithi086 ceph-mon[36053]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:33.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:33 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:33.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:33 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:33.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:33 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:33.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:33 smithi154 ceph-mon[36428]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:35.597 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:35 smithi045 ceph-mon[32299]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:35.597 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:35 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:35.597 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:35 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:35.597 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:35 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:35.597 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:35 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:35.598 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:35 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:35.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:35 smithi086 ceph-mon[36053]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:35.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:35 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:35.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:35 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:35.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:35 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:35.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:35 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:35.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:35 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:35.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:35 smithi154 ceph-mon[36428]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:35.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:35 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:35.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:35 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:35.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:35 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:35.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:35 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:35.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:35 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:35.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:35.723+0000 7f3e8d7fa700 1 -- 172.21.15.45:0/200392215 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3e900628d0 con 0x7f3e7c039330 2024-06-06T04:21:35.724 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 0 on host 'smithi045' 2024-06-06T04:21:35.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:35.725+0000 7f3e95ebe700 1 -- 172.21.15.45:0/200392215 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3e7c039330 msgr2=0x7f3e7c03b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:35.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:35.725+0000 7f3e95ebe700 1 --2- 172.21.15.45:0/200392215 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3e7c039330 0x7f3e7c03b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3e90103550 tx=0x7f3e8000b410 comp rx=0 tx=0).stop 2024-06-06T04:21:35.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:35.725+0000 7f3e95ebe700 1 -- 172.21.15.45:0/200392215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e90104090 msgr2=0x7f3e90074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:35.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:35.725+0000 7f3e95ebe700 1 --2- 172.21.15.45:0/200392215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e90104090 0x7f3e90074dc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009cd0 tx=0x7f3e88004d70 comp rx=0 tx=0).stop 2024-06-06T04:21:35.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:35.725+0000 7f3e95ebe700 1 -- 172.21.15.45:0/200392215 shutdown_connections 2024-06-06T04:21:35.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:35.725+0000 7f3e95ebe700 1 --2- 172.21.15.45:0/200392215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e90104090 0x7f3e90074dc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:35.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:35.725+0000 7f3e95ebe700 1 --2- 172.21.15.45:0/200392215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e901024f0 0x7f3e90076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:35.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:35.725+0000 7f3e95ebe700 1 --2- 172.21.15.45:0/200392215 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3e7c039330 0x7f3e7c03b7f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:35.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:35.725+0000 7f3e95ebe700 1 --2- 172.21.15.45:0/200392215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e901036f0 0x7f3e90074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:35.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:35.725+0000 7f3e95ebe700 1 -- 172.21.15.45:0/200392215 >> 172.21.15.45:0/200392215 conn(0x7f3e900fda90 msgr2=0x7f3e901069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:21:35.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:35.725+0000 7f3e95ebe700 1 -- 172.21.15.45:0/200392215 shutdown_connections 2024-06-06T04:21:35.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:21:35.725+0000 7f3e95ebe700 1 -- 172.21.15.45:0/200392215 wait complete. 2024-06-06T04:21:36.298 DEBUG:teuthology.orchestra.run.smithi045:osd.0> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.0.service 2024-06-06T04:21:36.300 INFO:tasks.cephadm:Deploying osd.1 on smithi086 with /dev/vg_nvme/lv_4... 2024-06-06T04:21:36.300 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-06T04:21:36.543 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T04:21:36.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:36 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:36.968 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:36 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:36.968 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:36 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:36.968 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:36 smithi154 ceph-mon[36428]: from='osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-06T04:21:36.975 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:36 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:36.975 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:36 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:36.975 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:36 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:36.975 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:36 smithi086 ceph-mon[36053]: from='osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-06T04:21:36.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:36 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:36.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:36 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:36.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:36 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:36.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:36 smithi045 ceph-mon[32299]: from='osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-06T04:21:37.821 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:37 smithi045 ceph-mon[32299]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:37.821 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:37 smithi045 ceph-mon[32299]: from='osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-06T04:21:37.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:37 smithi045 ceph-mon[32299]: osdmap e6: 1 total, 0 up, 1 in 2024-06-06T04:21:37.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:37 smithi045 ceph-mon[32299]: from='osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-06-06T04:21:37.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:37 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:37.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:37 smithi086 ceph-mon[36053]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:37.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:37 smithi086 ceph-mon[36053]: from='osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-06T04:21:37.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:37 smithi086 ceph-mon[36053]: osdmap e6: 1 total, 0 up, 1 in 2024-06-06T04:21:37.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:37 smithi086 ceph-mon[36053]: from='osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-06-06T04:21:37.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:37 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:38.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:37 smithi154 ceph-mon[36428]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:38.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:37 smithi154 ceph-mon[36428]: from='osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-06T04:21:38.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:37 smithi154 ceph-mon[36428]: osdmap e6: 1 total, 0 up, 1 in 2024-06-06T04:21:38.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:37 smithi154 ceph-mon[36428]: from='osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-06-06T04:21:38.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:37 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:38.412 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-06T04:21:38.412 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-06T04:21:38.412 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-06-06T04:21:38.412 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-06-06T04:21:38.412 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011362 s, 923 MB/s 2024-06-06T04:21:38.412 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-06-06T04:21:38.477 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:21:38 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-0[47449]: 2024-06-06T04:21:38.144+0000 7f45ecb7a700 -1 osd.0 0 waiting for initial osdmap 2024-06-06T04:21:38.477 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:21:38 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-0[47449]: 2024-06-06T04:21:38.147+0000 7f45e4965700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T04:21:38.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:38 smithi045 ceph-mon[32299]: from='osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-06-06T04:21:38.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:38 smithi045 ceph-mon[32299]: osdmap e7: 1 total, 0 up, 1 in 2024-06-06T04:21:38.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:38 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:38.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:38 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:38.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:38 smithi045 ceph-mon[32299]: from='osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250]' entity='osd.0' 2024-06-06T04:21:39.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:38 smithi086 ceph-mon[36053]: from='osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-06-06T04:21:39.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:38 smithi086 ceph-mon[36053]: osdmap e7: 1 total, 0 up, 1 in 2024-06-06T04:21:39.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:38 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:39.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:38 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:39.025 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:38 smithi086 ceph-mon[36053]: from='osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250]' entity='osd.0' 2024-06-06T04:21:39.052 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_4 2024-06-06T04:21:39.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:38 smithi154 ceph-mon[36428]: from='osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-06-06T04:21:39.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:38 smithi154 ceph-mon[36428]: osdmap e7: 1 total, 0 up, 1 in 2024-06-06T04:21:39.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:38 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:39.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:38 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:39.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:38 smithi154 ceph-mon[36428]: from='osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250]' entity='osd.0' 2024-06-06T04:21:39.289 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T04:21:39.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:39 smithi045 ceph-mon[32299]: purged_snaps scrub starts 2024-06-06T04:21:39.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:39 smithi045 ceph-mon[32299]: purged_snaps scrub ok 2024-06-06T04:21:39.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:39 smithi045 ceph-mon[32299]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:39.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:39 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:39.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:39 smithi045 ceph-mon[32299]: osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250] boot 2024-06-06T04:21:39.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:39 smithi045 ceph-mon[32299]: osdmap e8: 1 total, 1 up, 1 in 2024-06-06T04:21:39.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:39 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:39.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:39 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:39.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:39 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:39.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:39 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:21:39.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:39 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:39.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:39 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:39.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:39 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:39.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:39 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:40.072 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:39 smithi086 ceph-mon[36053]: purged_snaps scrub starts 2024-06-06T04:21:40.072 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:39 smithi086 ceph-mon[36053]: purged_snaps scrub ok 2024-06-06T04:21:40.072 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:39 smithi086 ceph-mon[36053]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:40.073 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:39 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:40.073 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:39 smithi086 ceph-mon[36053]: osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250] boot 2024-06-06T04:21:40.073 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:39 smithi086 ceph-mon[36053]: osdmap e8: 1 total, 1 up, 1 in 2024-06-06T04:21:40.073 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:39 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:40.073 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:39 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:40.073 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:39 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:40.073 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:39 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:21:40.073 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:39 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:40.073 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:39 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:40.073 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:39 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:40.073 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:39 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:40.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:39 smithi154 ceph-mon[36428]: purged_snaps scrub starts 2024-06-06T04:21:40.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:39 smithi154 ceph-mon[36428]: purged_snaps scrub ok 2024-06-06T04:21:40.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:39 smithi154 ceph-mon[36428]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T04:21:40.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:39 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:40.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:39 smithi154 ceph-mon[36428]: osd.0 [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250] boot 2024-06-06T04:21:40.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:39 smithi154 ceph-mon[36428]: osdmap e8: 1 total, 1 up, 1 in 2024-06-06T04:21:40.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:39 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T04:21:40.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:39 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:40.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:39 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:40.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:39 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:21:40.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:39 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:40.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:39 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:40.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:39 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:40.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:39 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.811+0000 7f82a7808700 1 -- 172.21.15.86:0/806856296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a0101410 msgr2=0x7f82a0101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.811+0000 7f82a7808700 1 --2- 172.21.15.86:0/806856296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a0101410 0x7f82a0101830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f829c0099c0 tx=0x7f829c009cd0 comp rx=0 tx=0).stop 2024-06-06T04:21:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.812+0000 7f82a7808700 1 -- 172.21.15.86:0/806856296 shutdown_connections 2024-06-06T04:21:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.812+0000 7f82a7808700 1 --2- 172.21.15.86:0/806856296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a0102fb0 0x7f82a0109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.812+0000 7f82a7808700 1 --2- 172.21.15.86:0/806856296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a0101410 0x7f82a0101830 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.812+0000 7f82a7808700 1 --2- 172.21.15.86:0/806856296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a0102610 0x7f82a0102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:40.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.812+0000 7f82a7808700 1 -- 172.21.15.86:0/806856296 >> 172.21.15.86:0/806856296 conn(0x7f82a00fc9f0 msgr2=0x7f82a00fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:21:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.812+0000 7f82a7808700 1 -- 172.21.15.86:0/806856296 shutdown_connections 2024-06-06T04:21:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.812+0000 7f82a7808700 1 -- 172.21.15.86:0/806856296 wait complete. 2024-06-06T04:21:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.812+0000 7f82a7808700 1 Processor -- start 2024-06-06T04:21:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.812+0000 7f82a7808700 1 -- start start 2024-06-06T04:21:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.812+0000 7f82a7808700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a0101410 0x7f82a01053a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.812+0000 7f82a7808700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a0102610 0x7f82a01039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.812+0000 7f82a7808700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a0102fb0 0x7f82a0103f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.812+0000 7f82a7808700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82a010cf10 con 0x7f82a0102610 2024-06-06T04:21:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.812+0000 7f82a7808700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82a010cd90 con 0x7f82a0102fb0 2024-06-06T04:21:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.812+0000 7f82a7808700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f82a010d090 con 0x7f82a0101410 2024-06-06T04:21:40.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.812+0000 7f82a5da5700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a0102fb0 0x7f82a0103f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.813+0000 7f82a5da5700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a0102fb0 0x7f82a0103f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35258/0 (socket says 172.21.15.86:35258) 2024-06-06T04:21:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.813+0000 7f82a5da5700 1 -- 172.21.15.86:0/1511234068 learned_addr learned my addr 172.21.15.86:0/1511234068 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-06-06T04:21:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.813+0000 7f82a55a4700 1 --2- 172.21.15.86:0/1511234068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a0101410 0x7f82a01053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.813+0000 7f82a4da3700 1 --2- 172.21.15.86:0/1511234068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a0102610 0x7f82a01039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.813+0000 7f82a5da5700 1 -- 172.21.15.86:0/1511234068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a0101410 msgr2=0x7f82a01053a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.813+0000 7f82a5da5700 1 --2- 172.21.15.86:0/1511234068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a0101410 0x7f82a01053a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.813+0000 7f82a5da5700 1 -- 172.21.15.86:0/1511234068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a0102610 msgr2=0x7f82a01039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.813+0000 7f82a5da5700 1 --2- 172.21.15.86:0/1511234068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a0102610 0x7f82a01039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.813+0000 7f82a5da5700 1 -- 172.21.15.86:0/1511234068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a01047e0 con 0x7f82a0102fb0 2024-06-06T04:21:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.813+0000 7f82a5da5700 1 --2- 172.21.15.86:0/1511234068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a0102fb0 0x7f82a0103f30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f829800b970 tx=0x7f829800bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:21:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.813+0000 7f82967fc700 1 -- 172.21.15.86:0/1511234068 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298019070 con 0x7f82a0102fb0 2024-06-06T04:21:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.813+0000 7f82967fc700 1 -- 172.21.15.86:0/1511234068 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f829800cde0 con 0x7f82a0102fb0 2024-06-06T04:21:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.813+0000 7f82a7808700 1 -- 172.21.15.86:0/1511234068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a01a27d0 con 0x7f82a0102fb0 2024-06-06T04:21:40.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.813+0000 7f82967fc700 1 -- 172.21.15.86:0/1511234068 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298007370 con 0x7f82a0102fb0 2024-06-06T04:21:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.813+0000 7f82a7808700 1 -- 172.21.15.86:0/1511234068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a01a2d60 con 0x7f82a0102fb0 2024-06-06T04:21:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.814+0000 7f82a7808700 1 -- 172.21.15.86:0/1511234068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a01a3020 con 0x7f82a0102fb0 2024-06-06T04:21:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.815+0000 7f82967fc700 1 -- 172.21.15.86:0/1511234068 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f829801c070 con 0x7f82a0102fb0 2024-06-06T04:21:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.816+0000 7f82967fc700 1 --2- 172.21.15.86:0/1511234068 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f828c0a2c10 0x7f828c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:21:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.816+0000 7f82967fc700 1 -- 172.21.15.86:0/1511234068 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f82980cca40 con 0x7f82a0102fb0 2024-06-06T04:21:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.816+0000 7f82a55a4700 1 --2- 172.21.15.86:0/1511234068 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f828c0a2c10 0x7f828c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:21:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.817+0000 7f82a55a4700 1 --2- 172.21.15.86:0/1511234068 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f828c0a2c10 0x7f828c0a50d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f829c000c00 tx=0x7f829c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:21:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:40.821+0000 7f82967fc700 1 -- 172.21.15.86:0/1511234068 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f829809b410 con 0x7f82a0102fb0 2024-06-06T04:21:41.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:41.026+0000 7f82a7808700 1 -- 172.21.15.86:0/1511234068 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f82a00febc0 con 0x7f828c0a2c10 2024-06-06T04:21:41.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:41 smithi086 ceph-mon[36053]: Detected new or changed devices on smithi045 2024-06-06T04:21:41.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:41 smithi086 ceph-mon[36053]: Adjusting osd_memory_target on smithi045 to 17092M 2024-06-06T04:21:41.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:41 smithi086 ceph-mon[36053]: osdmap e9: 1 total, 1 up, 1 in 2024-06-06T04:21:41.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:41 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:21:41.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:41 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:21:41.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:41 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:41.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:41 smithi154 ceph-mon[36428]: Detected new or changed devices on smithi045 2024-06-06T04:21:41.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:41 smithi154 ceph-mon[36428]: Adjusting osd_memory_target on smithi045 to 17092M 2024-06-06T04:21:41.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:41 smithi154 ceph-mon[36428]: osdmap e9: 1 total, 1 up, 1 in 2024-06-06T04:21:41.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:41 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:21:41.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:41 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:21:41.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:41 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:41.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:41 smithi045 ceph-mon[32299]: Detected new or changed devices on smithi045 2024-06-06T04:21:41.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:41 smithi045 ceph-mon[32299]: Adjusting osd_memory_target on smithi045 to 17092M 2024-06-06T04:21:41.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:41 smithi045 ceph-mon[32299]: osdmap e9: 1 total, 1 up, 1 in 2024-06-06T04:21:41.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:21:41.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:21:41.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:42.297 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:42 smithi086 ceph-mon[36053]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:42.297 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:42 smithi086 ceph-mon[36053]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:21:42.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:42 smithi154 ceph-mon[36428]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:42.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:42 smithi154 ceph-mon[36428]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:21:42.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:42 smithi045 ceph-mon[32299]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:42.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:42 smithi045 ceph-mon[32299]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:21:44.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:44 smithi086 ceph-mon[36053]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:44.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:44 smithi086 ceph-mon[36053]: from='client.? 172.21.15.86:0/3396573366' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad"}]: dispatch 2024-06-06T04:21:44.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:44 smithi086 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad"}]: dispatch 2024-06-06T04:21:44.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:44 smithi086 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad"}]': finished 2024-06-06T04:21:44.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:44 smithi086 ceph-mon[36053]: osdmap e10: 2 total, 1 up, 2 in 2024-06-06T04:21:44.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:44 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T04:21:44.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:44 smithi154 ceph-mon[36428]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:44.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:44 smithi154 ceph-mon[36428]: from='client.? 172.21.15.86:0/3396573366' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad"}]: dispatch 2024-06-06T04:21:44.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:44 smithi154 ceph-mon[36428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad"}]: dispatch 2024-06-06T04:21:44.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:44 smithi154 ceph-mon[36428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad"}]': finished 2024-06-06T04:21:44.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:44 smithi154 ceph-mon[36428]: osdmap e10: 2 total, 1 up, 2 in 2024-06-06T04:21:44.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:44 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T04:21:44.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:44 smithi045 ceph-mon[32299]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:44.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:44 smithi045 ceph-mon[32299]: from='client.? 172.21.15.86:0/3396573366' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad"}]: dispatch 2024-06-06T04:21:44.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:44 smithi045 ceph-mon[32299]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad"}]: dispatch 2024-06-06T04:21:44.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:44 smithi045 ceph-mon[32299]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad"}]': finished 2024-06-06T04:21:44.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:44 smithi045 ceph-mon[32299]: osdmap e10: 2 total, 1 up, 2 in 2024-06-06T04:21:44.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:44 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T04:21:45.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:45 smithi154 ceph-mon[36428]: from='client.? 172.21.15.86:0/3137233304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:21:45.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:45 smithi045 ceph-mon[32299]: from='client.? 172.21.15.86:0/3137233304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:21:45.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:45 smithi086 ceph-mon[36053]: from='client.? 172.21.15.86:0/3137233304' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:21:46.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:46 smithi154 ceph-mon[36428]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:46.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:46 smithi045 ceph-mon[32299]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:46.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:46 smithi086 ceph-mon[36053]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:48.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:48 smithi086 ceph-mon[36053]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:48.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:48 smithi154 ceph-mon[36428]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:48.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:48 smithi045 ceph-mon[32299]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:50.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:50 smithi154 ceph-mon[36428]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:50.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:50 smithi045 ceph-mon[32299]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:50.584 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:50 smithi086 ceph-mon[36053]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:51.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:51 smithi154 ceph-mon[36428]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:51.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:51 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-06T04:21:51.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:51 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:51.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:51 smithi154 ceph-mon[36428]: Deploying daemon osd.1 on smithi086 2024-06-06T04:21:51.474 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:51 smithi086 ceph-mon[36053]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:51.475 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:51 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-06T04:21:51.475 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:51 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:51.475 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:51 smithi086 ceph-mon[36053]: Deploying daemon osd.1 on smithi086 2024-06-06T04:21:51.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:51 smithi045 ceph-mon[32299]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:51.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:51 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-06T04:21:51.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:51 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:51.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:51 smithi045 ceph-mon[32299]: Deploying daemon osd.1 on smithi086 2024-06-06T04:21:53.571 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:53 smithi086 ceph-mon[36053]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:53.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:53 smithi154 ceph-mon[36428]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:53.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:53 smithi045 ceph-mon[32299]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:55.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:55 smithi086 ceph-mon[36053]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:55.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:55 smithi154 ceph-mon[36428]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:55.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:55 smithi045 ceph-mon[32299]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:57.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:56 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:57.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:56 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:57.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:56 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:57.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:56 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:57.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:56 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:57.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:56 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:57.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:56 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:57.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:56 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:57.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:56 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:21:58.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:57 smithi154 ceph-mon[36428]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:58.196 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:57 smithi086 ceph-mon[36053]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:58.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:57 smithi045 ceph-mon[32299]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:21:59.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:59 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:59.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:59 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:59.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:59 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:59.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:59 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:59.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:21:59 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:59.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:59 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:59.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:59 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:59.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:59 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:59.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:59 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:59.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:21:59 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:59.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:59 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:59.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:59 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:59.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:59 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:21:59.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:59 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:21:59.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:21:59 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:21:59.499 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 1 on host 'smithi086' 2024-06-06T04:21:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:59.498+0000 7f82967fc700 1 -- 172.21.15.86:0/1511234068 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f82a00febc0 con 0x7f828c0a2c10 2024-06-06T04:21:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:59.501+0000 7f82a7808700 1 -- 172.21.15.86:0/1511234068 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f828c0a2c10 msgr2=0x7f828c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:59.501+0000 7f82a7808700 1 --2- 172.21.15.86:0/1511234068 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f828c0a2c10 0x7f828c0a50d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f829c000c00 tx=0x7f829c005fb0 comp rx=0 tx=0).stop 2024-06-06T04:21:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:59.501+0000 7f82a7808700 1 -- 172.21.15.86:0/1511234068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a0102fb0 msgr2=0x7f82a0103f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:21:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:59.501+0000 7f82a7808700 1 --2- 172.21.15.86:0/1511234068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a0102fb0 0x7f82a0103f30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f829800b970 tx=0x7f829800bc80 comp rx=0 tx=0).stop 2024-06-06T04:21:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:59.501+0000 7f82a7808700 1 -- 172.21.15.86:0/1511234068 shutdown_connections 2024-06-06T04:21:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:59.501+0000 7f82a7808700 1 --2- 172.21.15.86:0/1511234068 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f828c0a2c10 0x7f828c0a50d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:59.501+0000 7f82a7808700 1 --2- 172.21.15.86:0/1511234068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a0102fb0 0x7f82a0103f30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:59.501+0000 7f82a7808700 1 --2- 172.21.15.86:0/1511234068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a0101410 0x7f82a01053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:59.501+0000 7f82a7808700 1 --2- 172.21.15.86:0/1511234068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a0102610 0x7f82a01039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:21:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:59.501+0000 7f82a7808700 1 -- 172.21.15.86:0/1511234068 >> 172.21.15.86:0/1511234068 conn(0x7f82a00fc9f0 msgr2=0x7f82a00fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:21:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:59.501+0000 7f82a7808700 1 -- 172.21.15.86:0/1511234068 shutdown_connections 2024-06-06T04:21:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:21:59.501+0000 7f82a7808700 1 -- 172.21.15.86:0/1511234068 wait complete. 2024-06-06T04:22:00.056 DEBUG:teuthology.orchestra.run.smithi086:osd.1> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.1.service 2024-06-06T04:22:00.058 INFO:tasks.cephadm:Deploying osd.2 on smithi154 with /dev/vg_nvme/lv_4... 2024-06-06T04:22:00.058 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-06T04:22:00.304 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T04:22:00.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:00 smithi086 ceph-mon[36053]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:22:00.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:00.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:00.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:22:00.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:00 smithi086 ceph-mon[36053]: from='osd.1 [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T04:22:00.346 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:00 smithi086 ceph-mon[36053]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T04:22:00.426 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:00 smithi154 ceph-mon[36428]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:22:00.426 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:00.426 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:00.426 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:22:00.426 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:00 smithi154 ceph-mon[36428]: from='osd.1 [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T04:22:00.426 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:00 smithi154 ceph-mon[36428]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T04:22:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:00 smithi045 ceph-mon[32299]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:22:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:22:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:00 smithi045 ceph-mon[32299]: from='osd.1 [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T04:22:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:00 smithi045 ceph-mon[32299]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T04:22:01.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:01 smithi154 ceph-mon[36428]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:22:01.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:01 smithi154 ceph-mon[36428]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-06T04:22:01.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:01 smithi154 ceph-mon[36428]: from='osd.1 [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:22:01.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:01 smithi154 ceph-mon[36428]: osdmap e11: 2 total, 1 up, 2 in 2024-06-06T04:22:01.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:01 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T04:22:01.536 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:01 smithi154 ceph-mon[36428]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:22:01.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:01 smithi086 ceph-mon[36053]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:22:01.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:01 smithi086 ceph-mon[36053]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-06T04:22:01.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:01 smithi086 ceph-mon[36053]: from='osd.1 [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:22:01.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:01 smithi086 ceph-mon[36053]: osdmap e11: 2 total, 1 up, 2 in 2024-06-06T04:22:01.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:01 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T04:22:01.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:01 smithi086 ceph-mon[36053]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:22:01.610 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:22:01 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-1[40608]: 2024-06-06T04:22:01.465+0000 7ff41999b700 -1 osd.1 0 waiting for initial osdmap 2024-06-06T04:22:01.610 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:22:01 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-1[40608]: 2024-06-06T04:22:01.468+0000 7ff41578e700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T04:22:01.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:01 smithi045 ceph-mon[32299]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:22:01.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:01 smithi045 ceph-mon[32299]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-06T04:22:01.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:01 smithi045 ceph-mon[32299]: from='osd.1 [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:22:01.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:01 smithi045 ceph-mon[32299]: osdmap e11: 2 total, 1 up, 2 in 2024-06-06T04:22:01.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:01 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T04:22:01.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:01 smithi045 ceph-mon[32299]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:22:02.165 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-06T04:22:02.166 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-06T04:22:02.166 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-06-06T04:22:02.166 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-06-06T04:22:02.166 INFO:teuthology.orchestra.run.smithi154.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126449 s, 829 MB/s 2024-06-06T04:22:02.166 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-06-06T04:22:02.719 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:02 smithi154 ceph-mon[36428]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-06-06T04:22:02.719 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:02 smithi154 ceph-mon[36428]: osdmap e12: 2 total, 1 up, 2 in 2024-06-06T04:22:02.719 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T04:22:02.719 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T04:22:02.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:02 smithi045 ceph-mon[32299]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-06-06T04:22:02.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:02 smithi045 ceph-mon[32299]: osdmap e12: 2 total, 1 up, 2 in 2024-06-06T04:22:02.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T04:22:02.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T04:22:02.746 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi154:vg_nvme/lv_4 2024-06-06T04:22:02.858 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:02 smithi086 ceph-mon[36053]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-06-06T04:22:02.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:02 smithi086 ceph-mon[36053]: osdmap e12: 2 total, 1 up, 2 in 2024-06-06T04:22:02.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T04:22:02.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T04:22:02.984 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T04:22:03.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:03 smithi154 ceph-mon[36428]: purged_snaps scrub starts 2024-06-06T04:22:03.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:03 smithi154 ceph-mon[36428]: purged_snaps scrub ok 2024-06-06T04:22:03.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:03 smithi154 ceph-mon[36428]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:22:03.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:03 smithi154 ceph-mon[36428]: osd.1 [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287] boot 2024-06-06T04:22:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:03 smithi154 ceph-mon[36428]: osdmap e13: 2 total, 2 up, 2 in 2024-06-06T04:22:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T04:22:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:03 smithi154 ceph-mon[36428]: Detected new or changed devices on smithi086 2024-06-06T04:22:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:22:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:03 smithi154 ceph-mon[36428]: Adjusting osd_memory_target on smithi086 to 17088M 2024-06-06T04:22:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:22:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:03 smithi045 ceph-mon[32299]: purged_snaps scrub starts 2024-06-06T04:22:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:03 smithi045 ceph-mon[32299]: purged_snaps scrub ok 2024-06-06T04:22:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:03 smithi045 ceph-mon[32299]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:22:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:03 smithi045 ceph-mon[32299]: osd.1 [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287] boot 2024-06-06T04:22:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:03 smithi045 ceph-mon[32299]: osdmap e13: 2 total, 2 up, 2 in 2024-06-06T04:22:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T04:22:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:03 smithi045 ceph-mon[32299]: Detected new or changed devices on smithi086 2024-06-06T04:22:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:22:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:03 smithi045 ceph-mon[32299]: Adjusting osd_memory_target on smithi086 to 17088M 2024-06-06T04:22:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:22:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:03 smithi086 ceph-mon[36053]: purged_snaps scrub starts 2024-06-06T04:22:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:03 smithi086 ceph-mon[36053]: purged_snaps scrub ok 2024-06-06T04:22:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:03 smithi086 ceph-mon[36053]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T04:22:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:03 smithi086 ceph-mon[36053]: osd.1 [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287] boot 2024-06-06T04:22:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:03 smithi086 ceph-mon[36053]: osdmap e13: 2 total, 2 up, 2 in 2024-06-06T04:22:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T04:22:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:03 smithi086 ceph-mon[36053]: Detected new or changed devices on smithi086 2024-06-06T04:22:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:03.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:22:03.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:03 smithi086 ceph-mon[36053]: Adjusting osd_memory_target on smithi086 to 17088M 2024-06-06T04:22:03.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:03.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:03.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:22:03.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:04.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.512+0000 7fed5dcd1700 1 -- 172.21.15.154:0/1264103065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed58101420 msgr2=0x7fed58101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:22:04.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.512+0000 7fed5dcd1700 1 --2- 172.21.15.154:0/1264103065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed58101420 0x7fed58101840 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fed48009990 tx=0x7fed48009ca0 comp rx=0 tx=0).stop 2024-06-06T04:22:04.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.512+0000 7fed5dcd1700 1 -- 172.21.15.154:0/1264103065 shutdown_connections 2024-06-06T04:22:04.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.512+0000 7fed5dcd1700 1 --2- 172.21.15.154:0/1264103065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed58102fc0 0x7fed58109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:04.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.512+0000 7fed5dcd1700 1 --2- 172.21.15.154:0/1264103065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed58101420 0x7fed58101840 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:04.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.512+0000 7fed5dcd1700 1 --2- 172.21.15.154:0/1264103065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed58102620 0x7fed58102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:04.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.512+0000 7fed5dcd1700 1 -- 172.21.15.154:0/1264103065 >> 172.21.15.154:0/1264103065 conn(0x7fed580fc9e0 msgr2=0x7fed580fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:22:04.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.512+0000 7fed5dcd1700 1 -- 172.21.15.154:0/1264103065 shutdown_connections 2024-06-06T04:22:04.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.512+0000 7fed5dcd1700 1 -- 172.21.15.154:0/1264103065 wait complete. 2024-06-06T04:22:04.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.513+0000 7fed5dcd1700 1 Processor -- start 2024-06-06T04:22:04.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.513+0000 7fed5dcd1700 1 -- start start 2024-06-06T04:22:04.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.513+0000 7fed5dcd1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed58101420 0x7fed58198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:22:04.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.513+0000 7fed5dcd1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed58102620 0x7fed58198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:22:04.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.513+0000 7fed5dcd1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed58102fc0 0x7fed5819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:22:04.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.513+0000 7fed5dcd1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fed5810cf70 con 0x7fed58102fc0 2024-06-06T04:22:04.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.513+0000 7fed5dcd1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed5810cdf0 con 0x7fed58101420 2024-06-06T04:22:04.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.513+0000 7fed5dcd1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fed5810d0f0 con 0x7fed58102620 2024-06-06T04:22:04.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.513+0000 7fed4ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed58102620 0x7fed58198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:22:04.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.513+0000 7fed4ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed58102620 0x7fed58198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59506/0 (socket says 172.21.15.154:59506) 2024-06-06T04:22:04.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.513+0000 7fed4ffff700 1 -- 172.21.15.154:0/297597254 learned_addr learned my addr 172.21.15.154:0/297597254 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-06-06T04:22:04.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.513+0000 7fed57fff700 1 --2- 172.21.15.154:0/297597254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed58102fc0 0x7fed5819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:22:04.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.513+0000 7fed577fe700 1 --2- 172.21.15.154:0/297597254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed58101420 0x7fed58198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:22:04.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.514+0000 7fed4ffff700 1 -- 172.21.15.154:0/297597254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed58101420 msgr2=0x7fed58198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:22:04.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.514+0000 7fed4ffff700 1 --2- 172.21.15.154:0/297597254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed58101420 0x7fed58198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:04.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.514+0000 7fed4ffff700 1 -- 172.21.15.154:0/297597254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed58102fc0 msgr2=0x7fed5819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:22:04.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.514+0000 7fed4ffff700 1 --2- 172.21.15.154:0/297597254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed58102fc0 0x7fed5819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:04.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.514+0000 7fed4ffff700 1 -- 172.21.15.154:0/297597254 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed581a0370 con 0x7fed58102620 2024-06-06T04:22:04.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.514+0000 7fed577fe700 1 --2- 172.21.15.154:0/297597254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed58101420 0x7fed58198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:22:04.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.514+0000 7fed4ffff700 1 --2- 172.21.15.154:0/297597254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed58102620 0x7fed58198c50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fed4000d980 tx=0x7fed4000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:22:04.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.514+0000 7fed557fa700 1 -- 172.21.15.154:0/297597254 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40016070 con 0x7fed58102620 2024-06-06T04:22:04.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.514+0000 7fed557fa700 1 -- 172.21.15.154:0/297597254 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed4000de70 con 0x7fed58102620 2024-06-06T04:22:04.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.514+0000 7fed557fa700 1 -- 172.21.15.154:0/297597254 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4000f3c0 con 0x7fed58102620 2024-06-06T04:22:04.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.514+0000 7fed5dcd1700 1 -- 172.21.15.154:0/297597254 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed581a0660 con 0x7fed58102620 2024-06-06T04:22:04.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.514+0000 7fed5dcd1700 1 -- 172.21.15.154:0/297597254 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed581a0bd0 con 0x7fed58102620 2024-06-06T04:22:04.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.515+0000 7fed5dcd1700 1 -- 172.21.15.154:0/297597254 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed58069180 con 0x7fed58102620 2024-06-06T04:22:04.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.516+0000 7fed557fa700 1 -- 172.21.15.154:0/297597254 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fed40019070 con 0x7fed58102620 2024-06-06T04:22:04.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.517+0000 7fed557fa700 1 --2- 172.21.15.154:0/297597254 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fed440a2b90 0x7fed440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:22:04.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.517+0000 7fed577fe700 1 --2- 172.21.15.154:0/297597254 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fed440a2b90 0x7fed440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:22:04.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.517+0000 7fed557fa700 1 -- 172.21.15.154:0/297597254 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fed400cd310 con 0x7fed58102620 2024-06-06T04:22:04.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.518+0000 7fed577fe700 1 --2- 172.21.15.154:0/297597254 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fed440a2b90 0x7fed440a5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fed4800b5c0 tx=0x7fed48005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:22:04.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.522+0000 7fed557fa700 1 -- 172.21.15.154:0/297597254 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fed4009bac0 con 0x7fed58102620 2024-06-06T04:22:04.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:04 smithi154 ceph-mon[36428]: osdmap e14: 2 total, 2 up, 2 in 2024-06-06T04:22:04.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:04.738+0000 7fed5dcd1700 1 -- 172.21.15.154:0/297597254 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fed5806b270 con 0x7fed440a2b90 2024-06-06T04:22:04.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:04 smithi086 ceph-mon[36053]: osdmap e14: 2 total, 2 up, 2 in 2024-06-06T04:22:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:04 smithi045 ceph-mon[32299]: osdmap e14: 2 total, 2 up, 2 in 2024-06-06T04:22:05.674 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:05 smithi154 ceph-mon[36428]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:05.674 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:05 smithi154 ceph-mon[36428]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:22:05.674 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:22:05.674 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:22:05.674 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:05.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:05 smithi086 ceph-mon[36053]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:05.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:05 smithi086 ceph-mon[36053]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:22:05.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:22:05.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:22:05.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:05 smithi045 ceph-mon[32299]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:05 smithi045 ceph-mon[32299]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:22:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:22:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:22:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:07.858 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:07 smithi086 ceph-mon[36053]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:07.861 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:07 smithi154 ceph-mon[36428]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:07.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:07 smithi045 ceph-mon[32299]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:08.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:08 smithi086 ceph-mon[36053]: from='client.? 172.21.15.154:0/1377011786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb20565f-f9ae-47e3-9604-c69f737cff4c"}]: dispatch 2024-06-06T04:22:08.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:08 smithi086 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb20565f-f9ae-47e3-9604-c69f737cff4c"}]: dispatch 2024-06-06T04:22:08.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:08 smithi086 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb20565f-f9ae-47e3-9604-c69f737cff4c"}]': finished 2024-06-06T04:22:08.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:08 smithi086 ceph-mon[36053]: osdmap e15: 3 total, 2 up, 3 in 2024-06-06T04:22:08.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:08 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:08.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:08 smithi086 ceph-mon[36053]: from='client.? 172.21.15.154:0/2245372018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:22:08.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:08 smithi154 ceph-mon[36428]: from='client.? 172.21.15.154:0/1377011786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb20565f-f9ae-47e3-9604-c69f737cff4c"}]: dispatch 2024-06-06T04:22:08.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:08 smithi154 ceph-mon[36428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb20565f-f9ae-47e3-9604-c69f737cff4c"}]: dispatch 2024-06-06T04:22:08.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:08 smithi154 ceph-mon[36428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb20565f-f9ae-47e3-9604-c69f737cff4c"}]': finished 2024-06-06T04:22:08.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:08 smithi154 ceph-mon[36428]: osdmap e15: 3 total, 2 up, 3 in 2024-06-06T04:22:08.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:08 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:08.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:08 smithi154 ceph-mon[36428]: from='client.? 172.21.15.154:0/2245372018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:22:08.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:08 smithi045 ceph-mon[32299]: from='client.? 172.21.15.154:0/1377011786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb20565f-f9ae-47e3-9604-c69f737cff4c"}]: dispatch 2024-06-06T04:22:08.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:08 smithi045 ceph-mon[32299]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb20565f-f9ae-47e3-9604-c69f737cff4c"}]: dispatch 2024-06-06T04:22:08.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:08 smithi045 ceph-mon[32299]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb20565f-f9ae-47e3-9604-c69f737cff4c"}]': finished 2024-06-06T04:22:08.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:08 smithi045 ceph-mon[32299]: osdmap e15: 3 total, 2 up, 3 in 2024-06-06T04:22:08.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:08 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:08.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:08 smithi045 ceph-mon[32299]: from='client.? 172.21.15.154:0/2245372018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:22:09.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:09 smithi086 ceph-mon[36053]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:09.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:09 smithi154 ceph-mon[36428]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:09.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:09 smithi045 ceph-mon[32299]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:11.727 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:11 smithi154 ceph-mon[36428]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:11 smithi086 ceph-mon[36053]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:11.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:11 smithi045 ceph-mon[32299]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:13.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:13 smithi154 ceph-mon[36428]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:13.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:13 smithi086 ceph-mon[36053]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:13.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:13 smithi045 ceph-mon[32299]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:14.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-06T04:22:14.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:14.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-06T04:22:14.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:14.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-06T04:22:14.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:15.574 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:15 smithi154 ceph-mon[36428]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:15.574 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:15 smithi154 ceph-mon[36428]: Deploying daemon osd.2 on smithi154 2024-06-06T04:22:15.858 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:15 smithi086 ceph-mon[36053]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:15.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:15 smithi086 ceph-mon[36053]: Deploying daemon osd.2 on smithi154 2024-06-06T04:22:15.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:15 smithi045 ceph-mon[32299]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:15.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:15 smithi045 ceph-mon[32299]: Deploying daemon osd.2 on smithi154 2024-06-06T04:22:17.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:17 smithi154 ceph-mon[36428]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:17.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:17 smithi086 ceph-mon[36053]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:17.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:17 smithi045 ceph-mon[32299]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:19.763 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:19 smithi154 ceph-mon[36428]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:19.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:19 smithi086 ceph-mon[36053]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:19.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:19 smithi045 ceph-mon[32299]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:21.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:20 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:21.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:20 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:21.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:20 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:22:21.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:20 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:21.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:20 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:21.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:20 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:22:21.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:20 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:21.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:20 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:21.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:20 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:22:22.081 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:21 smithi154 ceph-mon[36428]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:22.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:21 smithi086 ceph-mon[36053]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:22.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:21 smithi045 ceph-mon[32299]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:23.489 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:23 smithi154 ceph-mon[36428]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:23.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:23 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:23.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:23 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:23.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:23 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:23.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:23 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:22:23.490 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:23 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:23.557 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 2 on host 'smithi154' 2024-06-06T04:22:23.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:23.556+0000 7fed557fa700 1 -- 172.21.15.154:0/297597254 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fed5806b270 con 0x7fed440a2b90 2024-06-06T04:22:23.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:23.558+0000 7fed5dcd1700 1 -- 172.21.15.154:0/297597254 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fed440a2b90 msgr2=0x7fed440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:22:23.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:23.558+0000 7fed5dcd1700 1 --2- 172.21.15.154:0/297597254 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fed440a2b90 0x7fed440a5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fed4800b5c0 tx=0x7fed48005fd0 comp rx=0 tx=0).stop 2024-06-06T04:22:23.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:23.558+0000 7fed5dcd1700 1 -- 172.21.15.154:0/297597254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed58102620 msgr2=0x7fed58198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:22:23.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:23.558+0000 7fed5dcd1700 1 --2- 172.21.15.154:0/297597254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed58102620 0x7fed58198c50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fed4000d980 tx=0x7fed4000dc90 comp rx=0 tx=0).stop 2024-06-06T04:22:23.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:23.558+0000 7fed5dcd1700 1 -- 172.21.15.154:0/297597254 shutdown_connections 2024-06-06T04:22:23.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:23.558+0000 7fed5dcd1700 1 --2- 172.21.15.154:0/297597254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed58102fc0 0x7fed5819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:23.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:23.558+0000 7fed5dcd1700 1 --2- 172.21.15.154:0/297597254 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fed440a2b90 0x7fed440a5050 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:23.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:23.558+0000 7fed5dcd1700 1 --2- 172.21.15.154:0/297597254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed58101420 0x7fed58198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:23.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:23.558+0000 7fed5dcd1700 1 --2- 172.21.15.154:0/297597254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed58102620 0x7fed58198c50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:23.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:23.558+0000 7fed5dcd1700 1 -- 172.21.15.154:0/297597254 >> 172.21.15.154:0/297597254 conn(0x7fed580fc9e0 msgr2=0x7fed5806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:22:23.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:23.558+0000 7fed5dcd1700 1 -- 172.21.15.154:0/297597254 shutdown_connections 2024-06-06T04:22:23.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:22:23.558+0000 7fed5dcd1700 1 -- 172.21.15.154:0/297597254 wait complete. 2024-06-06T04:22:23.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:23 smithi045 ceph-mon[32299]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:23.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:23 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:23.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:23 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:23.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:23 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:23.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:23 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:22:23.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:23 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:23.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:23 smithi086 ceph-mon[36053]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:23.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:23 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:23.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:23 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:23.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:23 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:23.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:23 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:22:23.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:23 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:24.203 DEBUG:teuthology.orchestra.run.smithi154:osd.2> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.2.service 2024-06-06T04:22:24.206 INFO:tasks.cephadm:Deploying osd.3 on smithi045 with /dev/vg_nvme/lv_3... 2024-06-06T04:22:24.206 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-06T04:22:24.463 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:22:24.795 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:24 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:24.795 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:24 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:24.795 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:24 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:22:24.795 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:24 smithi045 ceph-mon[32299]: from='osd.2 [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T04:22:24.795 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:24 smithi045 ceph-mon[32299]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T04:22:24.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:24 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:24.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:24 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:24.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:24 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:22:24.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:24 smithi086 ceph-mon[36053]: from='osd.2 [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T04:22:24.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:24 smithi086 ceph-mon[36053]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T04:22:24.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:24 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:24.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:24 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:24.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:24 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:22:24.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:24 smithi154 ceph-mon[36428]: from='osd.2 [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T04:22:24.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:24 smithi154 ceph-mon[36428]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T04:22:25.647 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:25 smithi154 ceph-mon[36428]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:25.647 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:25 smithi154 ceph-mon[36428]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-06T04:22:25.647 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:25 smithi154 ceph-mon[36428]: osdmap e16: 3 total, 2 up, 3 in 2024-06-06T04:22:25.647 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:25 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:25.647 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:25 smithi154 ceph-mon[36428]: from='osd.2 [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:22:25.647 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:25 smithi154 ceph-mon[36428]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:22:25.648 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:22:25 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-2[41536]: 2024-06-06T04:22:25.545+0000 7fac1ef7e700 -1 osd.2 0 waiting for initial osdmap 2024-06-06T04:22:25.648 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:22:25 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-2[41536]: 2024-06-06T04:22:25.549+0000 7fac1ad71700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T04:22:25.791 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:25 smithi045 ceph-mon[32299]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:25.791 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:25 smithi045 ceph-mon[32299]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-06T04:22:25.791 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:25 smithi045 ceph-mon[32299]: osdmap e16: 3 total, 2 up, 3 in 2024-06-06T04:22:25.791 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:25 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:25.791 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:25 smithi045 ceph-mon[32299]: from='osd.2 [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:22:25.791 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:25 smithi045 ceph-mon[32299]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:22:25.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:25 smithi086 ceph-mon[36053]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:25.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:25 smithi086 ceph-mon[36053]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-06T04:22:25.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:25 smithi086 ceph-mon[36053]: osdmap e16: 3 total, 2 up, 3 in 2024-06-06T04:22:25.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:25 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:25.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:25 smithi086 ceph-mon[36053]: from='osd.2 [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:22:25.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:25 smithi086 ceph-mon[36053]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:22:26.384 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-06T04:22:26.384 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-06T04:22:26.384 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-06-06T04:22:26.384 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-06-06T04:22:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00839373 s, 1.2 GB/s 2024-06-06T04:22:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-06-06T04:22:26.772 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:26 smithi154 ceph-mon[36428]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-06-06T04:22:26.772 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:26 smithi154 ceph-mon[36428]: osdmap e17: 3 total, 2 up, 3 in 2024-06-06T04:22:26.772 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:26.773 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:26.782 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:26 smithi045 ceph-mon[32299]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-06-06T04:22:26.782 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:26 smithi045 ceph-mon[32299]: osdmap e17: 3 total, 2 up, 3 in 2024-06-06T04:22:26.782 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:26.782 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:26.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:26 smithi086 ceph-mon[36053]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-06-06T04:22:26.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:26 smithi086 ceph-mon[36053]: osdmap e17: 3 total, 2 up, 3 in 2024-06-06T04:22:26.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:26.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:27.054 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_3 2024-06-06T04:22:27.319 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:22:27.704 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:27 smithi045 ceph-mon[32299]: purged_snaps scrub starts 2024-06-06T04:22:27.704 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:27 smithi045 ceph-mon[32299]: purged_snaps scrub ok 2024-06-06T04:22:27.704 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:27 smithi045 ceph-mon[32299]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:27.704 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:27 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:27.704 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:27 smithi045 ceph-mon[32299]: osd.2 [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039] boot 2024-06-06T04:22:27.704 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:27 smithi045 ceph-mon[32299]: osdmap e18: 3 total, 3 up, 3 in 2024-06-06T04:22:27.704 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:27 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:27.704 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:27 smithi045 ceph-mon[32299]: Detected new or changed devices on smithi154 2024-06-06T04:22:27.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:27 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:27.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:27 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:27.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:27 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:22:27.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:27 smithi045 ceph-mon[32299]: Adjusting osd_memory_target on smithi154 to 17088M 2024-06-06T04:22:27.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:27 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:27.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:27 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:27.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:27 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:22:27.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:27 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:27 smithi086 ceph-mon[36053]: purged_snaps scrub starts 2024-06-06T04:22:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:27 smithi086 ceph-mon[36053]: purged_snaps scrub ok 2024-06-06T04:22:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:27 smithi086 ceph-mon[36053]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:27 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:27 smithi086 ceph-mon[36053]: osd.2 [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039] boot 2024-06-06T04:22:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:27 smithi086 ceph-mon[36053]: osdmap e18: 3 total, 3 up, 3 in 2024-06-06T04:22:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:27 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:27.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:27 smithi086 ceph-mon[36053]: Detected new or changed devices on smithi154 2024-06-06T04:22:27.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:27 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:27.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:27 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:27.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:27 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:22:27.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:27 smithi086 ceph-mon[36053]: Adjusting osd_memory_target on smithi154 to 17088M 2024-06-06T04:22:27.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:27 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:27.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:27 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:27.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:27 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:22:27.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:27 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:27.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:27 smithi154 ceph-mon[36428]: purged_snaps scrub starts 2024-06-06T04:22:27.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:27 smithi154 ceph-mon[36428]: purged_snaps scrub ok 2024-06-06T04:22:27.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:27 smithi154 ceph-mon[36428]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-06T04:22:27.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:27 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:27.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:27 smithi154 ceph-mon[36428]: osd.2 [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039] boot 2024-06-06T04:22:27.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:27 smithi154 ceph-mon[36428]: osdmap e18: 3 total, 3 up, 3 in 2024-06-06T04:22:27.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:27 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T04:22:27.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:27 smithi154 ceph-mon[36428]: Detected new or changed devices on smithi154 2024-06-06T04:22:27.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:27 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:27.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:27 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:27.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:27 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:22:27.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:27 smithi154 ceph-mon[36428]: Adjusting osd_memory_target on smithi154 to 17088M 2024-06-06T04:22:27.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:27 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:27.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:27 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:27.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:27 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:22:27.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:27 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.976+0000 7f488f83b700 1 -- 172.21.15.45:0/664197012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48881013f0 msgr2=0x7f4888101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:22:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.976+0000 7f488f83b700 1 --2- 172.21.15.45:0/664197012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48881013f0 0x7f4888101810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f48840099c0 tx=0x7f4884009cd0 comp rx=0 tx=0).stop 2024-06-06T04:22:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.976+0000 7f488f83b700 1 -- 172.21.15.45:0/664197012 shutdown_connections 2024-06-06T04:22:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.976+0000 7f488f83b700 1 --2- 172.21.15.45:0/664197012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4888102f90 0x7f4888109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.976+0000 7f488f83b700 1 --2- 172.21.15.45:0/664197012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48881013f0 0x7f4888101810 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.976+0000 7f488f83b700 1 --2- 172.21.15.45:0/664197012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48881025f0 0x7f4888102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.976+0000 7f488f83b700 1 -- 172.21.15.45:0/664197012 >> 172.21.15.45:0/664197012 conn(0x7f48880fc990 msgr2=0x7f48880fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:22:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.976+0000 7f488f83b700 1 -- 172.21.15.45:0/664197012 shutdown_connections 2024-06-06T04:22:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.976+0000 7f488f83b700 1 -- 172.21.15.45:0/664197012 wait complete. 2024-06-06T04:22:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.977+0000 7f488f83b700 1 Processor -- start 2024-06-06T04:22:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.977+0000 7f488f83b700 1 -- start start 2024-06-06T04:22:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.977+0000 7f488f83b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48881013f0 0x7f488806acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:22:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.977+0000 7f488f83b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48881025f0 0x7f488819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:22:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.977+0000 7f488f83b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4888102f90 0x7f488819fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:22:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.977+0000 7f488f83b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f488810ce80 con 0x7f48881013f0 2024-06-06T04:22:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.977+0000 7f488f83b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f488810cd00 con 0x7f48881025f0 2024-06-06T04:22:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.977+0000 7f488f83b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f488810d000 con 0x7f4888102f90 2024-06-06T04:22:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.977+0000 7f488d5d7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48881013f0 0x7f488806acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:22:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.978+0000 7f488d5d7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48881013f0 0x7f488806acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38906/0 (socket says 172.21.15.45:38906) 2024-06-06T04:22:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.978+0000 7f488d5d7700 1 -- 172.21.15.45:0/4260723490 learned_addr learned my addr 172.21.15.45:0/4260723490 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:22:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.978+0000 7f488cdd6700 1 --2- 172.21.15.45:0/4260723490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48881025f0 0x7f488819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:22:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.978+0000 7f488ddd8700 1 --2- 172.21.15.45:0/4260723490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4888102f90 0x7f488819fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:22:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.978+0000 7f488d5d7700 1 -- 172.21.15.45:0/4260723490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4888102f90 msgr2=0x7f488819fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:22:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.978+0000 7f488d5d7700 1 --2- 172.21.15.45:0/4260723490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4888102f90 0x7f488819fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.978+0000 7f488d5d7700 1 -- 172.21.15.45:0/4260723490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48881025f0 msgr2=0x7f488819d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:22:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.978+0000 7f488d5d7700 1 --2- 172.21.15.45:0/4260723490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48881025f0 0x7f488819d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.978+0000 7f488d5d7700 1 -- 172.21.15.45:0/4260723490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48881a02e0 con 0x7f48881013f0 2024-06-06T04:22:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.978+0000 7f488ddd8700 1 --2- 172.21.15.45:0/4260723490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4888102f90 0x7f488819fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:22:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.978+0000 7f488d5d7700 1 --2- 172.21.15.45:0/4260723490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48881013f0 0x7f488806acd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f48840098e0 tx=0x7f48840057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:22:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.978+0000 7f487e7fc700 1 -- 172.21.15.45:0/4260723490 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884027070 con 0x7f48881013f0 2024-06-06T04:22:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.978+0000 7f488f83b700 1 -- 172.21.15.45:0/4260723490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48881a0570 con 0x7f48881013f0 2024-06-06T04:22:28.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.978+0000 7f487e7fc700 1 -- 172.21.15.45:0/4260723490 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4884004050 con 0x7f48881013f0 2024-06-06T04:22:28.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.978+0000 7f487e7fc700 1 -- 172.21.15.45:0/4260723490 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884022d60 con 0x7f48881013f0 2024-06-06T04:22:28.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.978+0000 7f488f83b700 1 -- 172.21.15.45:0/4260723490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48881a0a20 con 0x7f48881013f0 2024-06-06T04:22:28.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.979+0000 7f488f83b700 1 -- 172.21.15.45:0/4260723490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4888069180 con 0x7f48881013f0 2024-06-06T04:22:28.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.981+0000 7f487e7fc700 1 -- 172.21.15.45:0/4260723490 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f488402a070 con 0x7f48881013f0 2024-06-06T04:22:28.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.981+0000 7f487e7fc700 1 --2- 172.21.15.45:0/4260723490 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f48740a2b90 0x7f48740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:22:28.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.981+0000 7f487e7fc700 1 -- 172.21.15.45:0/4260723490 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7f48840cf360 con 0x7f48881013f0 2024-06-06T04:22:28.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.981+0000 7f488cdd6700 1 --2- 172.21.15.45:0/4260723490 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f48740a2b90 0x7f48740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:22:28.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.982+0000 7f488cdd6700 1 --2- 172.21.15.45:0/4260723490 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f48740a2b90 0x7f48740a5050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f488806b250 tx=0x7f487800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:22:28.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:28.985+0000 7f487e7fc700 1 -- 172.21.15.45:0/4260723490 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4884013070 con 0x7f48881013f0 2024-06-06T04:22:29.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:28 smithi086 ceph-mon[36053]: osdmap e19: 3 total, 3 up, 3 in 2024-06-06T04:22:29.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:28 smithi154 ceph-mon[36428]: osdmap e19: 3 total, 3 up, 3 in 2024-06-06T04:22:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:29.201+0000 7f488f83b700 1 -- 172.21.15.45:0/4260723490 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f48880628d0 con 0x7f48740a2b90 2024-06-06T04:22:29.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:28 smithi045 ceph-mon[32299]: osdmap e19: 3 total, 3 up, 3 in 2024-06-06T04:22:30.101 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:29 smithi045 ceph-mon[32299]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:30.101 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:29 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-06T04:22:30.101 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:29 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-06T04:22:30.101 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:29 smithi045 ceph-mon[32299]: osdmap e20: 3 total, 3 up, 3 in 2024-06-06T04:22:30.101 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:29 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T04:22:30.101 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:29 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:22:30.101 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:29 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:22:30.101 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:29 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:30.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:29 smithi086 ceph-mon[36053]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:30.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:29 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-06T04:22:30.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:29 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-06T04:22:30.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:29 smithi086 ceph-mon[36053]: osdmap e20: 3 total, 3 up, 3 in 2024-06-06T04:22:30.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:29 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T04:22:30.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:29 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:22:30.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:29 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:22:30.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:29 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:30.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:29 smithi154 ceph-mon[36428]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:30.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:29 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-06T04:22:30.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:29 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-06T04:22:30.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:29 smithi154 ceph-mon[36428]: osdmap e20: 3 total, 3 up, 3 in 2024-06-06T04:22:30.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:29 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T04:22:30.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:29 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:22:30.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:29 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:22:30.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:29 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:30.727 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:22:30 smithi045 sudo[51978]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T04:22:30.727 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:22:30 smithi045 sudo[51978]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T04:22:30.727 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:22:30 smithi045 sudo[51978]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T04:22:30.727 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:22:30 smithi045 sudo[51978]: pam_unix(sudo:session): session closed for user root 2024-06-06T04:22:30.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:30.862+0000 7f487e7fc700 1 -- 172.21.15.45:0/4260723490 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4884099780 con 0x7f48881013f0 2024-06-06T04:22:31.081 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:30 smithi045 ceph-mon[32299]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:22:31.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:30 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-06T04:22:31.082 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:30 smithi045 ceph-mon[32299]: osdmap e21: 3 total, 3 up, 3 in 2024-06-06T04:22:31.082 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:22:30 smithi045 sudo[51995]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T04:22:31.082 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:22:30 smithi045 sudo[51995]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T04:22:31.082 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:22:30 smithi045 sudo[51995]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T04:22:31.082 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:22:30 smithi045 sudo[51995]: pam_unix(sudo:session): session closed for user root 2024-06-06T04:22:31.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:30 smithi086 ceph-mon[36053]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:22:31.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:30 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-06T04:22:31.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:30 smithi086 ceph-mon[36053]: osdmap e21: 3 total, 3 up, 3 in 2024-06-06T04:22:31.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:30 smithi154 ceph-mon[36428]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:22:31.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:30 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-06T04:22:31.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:30 smithi154 ceph-mon[36428]: osdmap e21: 3 total, 3 up, 3 in 2024-06-06T04:22:31.360 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:22:31 smithi086 sudo[44422]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T04:22:31.360 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:22:31 smithi086 sudo[44422]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T04:22:31.360 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:22:31 smithi086 sudo[44422]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T04:22:31.360 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:22:31 smithi086 sudo[44422]: pam_unix(sudo:session): session closed for user root 2024-06-06T04:22:31.859 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:22:31 smithi086 sudo[44425]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T04:22:31.859 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:22:31 smithi086 sudo[44425]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T04:22:31.859 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:22:31 smithi086 sudo[44425]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T04:22:31.859 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:22:31 smithi086 sudo[44425]: pam_unix(sudo:session): session closed for user root 2024-06-06T04:22:31.880 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:22:31 smithi154 sudo[45272]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T04:22:31.880 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:22:31 smithi154 sudo[45272]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T04:22:31.880 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:22:31 smithi154 sudo[45272]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T04:22:31.881 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:22:31 smithi154 sudo[45272]: pam_unix(sudo:session): session closed for user root 2024-06-06T04:22:31.881 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:31 smithi154 ceph-mon[36428]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:31.881 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:31 smithi154 ceph-mon[36428]: osdmap e22: 3 total, 3 up, 3 in 2024-06-06T04:22:31.881 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:31 smithi154 ceph-mon[36428]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-06T04:22:31.881 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:31 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:31.881 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:31 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:32.180 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:22:31 smithi154 sudo[45275]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T04:22:32.180 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:22:31 smithi154 sudo[45275]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T04:22:32.181 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:22:31 smithi154 sudo[45275]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T04:22:32.181 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:22:31 smithi154 sudo[45275]: pam_unix(sudo:session): session closed for user root 2024-06-06T04:22:32.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:31 smithi045 ceph-mon[32299]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:32.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:31 smithi045 ceph-mon[32299]: osdmap e22: 3 total, 3 up, 3 in 2024-06-06T04:22:32.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:31 smithi045 ceph-mon[32299]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-06T04:22:32.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:31 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:32.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:31 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:32.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:31 smithi086 ceph-mon[36053]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:32.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:31 smithi086 ceph-mon[36053]: osdmap e22: 3 total, 3 up, 3 in 2024-06-06T04:22:32.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:31 smithi086 ceph-mon[36053]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-06T04:22:32.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:31 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:32.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:31 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:32.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:32 smithi045 sudo[52057]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T04:22:32.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:32 smithi045 sudo[52057]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T04:22:32.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:32 smithi045 sudo[52057]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T04:22:32.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:32 smithi045 sudo[52057]: pam_unix(sudo:session): session closed for user root 2024-06-06T04:22:33.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:32 smithi045 sudo[52060]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T04:22:33.164 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:32 smithi045 sudo[52060]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T04:22:33.164 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:32 smithi045 sudo[52060]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T04:22:33.164 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:32 smithi045 sudo[52060]: pam_unix(sudo:session): session closed for user root 2024-06-06T04:22:33.298 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 sudo[45278]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T04:22:33.298 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 sudo[45278]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T04:22:33.298 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 sudo[45278]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T04:22:33.299 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 sudo[45278]: pam_unix(sudo:session): session closed for user root 2024-06-06T04:22:33.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:33 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:33.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:33 smithi045 ceph-mon[32299]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T04:22:33.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:33 smithi045 ceph-mon[32299]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T04:22:33.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:33 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T04:22:33.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:33 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3721220100' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1160799-7811-4326-9792-7e6c7d189c7b"}]: dispatch 2024-06-06T04:22:33.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:33 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:22:33.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:33 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:22:33.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:33 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3721220100' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1160799-7811-4326-9792-7e6c7d189c7b"}]': finished 2024-06-06T04:22:33.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:33 smithi045 ceph-mon[32299]: osdmap e23: 4 total, 3 up, 4 in 2024-06-06T04:22:33.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:33 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T04:22:33.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:33 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T04:22:33.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:33 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:22:33.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:33 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:22:33.554 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 sudo[45281]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T04:22:33.554 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 sudo[45281]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T04:22:33.555 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 sudo[45281]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T04:22:33.555 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 sudo[45281]: pam_unix(sudo:session): session closed for user root 2024-06-06T04:22:33.555 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:33.555 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 ceph-mon[36428]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T04:22:33.555 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 ceph-mon[36428]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T04:22:33.555 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T04:22:33.555 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3721220100' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1160799-7811-4326-9792-7e6c7d189c7b"}]: dispatch 2024-06-06T04:22:33.555 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:22:33.555 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:22:33.555 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3721220100' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1160799-7811-4326-9792-7e6c7d189c7b"}]': finished 2024-06-06T04:22:33.555 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 ceph-mon[36428]: osdmap e23: 4 total, 3 up, 4 in 2024-06-06T04:22:33.555 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T04:22:33.555 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T04:22:33.555 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:22:33.556 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:33 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:22:33.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 sudo[44428]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T04:22:33.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 sudo[44428]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T04:22:33.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 sudo[44428]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T04:22:33.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 sudo[44428]: pam_unix(sudo:session): session closed for user root 2024-06-06T04:22:33.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 sudo[44431]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T04:22:33.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 sudo[44431]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T04:22:33.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 sudo[44431]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T04:22:33.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 sudo[44431]: pam_unix(sudo:session): session closed for user root 2024-06-06T04:22:34.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:34.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 ceph-mon[36053]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T04:22:34.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 ceph-mon[36053]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T04:22:34.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T04:22:34.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3721220100' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1160799-7811-4326-9792-7e6c7d189c7b"}]: dispatch 2024-06-06T04:22:34.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:22:34.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:22:34.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3721220100' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1160799-7811-4326-9792-7e6c7d189c7b"}]': finished 2024-06-06T04:22:34.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 ceph-mon[36053]: osdmap e23: 4 total, 3 up, 4 in 2024-06-06T04:22:34.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T04:22:34.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T04:22:34.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:22:34.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:33 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:22:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:34 smithi086 ceph-mon[36053]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:34 smithi086 ceph-mon[36053]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T04:22:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:34 smithi086 ceph-mon[36053]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T04:22:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T04:22:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:22:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:22:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:34 smithi086 ceph-mon[36053]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T04:22:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:34 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3344096322' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:22:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:34 smithi086 ceph-mon[36053]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T04:22:34.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:34 smithi154 ceph-mon[36428]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:34.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:34 smithi154 ceph-mon[36428]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T04:22:34.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:34 smithi154 ceph-mon[36428]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T04:22:34.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T04:22:34.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:22:34.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:22:34.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:34 smithi154 ceph-mon[36428]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T04:22:34.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:34 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3344096322' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:22:34.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:34 smithi154 ceph-mon[36428]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T04:22:34.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:34 smithi045 ceph-mon[32299]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:34 smithi045 ceph-mon[32299]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T04:22:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:34 smithi045 ceph-mon[32299]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T04:22:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T04:22:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T04:22:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T04:22:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:34 smithi045 ceph-mon[32299]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T04:22:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:34 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3344096322' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:22:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:34 smithi045 ceph-mon[32299]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T04:22:35.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:35 smithi086 ceph-mon[36053]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:35.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:35 smithi154 ceph-mon[36428]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:35.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:35 smithi045 ceph-mon[32299]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:37.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:37 smithi045 ceph-mon[32299]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:37.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:37 smithi086 ceph-mon[36053]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:37.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:37 smithi154 ceph-mon[36428]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:39.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:39 smithi086 ceph-mon[36053]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:39.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:39 smithi154 ceph-mon[36428]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:39.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:39 smithi045 ceph-mon[32299]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:40.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:40 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-06T04:22:40.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:40 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:40.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:40 smithi045 ceph-mon[32299]: Deploying daemon osd.3 on smithi045 2024-06-06T04:22:40.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:40 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-06T04:22:40.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:40 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:40.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:40 smithi086 ceph-mon[36053]: Deploying daemon osd.3 on smithi045 2024-06-06T04:22:40.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:40 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-06T04:22:40.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:40 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:40.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:40 smithi154 ceph-mon[36428]: Deploying daemon osd.3 on smithi045 2024-06-06T04:22:41.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:41 smithi045 ceph-mon[32299]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-06T04:22:41.608 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:41 smithi086 ceph-mon[36053]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-06T04:22:41.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:41 smithi154 ceph-mon[36428]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-06T04:22:43.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:43 smithi045 ceph-mon[32299]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-06T04:22:43.608 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:43 smithi086 ceph-mon[36053]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-06T04:22:43.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:43 smithi154 ceph-mon[36428]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-06T04:22:45.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:45 smithi086 ceph-mon[36053]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-06T04:22:45.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:45 smithi154 ceph-mon[36428]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-06T04:22:45.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:45 smithi045 ceph-mon[32299]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-06T04:22:46.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:46 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:46.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:46 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:46.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:46 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:22:46.926 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:46 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:46.926 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:46 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:46.927 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:46 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:22:46.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:46 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:46.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:46 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:46.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:46 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:22:47.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:47 smithi086 ceph-mon[36053]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-06T04:22:47.873 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:47 smithi045 ceph-mon[32299]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-06T04:22:47.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:47 smithi154 ceph-mon[36428]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-06T04:22:48.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:48.861+0000 7f487e7fc700 1 -- 172.21.15.45:0/4260723490 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f48880628d0 con 0x7f48740a2b90 2024-06-06T04:22:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 3 on host 'smithi045' 2024-06-06T04:22:48.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:48.863+0000 7f488f83b700 1 -- 172.21.15.45:0/4260723490 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f48740a2b90 msgr2=0x7f48740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:22:48.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:48.863+0000 7f488f83b700 1 --2- 172.21.15.45:0/4260723490 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f48740a2b90 0x7f48740a5050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f488806b250 tx=0x7f487800b3c0 comp rx=0 tx=0).stop 2024-06-06T04:22:48.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:48.864+0000 7f488f83b700 1 -- 172.21.15.45:0/4260723490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48881013f0 msgr2=0x7f488806acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:22:48.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:48.864+0000 7f488f83b700 1 --2- 172.21.15.45:0/4260723490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48881013f0 0x7f488806acd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f48840098e0 tx=0x7f48840057c0 comp rx=0 tx=0).stop 2024-06-06T04:22:48.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:48.864+0000 7f488f83b700 1 -- 172.21.15.45:0/4260723490 shutdown_connections 2024-06-06T04:22:48.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:48.864+0000 7f488f83b700 1 --2- 172.21.15.45:0/4260723490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4888102f90 0x7f488819fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:48.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:48.864+0000 7f488f83b700 1 --2- 172.21.15.45:0/4260723490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48881013f0 0x7f488806acd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:48.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:48.864+0000 7f488f83b700 1 --2- 172.21.15.45:0/4260723490 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f48740a2b90 0x7f48740a5050 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:48.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:48.864+0000 7f488f83b700 1 --2- 172.21.15.45:0/4260723490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48881025f0 0x7f488819d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:48.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:48.864+0000 7f488f83b700 1 -- 172.21.15.45:0/4260723490 >> 172.21.15.45:0/4260723490 conn(0x7f48880fc990 msgr2=0x7f48881059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:22:48.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:48.864+0000 7f488f83b700 1 -- 172.21.15.45:0/4260723490 shutdown_connections 2024-06-06T04:22:48.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:22:48.864+0000 7f488f83b700 1 -- 172.21.15.45:0/4260723490 wait complete. 2024-06-06T04:22:49.551 DEBUG:teuthology.orchestra.run.smithi045:osd.3> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.3.service 2024-06-06T04:22:49.554 INFO:tasks.cephadm:Deploying osd.4 on smithi086 with /dev/vg_nvme/lv_3... 2024-06-06T04:22:49.554 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-06T04:22:49.798 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T04:22:50.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:49 smithi086 ceph-mon[36053]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-06T04:22:50.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:49 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:50.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:49 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:50.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:49 smithi086 ceph-mon[36053]: from='osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-06T04:22:50.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:49 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:50.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:49 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:50.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:49 smithi154 ceph-mon[36428]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-06T04:22:50.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:49 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:50.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:49 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:50.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:49 smithi154 ceph-mon[36428]: from='osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-06T04:22:50.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:49 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:50.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:49 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:50.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:49 smithi045 ceph-mon[32299]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-06T04:22:50.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:49 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:50.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:49 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:50.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:49 smithi045 ceph-mon[32299]: from='osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-06T04:22:50.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:49 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:50.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:49 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:51.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:50 smithi154 ceph-mon[36428]: from='osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-06T04:22:51.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:50 smithi154 ceph-mon[36428]: osdmap e24: 4 total, 3 up, 4 in 2024-06-06T04:22:51.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:50 smithi154 ceph-mon[36428]: from='osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-06-06T04:22:51.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:50 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T04:22:51.181 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:50 smithi086 ceph-mon[36053]: from='osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-06T04:22:51.181 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:50 smithi086 ceph-mon[36053]: osdmap e24: 4 total, 3 up, 4 in 2024-06-06T04:22:51.181 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:50 smithi086 ceph-mon[36053]: from='osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-06-06T04:22:51.181 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:50 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T04:22:51.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:50 smithi045 ceph-mon[32299]: from='osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-06T04:22:51.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:50 smithi045 ceph-mon[32299]: osdmap e24: 4 total, 3 up, 4 in 2024-06-06T04:22:51.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:50 smithi045 ceph-mon[32299]: from='osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-06-06T04:22:51.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:50 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T04:22:51.590 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:22:51 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-3[54073]: 2024-06-06T04:22:51.329+0000 7f7ffd7e1700 -1 osd.3 0 waiting for initial osdmap 2024-06-06T04:22:51.590 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:22:51 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-3[54073]: 2024-06-06T04:22:51.335+0000 7f7ff4dcb700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T04:22:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-06T04:22:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-06T04:22:51.748 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-06-06T04:22:51.749 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-06-06T04:22:51.749 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012485 s, 840 MB/s 2024-06-06T04:22:51.749 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-06-06T04:22:52.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:51 smithi154 ceph-mon[36428]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:52.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:51 smithi154 ceph-mon[36428]: from='osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-06-06T04:22:52.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:51 smithi154 ceph-mon[36428]: osdmap e25: 4 total, 3 up, 4 in 2024-06-06T04:22:52.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:51 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T04:22:52.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:51 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T04:22:52.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:51 smithi154 ceph-mon[36428]: from='osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135]' entity='osd.3' 2024-06-06T04:22:52.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:51 smithi154 ceph-mon[36428]: osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135] boot 2024-06-06T04:22:52.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:51 smithi154 ceph-mon[36428]: osdmap e26: 4 total, 4 up, 4 in 2024-06-06T04:22:52.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:51 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T04:22:52.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:51 smithi045 ceph-mon[32299]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:52.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:51 smithi045 ceph-mon[32299]: from='osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-06-06T04:22:52.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:51 smithi045 ceph-mon[32299]: osdmap e25: 4 total, 3 up, 4 in 2024-06-06T04:22:52.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:51 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T04:22:52.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:51 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T04:22:52.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:51 smithi045 ceph-mon[32299]: from='osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135]' entity='osd.3' 2024-06-06T04:22:52.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:51 smithi045 ceph-mon[32299]: osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135] boot 2024-06-06T04:22:52.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:51 smithi045 ceph-mon[32299]: osdmap e26: 4 total, 4 up, 4 in 2024-06-06T04:22:52.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:51 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T04:22:52.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:51 smithi086 ceph-mon[36053]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-06-06T04:22:52.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:51 smithi086 ceph-mon[36053]: from='osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-06-06T04:22:52.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:51 smithi086 ceph-mon[36053]: osdmap e25: 4 total, 3 up, 4 in 2024-06-06T04:22:52.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:51 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T04:22:52.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:51 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T04:22:52.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:51 smithi086 ceph-mon[36053]: from='osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135]' entity='osd.3' 2024-06-06T04:22:52.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:51 smithi086 ceph-mon[36053]: osd.3 [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135] boot 2024-06-06T04:22:52.344 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:51 smithi086 ceph-mon[36053]: osdmap e26: 4 total, 4 up, 4 in 2024-06-06T04:22:52.345 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:51 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T04:22:52.371 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_3 2024-06-06T04:22:52.613 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T04:22:52.996 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:52 smithi086 ceph-mon[36053]: purged_snaps scrub starts 2024-06-06T04:22:52.996 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:52 smithi086 ceph-mon[36053]: purged_snaps scrub ok 2024-06-06T04:22:52.996 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:52 smithi086 ceph-mon[36053]: osdmap e27: 4 total, 4 up, 4 in 2024-06-06T04:22:53.106 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:52 smithi045 ceph-mon[32299]: purged_snaps scrub starts 2024-06-06T04:22:53.107 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:52 smithi045 ceph-mon[32299]: purged_snaps scrub ok 2024-06-06T04:22:53.107 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:52 smithi045 ceph-mon[32299]: osdmap e27: 4 total, 4 up, 4 in 2024-06-06T04:22:53.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:52 smithi154 ceph-mon[36428]: purged_snaps scrub starts 2024-06-06T04:22:53.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:52 smithi154 ceph-mon[36428]: purged_snaps scrub ok 2024-06-06T04:22:53.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:52 smithi154 ceph-mon[36428]: osdmap e27: 4 total, 4 up, 4 in 2024-06-06T04:22:54.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.257+0000 7f78a68eb700 1 -- 172.21.15.86:0/203555030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78a01016b0 msgr2=0x7f78a0101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:22:54.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.257+0000 7f78a68eb700 1 --2- 172.21.15.86:0/203555030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78a01016b0 0x7f78a0101b30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7894009990 tx=0x7f7894009ca0 comp rx=0 tx=0).stop 2024-06-06T04:22:54.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.257+0000 7f78a68eb700 1 -- 172.21.15.86:0/203555030 shutdown_connections 2024-06-06T04:22:54.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.257+0000 7f78a68eb700 1 --2- 172.21.15.86:0/203555030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a0102070 0x7f78a0109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:54.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.257+0000 7f78a68eb700 1 --2- 172.21.15.86:0/203555030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78a0102a00 0x7f78a0102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:54.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.257+0000 7f78a68eb700 1 --2- 172.21.15.86:0/203555030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78a01016b0 0x7f78a0101b30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:54.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.257+0000 7f78a68eb700 1 -- 172.21.15.86:0/203555030 >> 172.21.15.86:0/203555030 conn(0x7f78a00fc100 msgr2=0x7f78a00fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:22:54.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.257+0000 7f78a68eb700 1 -- 172.21.15.86:0/203555030 shutdown_connections 2024-06-06T04:22:54.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.257+0000 7f78a68eb700 1 -- 172.21.15.86:0/203555030 wait complete. 2024-06-06T04:22:54.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.258+0000 7f78a68eb700 1 Processor -- start 2024-06-06T04:22:54.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.258+0000 7f78a68eb700 1 -- start start 2024-06-06T04:22:54.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.258+0000 7f78a68eb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a01016b0 0x7f78a0105400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:22:54.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.258+0000 7f78a68eb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78a0102070 0x7f78a0103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:22:54.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.258+0000 7f78a68eb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78a0102a00 0x7f78a0103fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:22:54.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.258+0000 7f78a68eb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f78a010cec0 con 0x7f78a01016b0 2024-06-06T04:22:54.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.258+0000 7f78a68eb700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78a010cd40 con 0x7f78a0102070 2024-06-06T04:22:54.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.258+0000 7f78a68eb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f78a010d040 con 0x7f78a0102a00 2024-06-06T04:22:54.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.258+0000 7f789f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78a0102070 0x7f78a0103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:22:54.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.258+0000 7f789f7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78a0102070 0x7f78a0103aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55754/0 (socket says 172.21.15.86:55754) 2024-06-06T04:22:54.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.258+0000 7f789f7fe700 1 -- 172.21.15.86:0/2731522613 learned_addr learned my addr 172.21.15.86:0/2731522613 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-06-06T04:22:54.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.259+0000 7f789ffff700 1 --2- 172.21.15.86:0/2731522613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a01016b0 0x7f78a0105400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:22:54.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.259+0000 7f789f7fe700 1 -- 172.21.15.86:0/2731522613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78a0102a00 msgr2=0x7f78a0103fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:22:54.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.259+0000 7f789f7fe700 1 --2- 172.21.15.86:0/2731522613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78a0102a00 0x7f78a0103fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:54.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.259+0000 7f789f7fe700 1 -- 172.21.15.86:0/2731522613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a01016b0 msgr2=0x7f78a0105400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:22:54.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.259+0000 7f789f7fe700 1 --2- 172.21.15.86:0/2731522613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a01016b0 0x7f78a0105400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:22:54.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.259+0000 7f789f7fe700 1 -- 172.21.15.86:0/2731522613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a01048b0 con 0x7f78a0102070 2024-06-06T04:22:54.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.259+0000 7f789f7fe700 1 --2- 172.21.15.86:0/2731522613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78a0102070 0x7f78a0103aa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f78940098e0 tx=0x7f78940057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:22:54.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.259+0000 7f789d7fa700 1 -- 172.21.15.86:0/2731522613 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894027070 con 0x7f78a0102070 2024-06-06T04:22:54.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.259+0000 7f78a68eb700 1 -- 172.21.15.86:0/2731522613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a01a2760 con 0x7f78a0102070 2024-06-06T04:22:54.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.259+0000 7f789d7fa700 1 -- 172.21.15.86:0/2731522613 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7894022470 con 0x7f78a0102070 2024-06-06T04:22:54.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.259+0000 7f789d7fa700 1 -- 172.21.15.86:0/2731522613 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789400f6c0 con 0x7f78a0102070 2024-06-06T04:22:54.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.259+0000 7f78a68eb700 1 -- 172.21.15.86:0/2731522613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a01a2c60 con 0x7f78a0102070 2024-06-06T04:22:54.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.260+0000 7f78a68eb700 1 -- 172.21.15.86:0/2731522613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a0069180 con 0x7f78a0102070 2024-06-06T04:22:54.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.261+0000 7f789d7fa700 1 -- 172.21.15.86:0/2731522613 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f789402a070 con 0x7f78a0102070 2024-06-06T04:22:54.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.262+0000 7f789d7fa700 1 --2- 172.21.15.86:0/2731522613 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f78880a2c20 0x7f78880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:22:54.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.262+0000 7f789d7fa700 1 -- 172.21.15.86:0/2731522613 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7f78940cf1c0 con 0x7f78a0102070 2024-06-06T04:22:54.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.262+0000 7f789ffff700 1 --2- 172.21.15.86:0/2731522613 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f78880a2c20 0x7f78880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:22:54.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.263+0000 7f789ffff700 1 --2- 172.21.15.86:0/2731522613 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f78880a2c20 0x7f78880a50e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7890005950 tx=0x7f789000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:22:54.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.266+0000 7f789d7fa700 1 -- 172.21.15.86:0/2731522613 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7894031080 con 0x7f78a0102070 2024-06-06T04:22:54.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:54 smithi086 ceph-mon[36053]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:22:54.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:54 smithi086 ceph-mon[36053]: Detected new or changed devices on smithi045 2024-06-06T04:22:54.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:54 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:54.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:54 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:54.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:54 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:22:54.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:54 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:22:54.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:54 smithi086 ceph-mon[36053]: Adjusting osd_memory_target on smithi045 to 8546M 2024-06-06T04:22:54.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:54 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:54.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:54 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:54.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:54 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:22:54.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:54 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:54.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:54 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:22:54.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:54 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:54.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:54 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:22:54.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:54 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:54.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:54 smithi086 ceph-mon[36053]: osdmap e28: 4 total, 4 up, 4 in 2024-06-06T04:22:54.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:54 smithi154 ceph-mon[36428]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:22:54.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:54 smithi154 ceph-mon[36428]: Detected new or changed devices on smithi045 2024-06-06T04:22:54.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:54.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:54.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:22:54.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:22:54.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:54 smithi154 ceph-mon[36428]: Adjusting osd_memory_target on smithi045 to 8546M 2024-06-06T04:22:54.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:54.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:54.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:22:54.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:54.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:22:54.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:54.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:22:54.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:54.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:54 smithi154 ceph-mon[36428]: osdmap e28: 4 total, 4 up, 4 in 2024-06-06T04:22:54.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:54 smithi045 ceph-mon[32299]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:22:54.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:54 smithi045 ceph-mon[32299]: Detected new or changed devices on smithi045 2024-06-06T04:22:54.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:54.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:54.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:22:54.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:22:54.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:54 smithi045 ceph-mon[32299]: Adjusting osd_memory_target on smithi045 to 8546M 2024-06-06T04:22:54.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:54.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:54.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:22:54.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:54.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:22:54.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:54.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:22:54.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:22:54.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:54 smithi045 ceph-mon[32299]: osdmap e28: 4 total, 4 up, 4 in 2024-06-06T04:22:54.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:22:54.473+0000 7f78a68eb700 1 -- 172.21.15.86:0/2731522613 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f78a00fe2c0 con 0x7f78880a2c20 2024-06-06T04:22:55.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:55 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:22:55.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:55 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:22:55.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:55 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:55.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:55 smithi086 ceph-mon[36053]: osdmap e29: 4 total, 4 up, 4 in 2024-06-06T04:22:55.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:55 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:22:55.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:55 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:22:55.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:55 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:55.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:55 smithi154 ceph-mon[36428]: osdmap e29: 4 total, 4 up, 4 in 2024-06-06T04:22:55.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:55 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:22:55.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:55 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:22:55.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:55 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:22:55.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:55 smithi045 ceph-mon[32299]: osdmap e29: 4 total, 4 up, 4 in 2024-06-06T04:22:56.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:56 smithi086 ceph-mon[36053]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:22:56.307 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:56 smithi086 ceph-mon[36053]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:22:56.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:56 smithi086 ceph-mon[36053]: osdmap e30: 4 total, 4 up, 4 in 2024-06-06T04:22:56.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:56 smithi045 ceph-mon[32299]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:22:56.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:56 smithi045 ceph-mon[32299]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:22:56.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:56 smithi045 ceph-mon[32299]: osdmap e30: 4 total, 4 up, 4 in 2024-06-06T04:22:56.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:56 smithi154 ceph-mon[36428]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:22:56.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:56 smithi154 ceph-mon[36428]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:22:56.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:56 smithi154 ceph-mon[36428]: osdmap e30: 4 total, 4 up, 4 in 2024-06-06T04:22:57.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:57 smithi045 ceph-mon[32299]: from='client.? 172.21.15.86:0/1084744124' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b389bc8-e19a-47f3-83ff-7b782d146d0c"}]: dispatch 2024-06-06T04:22:57.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:57 smithi045 ceph-mon[32299]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b389bc8-e19a-47f3-83ff-7b782d146d0c"}]: dispatch 2024-06-06T04:22:57.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:57 smithi045 ceph-mon[32299]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b389bc8-e19a-47f3-83ff-7b782d146d0c"}]': finished 2024-06-06T04:22:57.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:57 smithi045 ceph-mon[32299]: osdmap e31: 5 total, 4 up, 5 in 2024-06-06T04:22:57.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:57 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T04:22:57.492 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:57 smithi086 ceph-mon[36053]: from='client.? 172.21.15.86:0/1084744124' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b389bc8-e19a-47f3-83ff-7b782d146d0c"}]: dispatch 2024-06-06T04:22:57.492 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:57 smithi086 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b389bc8-e19a-47f3-83ff-7b782d146d0c"}]: dispatch 2024-06-06T04:22:57.492 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:57 smithi086 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b389bc8-e19a-47f3-83ff-7b782d146d0c"}]': finished 2024-06-06T04:22:57.492 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:57 smithi086 ceph-mon[36053]: osdmap e31: 5 total, 4 up, 5 in 2024-06-06T04:22:57.492 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:57 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T04:22:57.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:57 smithi154 ceph-mon[36428]: from='client.? 172.21.15.86:0/1084744124' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b389bc8-e19a-47f3-83ff-7b782d146d0c"}]: dispatch 2024-06-06T04:22:57.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:57 smithi154 ceph-mon[36428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b389bc8-e19a-47f3-83ff-7b782d146d0c"}]: dispatch 2024-06-06T04:22:57.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:57 smithi154 ceph-mon[36428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b389bc8-e19a-47f3-83ff-7b782d146d0c"}]': finished 2024-06-06T04:22:57.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:57 smithi154 ceph-mon[36428]: osdmap e31: 5 total, 4 up, 5 in 2024-06-06T04:22:57.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:57 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T04:22:58.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:58 smithi045 ceph-mon[32299]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:22:58.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:58 smithi045 ceph-mon[32299]: from='client.? 172.21.15.86:0/3057968139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:22:58.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:58 smithi086 ceph-mon[36053]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:22:58.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:58 smithi086 ceph-mon[36053]: from='client.? 172.21.15.86:0/3057968139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:22:58.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:58 smithi154 ceph-mon[36428]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:22:58.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:58 smithi154 ceph-mon[36428]: from='client.? 172.21.15.86:0/3057968139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:22:59.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:22:59 smithi045 ceph-mon[32299]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:22:59.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:22:59 smithi086 ceph-mon[36053]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:22:59.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:22:59 smithi154 ceph-mon[36428]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:23:01.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:01 smithi086 ceph-mon[36053]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T04:23:01.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:01 smithi154 ceph-mon[36428]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T04:23:01.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:01 smithi045 ceph-mon[32299]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T04:23:03.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:03 smithi086 ceph-mon[36053]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-06-06T04:23:03.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:03 smithi154 ceph-mon[36428]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-06-06T04:23:03.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:03 smithi045 ceph-mon[32299]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-06-06T04:23:05.435 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:05 smithi086 ceph-mon[36053]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-06-06T04:23:05.435 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-06T04:23:05.435 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:05.435 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:05 smithi086 ceph-mon[36053]: Deploying daemon osd.4 on smithi086 2024-06-06T04:23:05.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:05 smithi045 ceph-mon[32299]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-06-06T04:23:05.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-06T04:23:05.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:05.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:05 smithi045 ceph-mon[32299]: Deploying daemon osd.4 on smithi086 2024-06-06T04:23:05.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:05 smithi154 ceph-mon[36428]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-06-06T04:23:05.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-06T04:23:05.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:05.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:05 smithi154 ceph-mon[36428]: Deploying daemon osd.4 on smithi086 2024-06-06T04:23:07.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:07 smithi086 ceph-mon[36053]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T04:23:07.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:07 smithi154 ceph-mon[36428]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T04:23:07.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:07 smithi045 ceph-mon[32299]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T04:23:09.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:09 smithi086 ceph-mon[36053]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-06T04:23:09.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:09 smithi154 ceph-mon[36428]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-06T04:23:09.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:09 smithi045 ceph-mon[32299]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-06T04:23:11.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:11.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:11.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:23:11.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:11.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:11.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:23:11.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:11.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:11.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:23:12.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:11 smithi154 ceph-mon[36428]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-06T04:23:12.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:11 smithi045 ceph-mon[32299]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-06T04:23:12.273 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:11 smithi086 ceph-mon[36053]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-06T04:23:13.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:13 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:13.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:13 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:13.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:13 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:13.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:13 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:23:13.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:13 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:13.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:13 smithi045 ceph-mon[32299]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:23:13.566 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:13 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:13.566 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:13 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:13.566 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:13 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:13.567 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:13 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:23:13.567 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:13 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:13.567 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:13 smithi086 ceph-mon[36053]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:23:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:23:13.644+0000 7f789d7fa700 1 -- 172.21.15.86:0/2731522613 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f78a00fe2c0 con 0x7f78880a2c20 2024-06-06T04:23:13.645 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 4 on host 'smithi086' 2024-06-06T04:23:13.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:23:13.647+0000 7f78a68eb700 1 -- 172.21.15.86:0/2731522613 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f78880a2c20 msgr2=0x7f78880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:23:13.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:23:13.647+0000 7f78a68eb700 1 --2- 172.21.15.86:0/2731522613 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f78880a2c20 0x7f78880a50e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7890005950 tx=0x7f789000a300 comp rx=0 tx=0).stop 2024-06-06T04:23:13.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:23:13.647+0000 7f78a68eb700 1 -- 172.21.15.86:0/2731522613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78a0102070 msgr2=0x7f78a0103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:23:13.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:23:13.647+0000 7f78a68eb700 1 --2- 172.21.15.86:0/2731522613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78a0102070 0x7f78a0103aa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f78940098e0 tx=0x7f78940057c0 comp rx=0 tx=0).stop 2024-06-06T04:23:13.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:23:13.647+0000 7f78a68eb700 1 -- 172.21.15.86:0/2731522613 shutdown_connections 2024-06-06T04:23:13.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:23:13.647+0000 7f78a68eb700 1 --2- 172.21.15.86:0/2731522613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78a0102a00 0x7f78a0103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:13.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:23:13.647+0000 7f78a68eb700 1 --2- 172.21.15.86:0/2731522613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a01016b0 0x7f78a0105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:13.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:23:13.647+0000 7f78a68eb700 1 --2- 172.21.15.86:0/2731522613 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f78880a2c20 0x7f78880a50e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:13.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:23:13.647+0000 7f78a68eb700 1 --2- 172.21.15.86:0/2731522613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78a0102070 0x7f78a0103aa0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:13.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:23:13.647+0000 7f78a68eb700 1 -- 172.21.15.86:0/2731522613 >> 172.21.15.86:0/2731522613 conn(0x7f78a00fc100 msgr2=0x7f78a00fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:23:13.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:23:13.647+0000 7f78a68eb700 1 -- 172.21.15.86:0/2731522613 shutdown_connections 2024-06-06T04:23:13.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:23:13.647+0000 7f78a68eb700 1 -- 172.21.15.86:0/2731522613 wait complete. 2024-06-06T04:23:13.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:13 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:13.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:13 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:13.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:13 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:13.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:13 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:23:13.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:13 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:13.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:13 smithi154 ceph-mon[36428]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:23:14.271 DEBUG:teuthology.orchestra.run.smithi086:osd.4> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.4.service 2024-06-06T04:23:14.273 INFO:tasks.cephadm:Deploying osd.5 on smithi154 with /dev/vg_nvme/lv_3... 2024-06-06T04:23:14.273 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-06T04:23:14.519 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T04:23:14.807 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:14.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:14.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:23:14.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:14 smithi086 ceph-mon[36053]: from='osd.4 [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T04:23:14.808 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:14 smithi086 ceph-mon[36053]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T04:23:14.855 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:14.855 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:14.855 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:23:14.855 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:14 smithi154 ceph-mon[36428]: from='osd.4 [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T04:23:14.855 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:14 smithi154 ceph-mon[36428]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T04:23:14.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:14.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:14.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:23:14.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:14 smithi045 ceph-mon[32299]: from='osd.4 [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T04:23:14.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:14 smithi045 ceph-mon[32299]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T04:23:15.761 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:15 smithi154 ceph-mon[36428]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:23:15.761 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:15 smithi154 ceph-mon[36428]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-06T04:23:15.761 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:15 smithi154 ceph-mon[36428]: osdmap e32: 5 total, 4 up, 5 in 2024-06-06T04:23:15.761 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:15 smithi154 ceph-mon[36428]: from='osd.4 [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:23:15.761 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:15 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T04:23:15.761 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:15 smithi154 ceph-mon[36428]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:23:15.771 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:15 smithi086 ceph-mon[36053]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:23:15.771 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:15 smithi086 ceph-mon[36053]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-06T04:23:15.771 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:15 smithi086 ceph-mon[36053]: osdmap e32: 5 total, 4 up, 5 in 2024-06-06T04:23:15.772 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:15 smithi086 ceph-mon[36053]: from='osd.4 [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:23:15.772 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:15 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T04:23:15.772 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:15 smithi086 ceph-mon[36053]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:23:15.772 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:23:15 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4[47207]: 2024-06-06T04:23:15.613+0000 7f6b57609700 -1 osd.4 0 waiting for initial osdmap 2024-06-06T04:23:15.772 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:23:15 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4[47207]: 2024-06-06T04:23:15.619+0000 7f6b533fc700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T04:23:15.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:15 smithi045 ceph-mon[32299]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:23:15.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:15 smithi045 ceph-mon[32299]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-06T04:23:15.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:15 smithi045 ceph-mon[32299]: osdmap e32: 5 total, 4 up, 5 in 2024-06-06T04:23:15.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:15 smithi045 ceph-mon[32299]: from='osd.4 [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:23:15.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:15 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T04:23:15.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:15 smithi045 ceph-mon[32299]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:23:16.362 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-06T04:23:16.362 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-06T04:23:16.362 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-06-06T04:23:16.362 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-06-06T04:23:16.362 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0141921 s, 739 MB/s 2024-06-06T04:23:16.362 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-06-06T04:23:16.835 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:16 smithi086 ceph-mon[36053]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-06-06T04:23:16.835 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:16 smithi086 ceph-mon[36053]: osdmap e33: 5 total, 4 up, 5 in 2024-06-06T04:23:16.836 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:16 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T04:23:16.836 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:16 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T04:23:16.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:16 smithi154 ceph-mon[36428]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-06-06T04:23:16.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:16 smithi154 ceph-mon[36428]: osdmap e33: 5 total, 4 up, 5 in 2024-06-06T04:23:16.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:16 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T04:23:16.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:16 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T04:23:16.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:16 smithi045 ceph-mon[32299]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-06-06T04:23:16.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:16 smithi045 ceph-mon[32299]: osdmap e33: 5 total, 4 up, 5 in 2024-06-06T04:23:16.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:16 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T04:23:16.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:16 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T04:23:17.010 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi154:vg_nvme/lv_3 2024-06-06T04:23:17.251 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T04:23:17.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:17 smithi154 ceph-mon[36428]: purged_snaps scrub starts 2024-06-06T04:23:17.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:17 smithi154 ceph-mon[36428]: purged_snaps scrub ok 2024-06-06T04:23:17.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:17 smithi154 ceph-mon[36428]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:23:17.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:17 smithi154 ceph-mon[36428]: osd.4 [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796] boot 2024-06-06T04:23:17.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:17 smithi154 ceph-mon[36428]: osdmap e34: 5 total, 5 up, 5 in 2024-06-06T04:23:17.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:17 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T04:23:17.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:17 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:17.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:17 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:17.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:17 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:23:17.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:17 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:23:17.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:17 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:17.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:17 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:17.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:17 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:23:17.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:17 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:17 smithi045 ceph-mon[32299]: purged_snaps scrub starts 2024-06-06T04:23:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:17 smithi045 ceph-mon[32299]: purged_snaps scrub ok 2024-06-06T04:23:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:17 smithi045 ceph-mon[32299]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:23:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:17 smithi045 ceph-mon[32299]: osd.4 [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796] boot 2024-06-06T04:23:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:17 smithi045 ceph-mon[32299]: osdmap e34: 5 total, 5 up, 5 in 2024-06-06T04:23:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T04:23:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:23:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:23:17.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:17.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:17.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:23:17.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:18.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:17 smithi086 ceph-mon[36053]: purged_snaps scrub starts 2024-06-06T04:23:18.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:17 smithi086 ceph-mon[36053]: purged_snaps scrub ok 2024-06-06T04:23:18.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:17 smithi086 ceph-mon[36053]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-06-06T04:23:18.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:17 smithi086 ceph-mon[36053]: osd.4 [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796] boot 2024-06-06T04:23:18.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:17 smithi086 ceph-mon[36053]: osdmap e34: 5 total, 5 up, 5 in 2024-06-06T04:23:18.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:17 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T04:23:18.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:17 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:18.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:17 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:18.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:17 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:23:18.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:17 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:23:18.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:17 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:18.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:17 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:18.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:17 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:23:18.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:17 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:18.868 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:18 smithi154 ceph-mon[36428]: Detected new or changed devices on smithi086 2024-06-06T04:23:18.869 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:18 smithi154 ceph-mon[36428]: Adjusting osd_memory_target on smithi086 to 8544M 2024-06-06T04:23:18.869 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:18 smithi154 ceph-mon[36428]: osdmap e35: 5 total, 5 up, 5 in 2024-06-06T04:23:18.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.874+0000 7f2f56d6b700 1 -- 172.21.15.154:0/392912359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f50102fb0 msgr2=0x7f2f50109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:23:18.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.874+0000 7f2f56d6b700 1 --2- 172.21.15.154:0/392912359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f50102fb0 0x7f2f50109860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800b0a0 tx=0x7f2f4800b3b0 comp rx=0 tx=0).stop 2024-06-06T04:23:18.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.874+0000 7f2f56d6b700 1 -- 172.21.15.154:0/392912359 shutdown_connections 2024-06-06T04:23:18.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.874+0000 7f2f56d6b700 1 --2- 172.21.15.154:0/392912359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f50102fb0 0x7f2f50109860 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:18.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.874+0000 7f2f56d6b700 1 --2- 172.21.15.154:0/392912359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f50101410 0x7f2f50101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:18.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.874+0000 7f2f56d6b700 1 --2- 172.21.15.154:0/392912359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f50102610 0x7f2f50102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:18.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.874+0000 7f2f56d6b700 1 -- 172.21.15.154:0/392912359 >> 172.21.15.154:0/392912359 conn(0x7f2f500fc9f0 msgr2=0x7f2f500fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:23:18.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.874+0000 7f2f56d6b700 1 -- 172.21.15.154:0/392912359 shutdown_connections 2024-06-06T04:23:18.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.874+0000 7f2f56d6b700 1 -- 172.21.15.154:0/392912359 wait complete. 2024-06-06T04:23:18.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f56d6b700 1 Processor -- start 2024-06-06T04:23:18.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f56d6b700 1 -- start start 2024-06-06T04:23:18.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f56d6b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f50101410 0x7f2f501053a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:23:18.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f56d6b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f50102610 0x7f2f501039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:23:18.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f56d6b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f50102fb0 0x7f2f50103f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:23:18.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f56d6b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5010cea0 con 0x7f2f50102fb0 2024-06-06T04:23:18.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f56d6b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5010cd20 con 0x7f2f50102610 2024-06-06T04:23:18.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f56d6b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5010d020 con 0x7f2f50101410 2024-06-06T04:23:18.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f54b07700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f50101410 0x7f2f501053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:23:18.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f54b07700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f50101410 0x7f2f501053a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57184/0 (socket says 172.21.15.154:57184) 2024-06-06T04:23:18.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f54b07700 1 -- 172.21.15.154:0/2325668168 learned_addr learned my addr 172.21.15.154:0/2325668168 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-06-06T04:23:18.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f55308700 1 --2- 172.21.15.154:0/2325668168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f50102fb0 0x7f2f50103f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:23:18.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f54b07700 1 -- 172.21.15.154:0/2325668168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f50102610 msgr2=0x7f2f501039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:23:18.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f4ffff700 1 --2- 172.21.15.154:0/2325668168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f50102610 0x7f2f501039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:23:18.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f54b07700 1 --2- 172.21.15.154:0/2325668168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f50102610 0x7f2f501039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:18.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f54b07700 1 -- 172.21.15.154:0/2325668168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f50102fb0 msgr2=0x7f2f50103f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:23:18.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f54b07700 1 --2- 172.21.15.154:0/2325668168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f50102fb0 0x7f2f50103f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:18.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f54b07700 1 -- 172.21.15.154:0/2325668168 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f501047e0 con 0x7f2f50101410 2024-06-06T04:23:18.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f55308700 1 --2- 172.21.15.154:0/2325668168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f50102fb0 0x7f2f50103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:23:18.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f4ffff700 1 --2- 172.21.15.154:0/2325668168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f50102610 0x7f2f501039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:23:18.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f54b07700 1 --2- 172.21.15.154:0/2325668168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f50101410 0x7f2f501053a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400d980 tx=0x7f2f4400dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:23:18.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.875+0000 7f2f4dffb700 1 -- 172.21.15.154:0/2325668168 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44017070 con 0x7f2f50101410 2024-06-06T04:23:18.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.876+0000 7f2f4dffb700 1 -- 172.21.15.154:0/2325668168 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f44010460 con 0x7f2f50101410 2024-06-06T04:23:18.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.876+0000 7f2f56d6b700 1 -- 172.21.15.154:0/2325668168 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f501a2740 con 0x7f2f50101410 2024-06-06T04:23:18.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.876+0000 7f2f4dffb700 1 -- 172.21.15.154:0/2325668168 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4400de60 con 0x7f2f50101410 2024-06-06T04:23:18.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.876+0000 7f2f56d6b700 1 -- 172.21.15.154:0/2325668168 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f501a2c00 con 0x7f2f50101410 2024-06-06T04:23:18.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.877+0000 7f2f56d6b700 1 -- 172.21.15.154:0/2325668168 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f50069180 con 0x7f2f50101410 2024-06-06T04:23:18.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.878+0000 7f2f4dffb700 1 -- 172.21.15.154:0/2325668168 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2f44010610 con 0x7f2f50101410 2024-06-06T04:23:18.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.878+0000 7f2f4dffb700 1 --2- 172.21.15.154:0/2325668168 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2f400a2c20 0x7f2f400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:23:18.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.878+0000 7f2f4dffb700 1 -- 172.21.15.154:0/2325668168 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f2f440cdb50 con 0x7f2f50101410 2024-06-06T04:23:18.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.879+0000 7f2f4ffff700 1 --2- 172.21.15.154:0/2325668168 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2f400a2c20 0x7f2f400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:23:18.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.879+0000 7f2f4ffff700 1 --2- 172.21.15.154:0/2325668168 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2f400a2c20 0x7f2f400a50e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c005d90 tx=0x7f2f3c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:23:18.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:18.883+0000 7f2f4dffb700 1 -- 172.21.15.154:0/2325668168 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2f4409bb20 con 0x7f2f50101410 2024-06-06T04:23:18.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:18 smithi045 ceph-mon[32299]: Detected new or changed devices on smithi086 2024-06-06T04:23:18.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:18 smithi045 ceph-mon[32299]: Adjusting osd_memory_target on smithi086 to 8544M 2024-06-06T04:23:18.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:18 smithi045 ceph-mon[32299]: osdmap e35: 5 total, 5 up, 5 in 2024-06-06T04:23:19.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:19.087+0000 7f2f56d6b700 1 -- 172.21.15.154:0/2325668168 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2f500feb70 con 0x7f2f400a2c20 2024-06-06T04:23:19.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:18 smithi086 ceph-mon[36053]: Detected new or changed devices on smithi086 2024-06-06T04:23:19.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:18 smithi086 ceph-mon[36053]: Adjusting osd_memory_target on smithi086 to 8544M 2024-06-06T04:23:19.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:18 smithi086 ceph-mon[36053]: osdmap e35: 5 total, 5 up, 5 in 2024-06-06T04:23:19.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:19 smithi154 ceph-mon[36428]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:19.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:19 smithi154 ceph-mon[36428]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:23:19.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:23:19.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:23:19.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:19.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:19 smithi045 ceph-mon[32299]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:19.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:19 smithi045 ceph-mon[32299]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:23:19.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:23:19.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:23:19.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:20.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:19 smithi086 ceph-mon[36053]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:20.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:19 smithi086 ceph-mon[36053]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:23:20.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:23:20.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:23:20.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:21.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:21 smithi154 ceph-mon[36428]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:21.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:21 smithi045 ceph-mon[32299]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:22.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:21 smithi086 ceph-mon[36053]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:22.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:22 smithi154 ceph-mon[36428]: from='client.? 172.21.15.154:0/2298164518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "069de7b2-89bf-4150-bed7-7b8832573dd1"}]: dispatch 2024-06-06T04:23:22.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:22 smithi154 ceph-mon[36428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "069de7b2-89bf-4150-bed7-7b8832573dd1"}]: dispatch 2024-06-06T04:23:22.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:22 smithi154 ceph-mon[36428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "069de7b2-89bf-4150-bed7-7b8832573dd1"}]': finished 2024-06-06T04:23:22.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:22 smithi154 ceph-mon[36428]: osdmap e36: 6 total, 5 up, 6 in 2024-06-06T04:23:22.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:22 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T04:23:22.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:22 smithi154 ceph-mon[36428]: from='client.? 172.21.15.154:0/1386968133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:23:22.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:22 smithi045 ceph-mon[32299]: from='client.? 172.21.15.154:0/2298164518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "069de7b2-89bf-4150-bed7-7b8832573dd1"}]: dispatch 2024-06-06T04:23:22.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:22 smithi045 ceph-mon[32299]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "069de7b2-89bf-4150-bed7-7b8832573dd1"}]: dispatch 2024-06-06T04:23:22.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:22 smithi045 ceph-mon[32299]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "069de7b2-89bf-4150-bed7-7b8832573dd1"}]': finished 2024-06-06T04:23:22.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:22 smithi045 ceph-mon[32299]: osdmap e36: 6 total, 5 up, 6 in 2024-06-06T04:23:22.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:22 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T04:23:22.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:22 smithi045 ceph-mon[32299]: from='client.? 172.21.15.154:0/1386968133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:23:23.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:22 smithi086 ceph-mon[36053]: from='client.? 172.21.15.154:0/2298164518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "069de7b2-89bf-4150-bed7-7b8832573dd1"}]: dispatch 2024-06-06T04:23:23.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:22 smithi086 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "069de7b2-89bf-4150-bed7-7b8832573dd1"}]: dispatch 2024-06-06T04:23:23.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:22 smithi086 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "069de7b2-89bf-4150-bed7-7b8832573dd1"}]': finished 2024-06-06T04:23:23.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:22 smithi086 ceph-mon[36053]: osdmap e36: 6 total, 5 up, 6 in 2024-06-06T04:23:23.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:22 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T04:23:23.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:22 smithi086 ceph-mon[36053]: from='client.? 172.21.15.154:0/1386968133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:23:24.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:23 smithi045 ceph-mon[32299]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:24.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:23 smithi086 ceph-mon[36053]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:24.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:23 smithi154 ceph-mon[36428]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:26.176 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:25 smithi154 ceph-mon[36428]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:26.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:25 smithi045 ceph-mon[32299]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:26.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:25 smithi086 ceph-mon[36053]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:28.112 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:27 smithi154 ceph-mon[36428]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:28.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:27 smithi045 ceph-mon[32299]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:28.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:27 smithi086 ceph-mon[36053]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:30.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:29 smithi154 ceph-mon[36428]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:30.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:29 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-06T04:23:30.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:29 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:30.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:29 smithi045 ceph-mon[32299]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:30.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:29 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-06T04:23:30.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:29 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:30.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:29 smithi086 ceph-mon[36053]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:30.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:29 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-06T04:23:30.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:29 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:31.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:30 smithi154 ceph-mon[36428]: Deploying daemon osd.5 on smithi154 2024-06-06T04:23:31.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:30 smithi045 ceph-mon[32299]: Deploying daemon osd.5 on smithi154 2024-06-06T04:23:31.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:30 smithi086 ceph-mon[36053]: Deploying daemon osd.5 on smithi154 2024-06-06T04:23:32.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:31 smithi045 ceph-mon[32299]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:32.237 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:31 smithi154 ceph-mon[36428]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:32.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:31 smithi086 ceph-mon[36053]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:34.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:33 smithi045 ceph-mon[32299]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:34.246 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:33 smithi154 ceph-mon[36428]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:34.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:33 smithi086 ceph-mon[36053]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:36.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:35 smithi154 ceph-mon[36428]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:36.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:35 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:36.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:35 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:36.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:35 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:23:36.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:35 smithi045 ceph-mon[32299]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:36.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:35 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:36.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:35 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:36.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:35 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:23:36.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:35 smithi086 ceph-mon[36053]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:36.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:35 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:36.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:35 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:36.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:35 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:23:38.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:37 smithi154 ceph-mon[36428]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:38.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:37 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:38.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:37 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:38.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:37 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:38.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:37 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:23:38.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:37 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:38.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:37 smithi045 ceph-mon[32299]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:38.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:37 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:38.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:37 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:38.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:37 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:38.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:37 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:23:38.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:37 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:38.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:37 smithi086 ceph-mon[36053]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:38.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:37 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:38.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:37 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:38.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:37 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:38.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:37 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:23:38.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:37 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:38.709 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 5 on host 'smithi154' 2024-06-06T04:23:38.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:38.708+0000 7f2f4dffb700 1 -- 172.21.15.154:0/2325668168 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2f500feb70 con 0x7f2f400a2c20 2024-06-06T04:23:38.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:38.711+0000 7f2f56d6b700 1 -- 172.21.15.154:0/2325668168 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2f400a2c20 msgr2=0x7f2f400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:23:38.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:38.711+0000 7f2f56d6b700 1 --2- 172.21.15.154:0/2325668168 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2f400a2c20 0x7f2f400a50e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c005d90 tx=0x7f2f3c005d00 comp rx=0 tx=0).stop 2024-06-06T04:23:38.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:38.711+0000 7f2f56d6b700 1 -- 172.21.15.154:0/2325668168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f50101410 msgr2=0x7f2f501053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:23:38.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:38.711+0000 7f2f56d6b700 1 --2- 172.21.15.154:0/2325668168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f50101410 0x7f2f501053a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400d980 tx=0x7f2f4400dd40 comp rx=0 tx=0).stop 2024-06-06T04:23:38.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:38.711+0000 7f2f56d6b700 1 -- 172.21.15.154:0/2325668168 shutdown_connections 2024-06-06T04:23:38.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:38.711+0000 7f2f56d6b700 1 --2- 172.21.15.154:0/2325668168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f50102fb0 0x7f2f50103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:38.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:38.711+0000 7f2f56d6b700 1 --2- 172.21.15.154:0/2325668168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f50101410 0x7f2f501053a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:38.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:38.711+0000 7f2f56d6b700 1 --2- 172.21.15.154:0/2325668168 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2f400a2c20 0x7f2f400a50e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:38.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:38.711+0000 7f2f56d6b700 1 --2- 172.21.15.154:0/2325668168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f50102610 0x7f2f501039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:38.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:38.711+0000 7f2f56d6b700 1 -- 172.21.15.154:0/2325668168 >> 172.21.15.154:0/2325668168 conn(0x7f2f500fc9f0 msgr2=0x7f2f500fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:23:38.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:38.711+0000 7f2f56d6b700 1 -- 172.21.15.154:0/2325668168 shutdown_connections 2024-06-06T04:23:38.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:23:38.711+0000 7f2f56d6b700 1 -- 172.21.15.154:0/2325668168 wait complete. 2024-06-06T04:23:39.382 DEBUG:teuthology.orchestra.run.smithi154:osd.5> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.5.service 2024-06-06T04:23:39.385 INFO:tasks.cephadm:Deploying osd.6 on smithi045 with /dev/vg_nvme/lv_2... 2024-06-06T04:23:39.385 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-06T04:23:39.641 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:23:39.926 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:39 smithi045 ceph-mon[32299]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:39.927 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:39 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:39.927 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:39 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:39.927 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:39 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:23:39.927 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:39 smithi045 ceph-mon[32299]: from='osd.5 [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T04:23:39.927 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:39 smithi045 ceph-mon[32299]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T04:23:39.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:39 smithi154 ceph-mon[36428]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:39.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:39 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:39.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:39 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:39.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:39 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:23:39.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:39 smithi154 ceph-mon[36428]: from='osd.5 [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T04:23:39.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:39 smithi154 ceph-mon[36428]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T04:23:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:39 smithi086 ceph-mon[36053]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:39 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:39 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:39 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:23:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:39 smithi086 ceph-mon[36053]: from='osd.5 [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T04:23:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:39 smithi086 ceph-mon[36053]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T04:23:40.774 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:40 smithi154 ceph-mon[36428]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-06T04:23:40.774 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:40 smithi154 ceph-mon[36428]: osdmap e37: 6 total, 5 up, 6 in 2024-06-06T04:23:40.774 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:40 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T04:23:40.774 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:40 smithi154 ceph-mon[36428]: from='osd.5 [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:23:40.774 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:40 smithi154 ceph-mon[36428]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:23:40.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:40 smithi045 ceph-mon[32299]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-06T04:23:40.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:40 smithi045 ceph-mon[32299]: osdmap e37: 6 total, 5 up, 6 in 2024-06-06T04:23:40.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:40 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T04:23:40.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:40 smithi045 ceph-mon[32299]: from='osd.5 [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:23:40.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:40 smithi045 ceph-mon[32299]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:23:41.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:40 smithi086 ceph-mon[36053]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-06T04:23:41.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:40 smithi086 ceph-mon[36053]: osdmap e37: 6 total, 5 up, 6 in 2024-06-06T04:23:41.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:40 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T04:23:41.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:40 smithi086 ceph-mon[36053]: from='osd.5 [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:23:41.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:40 smithi086 ceph-mon[36053]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:23:41.430 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:23:41 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:23:41.110+0000 7ffa96b67700 -1 osd.5 0 waiting for initial osdmap 2024-06-06T04:23:41.430 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:23:41 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:23:41.118+0000 7ffa8e952700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T04:23:41.756 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-06T04:23:41.756 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-06T04:23:41.756 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-06-06T04:23:41.756 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-06-06T04:23:41.756 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00855767 s, 1.2 GB/s 2024-06-06T04:23:41.756 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-06-06T04:23:41.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:41 smithi154 ceph-mon[36428]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:41.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:41 smithi154 ceph-mon[36428]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-06-06T04:23:41.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:41 smithi154 ceph-mon[36428]: osdmap e38: 6 total, 5 up, 6 in 2024-06-06T04:23:41.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:41 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T04:23:41.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:41 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T04:23:41.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:41 smithi154 ceph-mon[36428]: from='osd.5 ' entity='osd.5' 2024-06-06T04:23:41.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:41 smithi154 ceph-mon[36428]: osd.5 [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646] boot 2024-06-06T04:23:41.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:41 smithi154 ceph-mon[36428]: osdmap e39: 6 total, 6 up, 6 in 2024-06-06T04:23:41.891 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:41 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T04:23:41.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:41 smithi045 ceph-mon[32299]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:41.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:41 smithi045 ceph-mon[32299]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-06-06T04:23:41.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:41 smithi045 ceph-mon[32299]: osdmap e38: 6 total, 5 up, 6 in 2024-06-06T04:23:41.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T04:23:41.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T04:23:41.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:41 smithi045 ceph-mon[32299]: from='osd.5 ' entity='osd.5' 2024-06-06T04:23:41.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:41 smithi045 ceph-mon[32299]: osd.5 [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646] boot 2024-06-06T04:23:41.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:41 smithi045 ceph-mon[32299]: osdmap e39: 6 total, 6 up, 6 in 2024-06-06T04:23:41.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T04:23:42.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:41 smithi086 ceph-mon[36053]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-06-06T04:23:42.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:41 smithi086 ceph-mon[36053]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-06-06T04:23:42.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:41 smithi086 ceph-mon[36053]: osdmap e38: 6 total, 5 up, 6 in 2024-06-06T04:23:42.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:41 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T04:23:42.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:41 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T04:23:42.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:41 smithi086 ceph-mon[36053]: from='osd.5 ' entity='osd.5' 2024-06-06T04:23:42.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:41 smithi086 ceph-mon[36053]: osd.5 [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646] boot 2024-06-06T04:23:42.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:41 smithi086 ceph-mon[36053]: osdmap e39: 6 total, 6 up, 6 in 2024-06-06T04:23:42.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:41 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T04:23:42.467 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_2 2024-06-06T04:23:42.720 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:23:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:42 smithi045 ceph-mon[32299]: purged_snaps scrub starts 2024-06-06T04:23:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:42 smithi045 ceph-mon[32299]: purged_snaps scrub ok 2024-06-06T04:23:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:42 smithi045 ceph-mon[32299]: Detected new or changed devices on smithi154 2024-06-06T04:23:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:42 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:42 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:42 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:23:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:42 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:23:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:42 smithi045 ceph-mon[32299]: Adjusting osd_memory_target on smithi154 to 8544M 2024-06-06T04:23:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:42 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:42 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:42 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:23:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:42 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:42.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:42 smithi045 ceph-mon[32299]: osdmap e40: 6 total, 6 up, 6 in 2024-06-06T04:23:43.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:42 smithi086 ceph-mon[36053]: purged_snaps scrub starts 2024-06-06T04:23:43.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:42 smithi086 ceph-mon[36053]: purged_snaps scrub ok 2024-06-06T04:23:43.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:42 smithi086 ceph-mon[36053]: Detected new or changed devices on smithi154 2024-06-06T04:23:43.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:42 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:43.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:42 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:43.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:42 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:23:43.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:42 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:23:43.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:42 smithi086 ceph-mon[36053]: Adjusting osd_memory_target on smithi154 to 8544M 2024-06-06T04:23:43.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:42 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:43.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:42 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:43.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:42 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:23:43.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:42 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:43.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:42 smithi086 ceph-mon[36053]: osdmap e40: 6 total, 6 up, 6 in 2024-06-06T04:23:43.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:42 smithi154 ceph-mon[36428]: purged_snaps scrub starts 2024-06-06T04:23:43.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:42 smithi154 ceph-mon[36428]: purged_snaps scrub ok 2024-06-06T04:23:43.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:42 smithi154 ceph-mon[36428]: Detected new or changed devices on smithi154 2024-06-06T04:23:43.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:42 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:43.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:42 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:43.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:42 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:23:43.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:42 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:23:43.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:42 smithi154 ceph-mon[36428]: Adjusting osd_memory_target on smithi154 to 8544M 2024-06-06T04:23:43.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:42 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:43.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:42 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:43.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:42 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:23:43.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:42 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:23:43.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:42 smithi154 ceph-mon[36428]: osdmap e40: 6 total, 6 up, 6 in 2024-06-06T04:23:43.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:43 smithi045 ceph-mon[32299]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:23:44.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:43 smithi086 ceph-mon[36053]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:23:44.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:43 smithi154 ceph-mon[36428]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:23:44.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.325+0000 7fec0c76f700 1 -- 172.21.15.45:0/1021676227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec04102500 msgr2=0x7fec04102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:23:44.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.325+0000 7fec0c76f700 1 --2- 172.21.15.45:0/1021676227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec04102500 0x7fec04102920 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fec000099c0 tx=0x7fec00009cd0 comp rx=0 tx=0).stop 2024-06-06T04:23:44.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.325+0000 7fec0c76f700 1 -- 172.21.15.45:0/1021676227 shutdown_connections 2024-06-06T04:23:44.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.325+0000 7fec0c76f700 1 --2- 172.21.15.45:0/1021676227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec041040a0 0x7fec0410a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:44.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.325+0000 7fec0c76f700 1 --2- 172.21.15.45:0/1021676227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec04102500 0x7fec04102920 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:44.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.325+0000 7fec0c76f700 1 --2- 172.21.15.45:0/1021676227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec04103700 0x7fec04103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:44.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.325+0000 7fec0c76f700 1 -- 172.21.15.45:0/1021676227 >> 172.21.15.45:0/1021676227 conn(0x7fec040fda60 msgr2=0x7fec040ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:23:44.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.325+0000 7fec0c76f700 1 -- 172.21.15.45:0/1021676227 shutdown_connections 2024-06-06T04:23:44.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.325+0000 7fec0c76f700 1 -- 172.21.15.45:0/1021676227 wait complete. 2024-06-06T04:23:44.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.326+0000 7fec0c76f700 1 Processor -- start 2024-06-06T04:23:44.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.326+0000 7fec0c76f700 1 -- start start 2024-06-06T04:23:44.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.326+0000 7fec0c76f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec04102500 0x7fec041986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:23:44.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.326+0000 7fec0c76f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec04103700 0x7fec04198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:23:44.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.326+0000 7fec0c76f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec041040a0 0x7fec0419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:23:44.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.326+0000 7fec0c76f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fec0410dff0 con 0x7fec04102500 2024-06-06T04:23:44.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.326+0000 7fec0c76f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec0410de70 con 0x7fec04103700 2024-06-06T04:23:44.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.326+0000 7fec0c76f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fec0410e170 con 0x7fec041040a0 2024-06-06T04:23:44.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.326+0000 7fec0a50b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec04102500 0x7fec041986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:23:44.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.326+0000 7fec0a50b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec04102500 0x7fec041986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35182/0 (socket says 172.21.15.45:35182) 2024-06-06T04:23:44.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.326+0000 7fec0a50b700 1 -- 172.21.15.45:0/317202617 learned_addr learned my addr 172.21.15.45:0/317202617 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:23:44.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.326+0000 7fec0a50b700 1 -- 172.21.15.45:0/317202617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec041040a0 msgr2=0x7fec0419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:23:44.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.326+0000 7fec09d0a700 1 --2- 172.21.15.45:0/317202617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec04103700 0x7fec04198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:23:44.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.326+0000 7fec0ad0c700 1 --2- 172.21.15.45:0/317202617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec041040a0 0x7fec0419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:23:44.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.327+0000 7fec0a50b700 1 --2- 172.21.15.45:0/317202617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec041040a0 0x7fec0419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:44.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.327+0000 7fec0a50b700 1 -- 172.21.15.45:0/317202617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec04103700 msgr2=0x7fec04198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:23:44.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.327+0000 7fec0a50b700 1 --2- 172.21.15.45:0/317202617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec04103700 0x7fec04198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:23:44.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.327+0000 7fec0a50b700 1 -- 172.21.15.45:0/317202617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec041a0320 con 0x7fec04102500 2024-06-06T04:23:44.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.327+0000 7fec09d0a700 1 --2- 172.21.15.45:0/317202617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec04103700 0x7fec04198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:23:44.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.327+0000 7fec0ad0c700 1 --2- 172.21.15.45:0/317202617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec041040a0 0x7fec0419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:23:44.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.327+0000 7fec0a50b700 1 --2- 172.21.15.45:0/317202617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec04102500 0x7fec041986c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fec0000c010 tx=0x7fec00003710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:23:44.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.327+0000 7febfb7fe700 1 -- 172.21.15.45:0/317202617 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00027070 con 0x7fec04102500 2024-06-06T04:23:44.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.327+0000 7febfb7fe700 1 -- 172.21.15.45:0/317202617 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec00022470 con 0x7fec04102500 2024-06-06T04:23:44.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.327+0000 7fec0c76f700 1 -- 172.21.15.45:0/317202617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec041a05b0 con 0x7fec04102500 2024-06-06T04:23:44.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.327+0000 7febfb7fe700 1 -- 172.21.15.45:0/317202617 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0000f6c0 con 0x7fec04102500 2024-06-06T04:23:44.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.327+0000 7fec0c76f700 1 -- 172.21.15.45:0/317202617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec041a0a00 con 0x7fec04102500 2024-06-06T04:23:44.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.328+0000 7fec0c76f700 1 -- 172.21.15.45:0/317202617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec04069180 con 0x7fec04102500 2024-06-06T04:23:44.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.329+0000 7febfb7fe700 1 -- 172.21.15.45:0/317202617 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fec0002a070 con 0x7fec04102500 2024-06-06T04:23:44.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.330+0000 7febfb7fe700 1 --2- 172.21.15.45:0/317202617 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7febf00a2c20 0x7febf00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:23:44.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.330+0000 7fec09d0a700 1 --2- 172.21.15.45:0/317202617 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7febf00a2c20 0x7febf00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:23:44.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.330+0000 7febfb7fe700 1 -- 172.21.15.45:0/317202617 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7fec000cf690 con 0x7fec04102500 2024-06-06T04:23:44.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.330+0000 7fec09d0a700 1 --2- 172.21.15.45:0/317202617 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7febf00a2c20 0x7febf00a50e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7febf4005fd0 tx=0x7febf4005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:23:44.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.334+0000 7febfb7fe700 1 -- 172.21.15.45:0/317202617 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fec00031070 con 0x7fec04102500 2024-06-06T04:23:44.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:23:44.550+0000 7fec0c76f700 1 -- 172.21.15.45:0/317202617 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fec040628d0 con 0x7febf00a2c20 2024-06-06T04:23:44.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:44 smithi045 ceph-mon[32299]: osdmap e41: 6 total, 6 up, 6 in 2024-06-06T04:23:44.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:44 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:23:44.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:44 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:23:44.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:44 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:45.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:44 smithi086 ceph-mon[36053]: osdmap e41: 6 total, 6 up, 6 in 2024-06-06T04:23:45.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:44 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:23:45.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:44 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:23:45.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:44 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:45.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:44 smithi154 ceph-mon[36428]: osdmap e41: 6 total, 6 up, 6 in 2024-06-06T04:23:45.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:44 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:23:45.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:44 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:23:45.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:44 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:45.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:45 smithi045 ceph-mon[32299]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:23:45.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:45 smithi045 ceph-mon[32299]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:23:45.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:45 smithi045 ceph-mon[32299]: osdmap e42: 6 total, 6 up, 6 in 2024-06-06T04:23:46.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:45 smithi086 ceph-mon[36053]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:23:46.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:45 smithi086 ceph-mon[36053]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:23:46.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:45 smithi086 ceph-mon[36053]: osdmap e42: 6 total, 6 up, 6 in 2024-06-06T04:23:46.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:45 smithi154 ceph-mon[36428]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:23:46.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:45 smithi154 ceph-mon[36428]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:23:46.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:45 smithi154 ceph-mon[36428]: osdmap e42: 6 total, 6 up, 6 in 2024-06-06T04:23:46.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:46 smithi045 ceph-mon[32299]: osdmap e43: 6 total, 6 up, 6 in 2024-06-06T04:23:47.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:46 smithi086 ceph-mon[36053]: osdmap e43: 6 total, 6 up, 6 in 2024-06-06T04:23:47.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:46 smithi154 ceph-mon[36428]: osdmap e43: 6 total, 6 up, 6 in 2024-06-06T04:23:47.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:47 smithi045 ceph-mon[32299]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:23:47.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:47 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/340900547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c70b627-eafa-4e27-ba2a-4d8e1e335962"}]: dispatch 2024-06-06T04:23:47.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:47 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/340900547' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c70b627-eafa-4e27-ba2a-4d8e1e335962"}]': finished 2024-06-06T04:23:47.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:47 smithi045 ceph-mon[32299]: osdmap e44: 7 total, 6 up, 7 in 2024-06-06T04:23:47.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:47 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:23:48.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:47 smithi086 ceph-mon[36053]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:23:48.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:47 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/340900547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c70b627-eafa-4e27-ba2a-4d8e1e335962"}]: dispatch 2024-06-06T04:23:48.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:47 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/340900547' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c70b627-eafa-4e27-ba2a-4d8e1e335962"}]': finished 2024-06-06T04:23:48.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:47 smithi086 ceph-mon[36053]: osdmap e44: 7 total, 6 up, 7 in 2024-06-06T04:23:48.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:47 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:23:48.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:47 smithi154 ceph-mon[36428]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:23:48.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:47 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/340900547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c70b627-eafa-4e27-ba2a-4d8e1e335962"}]: dispatch 2024-06-06T04:23:48.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:47 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/340900547' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c70b627-eafa-4e27-ba2a-4d8e1e335962"}]': finished 2024-06-06T04:23:48.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:47 smithi154 ceph-mon[36428]: osdmap e44: 7 total, 6 up, 7 in 2024-06-06T04:23:48.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:47 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:23:48.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:48 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2321916947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:23:49.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:48 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2321916947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:23:49.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:48 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2321916947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:23:49.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:49 smithi045 ceph-mon[32299]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:23:50.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:49 smithi086 ceph-mon[36053]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:23:50.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:49 smithi154 ceph-mon[36428]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:23:51.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:51 smithi045 ceph-mon[32299]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T04:23:52.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:51 smithi086 ceph-mon[36053]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T04:23:52.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:51 smithi154 ceph-mon[36428]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T04:23:53.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:53 smithi045 ceph-mon[32299]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-06-06T04:23:54.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:53 smithi086 ceph-mon[36053]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-06-06T04:23:54.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:53 smithi154 ceph-mon[36428]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-06-06T04:23:55.842 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:55 smithi045 ceph-mon[32299]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-06-06T04:23:55.842 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:55 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-06T04:23:55.842 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:55 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:55.842 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:55 smithi045 ceph-mon[32299]: Deploying daemon osd.6 on smithi045 2024-06-06T04:23:56.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:55 smithi086 ceph-mon[36053]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-06-06T04:23:56.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:55 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-06T04:23:56.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:55 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:56.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:55 smithi086 ceph-mon[36053]: Deploying daemon osd.6 on smithi045 2024-06-06T04:23:56.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:55 smithi154 ceph-mon[36428]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-06-06T04:23:56.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:55 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-06T04:23:56.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:55 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:23:56.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:55 smithi154 ceph-mon[36428]: Deploying daemon osd.6 on smithi045 2024-06-06T04:23:57.797 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:57 smithi045 ceph-mon[32299]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T04:23:58.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:57 smithi086 ceph-mon[36053]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T04:23:58.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:57 smithi154 ceph-mon[36428]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T04:23:59.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:23:59 smithi045 ceph-mon[32299]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-06T04:24:00.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:23:59 smithi086 ceph-mon[36053]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-06T04:24:00.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:23:59 smithi154 ceph-mon[36428]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-06T04:24:02.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:02 smithi086 ceph-mon[36053]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-06T04:24:02.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:02.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:02.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:02.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:02 smithi154 ceph-mon[36428]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-06T04:24:02.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:02.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:02.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:02.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:02 smithi045 ceph-mon[32299]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-06T04:24:02.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:02.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:02.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:04.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:04 smithi045 ceph-mon[32299]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:24:04.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:04 smithi086 ceph-mon[36053]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:24:04.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:04 smithi154 ceph-mon[36428]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:24:04.668 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 6 on host 'smithi045' 2024-06-06T04:24:04.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:24:04.667+0000 7febfb7fe700 1 -- 172.21.15.45:0/317202617 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fec040628d0 con 0x7febf00a2c20 2024-06-06T04:24:04.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:24:04.670+0000 7fec0c76f700 1 -- 172.21.15.45:0/317202617 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7febf00a2c20 msgr2=0x7febf00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:24:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:24:04.670+0000 7fec0c76f700 1 --2- 172.21.15.45:0/317202617 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7febf00a2c20 0x7febf00a50e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7febf4005fd0 tx=0x7febf4005dc0 comp rx=0 tx=0).stop 2024-06-06T04:24:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:24:04.670+0000 7fec0c76f700 1 -- 172.21.15.45:0/317202617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec04102500 msgr2=0x7fec041986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:24:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:24:04.670+0000 7fec0c76f700 1 --2- 172.21.15.45:0/317202617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec04102500 0x7fec041986c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fec0000c010 tx=0x7fec00003710 comp rx=0 tx=0).stop 2024-06-06T04:24:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:24:04.670+0000 7fec0c76f700 1 -- 172.21.15.45:0/317202617 shutdown_connections 2024-06-06T04:24:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:24:04.670+0000 7fec0c76f700 1 --2- 172.21.15.45:0/317202617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec041040a0 0x7fec0419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:24:04.670+0000 7fec0c76f700 1 --2- 172.21.15.45:0/317202617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec04102500 0x7fec041986c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:24:04.670+0000 7fec0c76f700 1 --2- 172.21.15.45:0/317202617 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7febf00a2c20 0x7febf00a50e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:24:04.670+0000 7fec0c76f700 1 --2- 172.21.15.45:0/317202617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec04103700 0x7fec04198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:24:04.670+0000 7fec0c76f700 1 -- 172.21.15.45:0/317202617 >> 172.21.15.45:0/317202617 conn(0x7fec040fda60 msgr2=0x7fec04106aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:24:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:24:04.670+0000 7fec0c76f700 1 -- 172.21.15.45:0/317202617 shutdown_connections 2024-06-06T04:24:04.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:24:04.670+0000 7fec0c76f700 1 -- 172.21.15.45:0/317202617 wait complete. 2024-06-06T04:24:05.453 DEBUG:teuthology.orchestra.run.smithi045:osd.6> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.6.service 2024-06-06T04:24:05.455 INFO:tasks.cephadm:Deploying osd.7 on smithi086 with /dev/vg_nvme/lv_2... 2024-06-06T04:24:05.456 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-06T04:24:05.704 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T04:24:05.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:05 smithi045 ceph-mon[32299]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:24:05.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:05.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:05.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:05 smithi045 ceph-mon[32299]: from='osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-06T04:24:05.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:05.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:05.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:05 smithi086 ceph-mon[36053]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:24:05.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:05.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:05.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:05 smithi086 ceph-mon[36053]: from='osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-06T04:24:05.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:05.827 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:05.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:05 smithi154 ceph-mon[36428]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:24:05.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:05.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:05.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:05 smithi154 ceph-mon[36428]: from='osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-06T04:24:05.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:05.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:06.925 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:06 smithi086 ceph-mon[36053]: from='osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-06T04:24:06.926 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:06 smithi086 ceph-mon[36053]: osdmap e45: 7 total, 6 up, 7 in 2024-06-06T04:24:06.926 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:06 smithi086 ceph-mon[36053]: from='osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-06-06T04:24:06.926 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:06 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:24:06.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:06 smithi154 ceph-mon[36428]: from='osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-06T04:24:06.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:06 smithi154 ceph-mon[36428]: osdmap e45: 7 total, 6 up, 7 in 2024-06-06T04:24:06.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:06 smithi154 ceph-mon[36428]: from='osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-06-06T04:24:06.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:06 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:24:06.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:06 smithi045 ceph-mon[32299]: from='osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-06T04:24:06.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:06 smithi045 ceph-mon[32299]: osdmap e45: 7 total, 6 up, 7 in 2024-06-06T04:24:06.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:06 smithi045 ceph-mon[32299]: from='osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-06-06T04:24:06.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:06 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:24:07.429 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:24:07 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:24:07.122+0000 7fcab5df4700 -1 osd.6 0 waiting for initial osdmap 2024-06-06T04:24:07.429 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:24:07 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:24:07.131+0000 7fcaad3de700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T04:24:07.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:07 smithi045 ceph-mon[32299]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:24:07.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:07 smithi045 ceph-mon[32299]: from='osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-06-06T04:24:07.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:07 smithi045 ceph-mon[32299]: osdmap e46: 7 total, 6 up, 7 in 2024-06-06T04:24:07.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:24:07.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:24:07.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:07 smithi045 ceph-mon[32299]: from='osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939]' entity='osd.6' 2024-06-06T04:24:07.819 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-06T04:24:07.819 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-06T04:24:07.819 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-06-06T04:24:07.819 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-06-06T04:24:07.820 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0130002 s, 807 MB/s 2024-06-06T04:24:07.820 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-06-06T04:24:07.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:07 smithi154 ceph-mon[36428]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:24:07.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:07 smithi154 ceph-mon[36428]: from='osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-06-06T04:24:07.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:07 smithi154 ceph-mon[36428]: osdmap e46: 7 total, 6 up, 7 in 2024-06-06T04:24:07.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:07 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:24:07.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:07 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:24:07.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:07 smithi154 ceph-mon[36428]: from='osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939]' entity='osd.6' 2024-06-06T04:24:08.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:07 smithi086 ceph-mon[36053]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:24:08.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:07 smithi086 ceph-mon[36053]: from='osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-06-06T04:24:08.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:07 smithi086 ceph-mon[36053]: osdmap e46: 7 total, 6 up, 7 in 2024-06-06T04:24:08.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:07 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:24:08.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:07 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:24:08.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:07 smithi086 ceph-mon[36053]: from='osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939]' entity='osd.6' 2024-06-06T04:24:08.478 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_2 2024-06-06T04:24:08.718 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T04:24:08.753 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:08 smithi086 ceph-mon[36053]: purged_snaps scrub starts 2024-06-06T04:24:08.753 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:08 smithi086 ceph-mon[36053]: purged_snaps scrub ok 2024-06-06T04:24:08.753 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:08 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:24:08.754 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:08 smithi086 ceph-mon[36053]: osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939] boot 2024-06-06T04:24:08.754 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:08 smithi086 ceph-mon[36053]: osdmap e47: 7 total, 7 up, 7 in 2024-06-06T04:24:08.754 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:08 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:24:08.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:08 smithi154 ceph-mon[36428]: purged_snaps scrub starts 2024-06-06T04:24:08.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:08 smithi154 ceph-mon[36428]: purged_snaps scrub ok 2024-06-06T04:24:08.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:08 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:24:08.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:08 smithi154 ceph-mon[36428]: osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939] boot 2024-06-06T04:24:08.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:08 smithi154 ceph-mon[36428]: osdmap e47: 7 total, 7 up, 7 in 2024-06-06T04:24:08.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:08 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:24:08.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:08 smithi045 ceph-mon[32299]: purged_snaps scrub starts 2024-06-06T04:24:08.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:08 smithi045 ceph-mon[32299]: purged_snaps scrub ok 2024-06-06T04:24:08.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:08 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:24:08.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:08 smithi045 ceph-mon[32299]: osd.6 [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939] boot 2024-06-06T04:24:08.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:08 smithi045 ceph-mon[32299]: osdmap e47: 7 total, 7 up, 7 in 2024-06-06T04:24:08.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:08 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T04:24:10.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:10 smithi086 ceph-mon[36053]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:24:10.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:10.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:10.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:10.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:10.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:10.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:10.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:10.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:24:10.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:10.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:10 smithi086 ceph-mon[36053]: osdmap e48: 7 total, 7 up, 7 in 2024-06-06T04:24:10.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:10.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:10.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:24:10.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:10.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:10 smithi154 ceph-mon[36428]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:24:10.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:10.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:10.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:10.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:10.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:10.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:10.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:10.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:24:10.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:10.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:10 smithi154 ceph-mon[36428]: osdmap e48: 7 total, 7 up, 7 in 2024-06-06T04:24:10.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:10.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:10.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:24:10.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:10.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.439+0000 7f7d2c147700 1 -- 172.21.15.86:0/2227432143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d241025e0 msgr2=0x7f7d24102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:24:10.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.439+0000 7f7d2c147700 1 --2- 172.21.15.86:0/2227432143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d241025e0 0x7f7d24102a40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7d140099c0 tx=0x7f7d14009cd0 comp rx=0 tx=0).stop 2024-06-06T04:24:10.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.439+0000 7f7d2c147700 1 -- 172.21.15.86:0/2227432143 shutdown_connections 2024-06-06T04:24:10.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.439+0000 7f7d2c147700 1 --2- 172.21.15.86:0/2227432143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d24102f80 0x7f7d24109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:10.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.439+0000 7f7d2c147700 1 --2- 172.21.15.86:0/2227432143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d241013e0 0x7f7d24101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:10.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.439+0000 7f7d2c147700 1 --2- 172.21.15.86:0/2227432143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d241025e0 0x7f7d24102a40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:10.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.439+0000 7f7d2c147700 1 -- 172.21.15.86:0/2227432143 >> 172.21.15.86:0/2227432143 conn(0x7f7d240fc980 msgr2=0x7f7d240fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:24:10.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.439+0000 7f7d2c147700 1 -- 172.21.15.86:0/2227432143 shutdown_connections 2024-06-06T04:24:10.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.439+0000 7f7d2c147700 1 -- 172.21.15.86:0/2227432143 wait complete. 2024-06-06T04:24:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.440+0000 7f7d2c147700 1 Processor -- start 2024-06-06T04:24:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.440+0000 7f7d2c147700 1 -- start start 2024-06-06T04:24:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.440+0000 7f7d2c147700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d241013e0 0x7f7d24197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:24:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.440+0000 7f7d2c147700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d241025e0 0x7f7d24197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:24:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.440+0000 7f7d2c147700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d24102f80 0x7f7d2419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:24:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.440+0000 7f7d2c147700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410ce60 con 0x7f7d24102f80 2024-06-06T04:24:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.440+0000 7f7d2c147700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410cce0 con 0x7f7d241025e0 2024-06-06T04:24:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.440+0000 7f7d2c147700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410cfe0 con 0x7f7d241013e0 2024-06-06T04:24:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.440+0000 7f7d296e2700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d241025e0 0x7f7d24197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:24:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.440+0000 7f7d296e2700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d241025e0 0x7f7d24197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57376/0 (socket says 172.21.15.86:57376) 2024-06-06T04:24:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.440+0000 7f7d296e2700 1 -- 172.21.15.86:0/1123209342 learned_addr learned my addr 172.21.15.86:0/1123209342 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-06-06T04:24:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.441+0000 7f7d296e2700 1 -- 172.21.15.86:0/1123209342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d241013e0 msgr2=0x7f7d24197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:24:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.441+0000 7f7d29ee3700 1 --2- 172.21.15.86:0/1123209342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d241013e0 0x7f7d24197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:24:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.441+0000 7f7d296e2700 1 --2- 172.21.15.86:0/1123209342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d241013e0 0x7f7d24197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.441+0000 7f7d296e2700 1 -- 172.21.15.86:0/1123209342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d24102f80 msgr2=0x7f7d2419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:24:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.441+0000 7f7d296e2700 1 --2- 172.21.15.86:0/1123209342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d24102f80 0x7f7d2419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.441+0000 7f7d296e2700 1 -- 172.21.15.86:0/1123209342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2419f190 con 0x7f7d241025e0 2024-06-06T04:24:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.441+0000 7f7d296e2700 1 --2- 172.21.15.86:0/1123209342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d241025e0 0x7f7d24197a70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7d140098e0 tx=0x7f7d14009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:24:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.441+0000 7f7d1affd700 1 -- 172.21.15.86:0/1123209342 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14027070 con 0x7f7d241025e0 2024-06-06T04:24:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.441+0000 7f7d1affd700 1 -- 172.21.15.86:0/1123209342 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d14004050 con 0x7f7d241025e0 2024-06-06T04:24:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.441+0000 7f7d1affd700 1 -- 172.21.15.86:0/1123209342 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1400f460 con 0x7f7d241025e0 2024-06-06T04:24:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.441+0000 7f7d2c147700 1 -- 172.21.15.86:0/1123209342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2419f420 con 0x7f7d241025e0 2024-06-06T04:24:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.441+0000 7f7d2c147700 1 -- 172.21.15.86:0/1123209342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2419f8d0 con 0x7f7d241025e0 2024-06-06T04:24:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.442+0000 7f7d2c147700 1 -- 172.21.15.86:0/1123209342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d24069180 con 0x7f7d241025e0 2024-06-06T04:24:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.443+0000 7f7d1affd700 1 -- 172.21.15.86:0/1123209342 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7d1402a070 con 0x7f7d241025e0 2024-06-06T04:24:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.444+0000 7f7d1affd700 1 --2- 172.21.15.86:0/1123209342 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7d100a2ca0 0x7f7d100a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:24:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.444+0000 7f7d1affd700 1 -- 172.21.15.86:0/1123209342 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f7d140cfb60 con 0x7f7d241025e0 2024-06-06T04:24:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.444+0000 7f7d29ee3700 1 --2- 172.21.15.86:0/1123209342 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7d100a2ca0 0x7f7d100a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:24:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.445+0000 7f7d29ee3700 1 --2- 172.21.15.86:0/1123209342 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7d100a2ca0 0x7f7d100a5160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d24102440 tx=0x7f7d20009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:24:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.448+0000 7f7d1affd700 1 -- 172.21.15.86:0/1123209342 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7d14031020 con 0x7f7d241025e0 2024-06-06T04:24:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:10 smithi045 ceph-mon[32299]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-06T04:24:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:24:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:10 smithi045 ceph-mon[32299]: osdmap e48: 7 total, 7 up, 7 in 2024-06-06T04:24:10.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:10.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:10.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:24:10.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:10.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:10.652+0000 7f7d2c147700 1 -- 172.21.15.86:0/1123209342 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7d240628d0 con 0x7f7d100a2ca0 2024-06-06T04:24:11.411 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:11 smithi086 ceph-mon[36053]: Detected new or changed devices on smithi045 2024-06-06T04:24:11.411 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:11 smithi086 ceph-mon[36053]: Adjusting osd_memory_target on smithi045 to 5697M 2024-06-06T04:24:11.411 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:24:11.411 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:24:11.411 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:11.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:11 smithi154 ceph-mon[36428]: Detected new or changed devices on smithi045 2024-06-06T04:24:11.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:11 smithi154 ceph-mon[36428]: Adjusting osd_memory_target on smithi045 to 5697M 2024-06-06T04:24:11.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:24:11.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:24:11.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:11.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:11 smithi045 ceph-mon[32299]: Detected new or changed devices on smithi045 2024-06-06T04:24:11.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:11 smithi045 ceph-mon[32299]: Adjusting osd_memory_target on smithi045 to 5697M 2024-06-06T04:24:11.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:24:11.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:24:11.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:12.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:12 smithi154 ceph-mon[36428]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:12.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:12 smithi154 ceph-mon[36428]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:24:12.470 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:12 smithi086 ceph-mon[36053]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:12.471 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:12 smithi086 ceph-mon[36053]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:24:12.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:12 smithi045 ceph-mon[32299]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:12.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:12 smithi045 ceph-mon[32299]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:24:14.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:14 smithi154 ceph-mon[36428]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:14.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:14 smithi154 ceph-mon[36428]: from='client.? 172.21.15.86:0/2039726835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b09b875-be52-4be0-b116-7fd9b0c26705"}]: dispatch 2024-06-06T04:24:14.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:14 smithi154 ceph-mon[36428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b09b875-be52-4be0-b116-7fd9b0c26705"}]: dispatch 2024-06-06T04:24:14.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:14 smithi154 ceph-mon[36428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b09b875-be52-4be0-b116-7fd9b0c26705"}]': finished 2024-06-06T04:24:14.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:14 smithi154 ceph-mon[36428]: osdmap e49: 8 total, 7 up, 8 in 2024-06-06T04:24:14.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T04:24:14.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:14 smithi154 ceph-mon[36428]: from='client.? 172.21.15.86:0/3242333436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:24:14.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:14 smithi045 ceph-mon[32299]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:14.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:14 smithi045 ceph-mon[32299]: from='client.? 172.21.15.86:0/2039726835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b09b875-be52-4be0-b116-7fd9b0c26705"}]: dispatch 2024-06-06T04:24:14.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:14 smithi045 ceph-mon[32299]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b09b875-be52-4be0-b116-7fd9b0c26705"}]: dispatch 2024-06-06T04:24:14.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:14 smithi045 ceph-mon[32299]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b09b875-be52-4be0-b116-7fd9b0c26705"}]': finished 2024-06-06T04:24:14.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:14 smithi045 ceph-mon[32299]: osdmap e49: 8 total, 7 up, 8 in 2024-06-06T04:24:14.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T04:24:14.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:14 smithi045 ceph-mon[32299]: from='client.? 172.21.15.86:0/3242333436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:24:14.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:14 smithi086 ceph-mon[36053]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:14.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:14 smithi086 ceph-mon[36053]: from='client.? 172.21.15.86:0/2039726835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b09b875-be52-4be0-b116-7fd9b0c26705"}]: dispatch 2024-06-06T04:24:14.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:14 smithi086 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b09b875-be52-4be0-b116-7fd9b0c26705"}]: dispatch 2024-06-06T04:24:14.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:14 smithi086 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b09b875-be52-4be0-b116-7fd9b0c26705"}]': finished 2024-06-06T04:24:14.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:14 smithi086 ceph-mon[36053]: osdmap e49: 8 total, 7 up, 8 in 2024-06-06T04:24:14.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T04:24:14.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:14 smithi086 ceph-mon[36053]: from='client.? 172.21.15.86:0/3242333436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:24:16.411 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:16 smithi086 ceph-mon[36053]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:16.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:16 smithi154 ceph-mon[36428]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:16.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:16 smithi045 ceph-mon[32299]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:18.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:18 smithi154 ceph-mon[36428]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:18.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:18 smithi086 ceph-mon[36053]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:18.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:18 smithi045 ceph-mon[32299]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:20.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:20 smithi154 ceph-mon[36428]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:20.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:20 smithi045 ceph-mon[32299]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:20.512 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:20 smithi086 ceph-mon[36053]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:21.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:21 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-06T04:24:21.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:21 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:21.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:21 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-06T04:24:21.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:21 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:21.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:21 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-06T04:24:21.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:21 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:22.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:22 smithi154 ceph-mon[36428]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:22.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:22 smithi154 ceph-mon[36428]: Deploying daemon osd.7 on smithi086 2024-06-06T04:24:22.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:22 smithi045 ceph-mon[32299]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:22.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:22 smithi045 ceph-mon[32299]: Deploying daemon osd.7 on smithi086 2024-06-06T04:24:22.484 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:22 smithi086 ceph-mon[36053]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:22.484 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:22 smithi086 ceph-mon[36053]: Deploying daemon osd.7 on smithi086 2024-06-06T04:24:24.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:24 smithi086 ceph-mon[36053]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:24.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:24 smithi154 ceph-mon[36428]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:24.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:24 smithi045 ceph-mon[32299]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:26.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:26 smithi154 ceph-mon[36428]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:26.457 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:26 smithi086 ceph-mon[36053]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:26.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:26 smithi045 ceph-mon[32299]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:27 smithi086 ceph-mon[36053]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:27 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:27 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:27 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:27.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:27 smithi154 ceph-mon[36428]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:27.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:27 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:27.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:27 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:27.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:27 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:27.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:27 smithi045 ceph-mon[32299]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:27.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:27 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:27.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:27 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:27.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:27 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:29.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:29 smithi086 ceph-mon[36053]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:29.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:29 smithi154 ceph-mon[36428]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:29.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:29 smithi045 ceph-mon[32299]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:30.427 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 7 on host 'smithi086' 2024-06-06T04:24:30.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:30.426+0000 7f7d1affd700 1 -- 172.21.15.86:0/1123209342 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7d240628d0 con 0x7f7d100a2ca0 2024-06-06T04:24:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:30.428+0000 7f7d2c147700 1 -- 172.21.15.86:0/1123209342 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7d100a2ca0 msgr2=0x7f7d100a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:24:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:30.428+0000 7f7d2c147700 1 --2- 172.21.15.86:0/1123209342 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7d100a2ca0 0x7f7d100a5160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d24102440 tx=0x7f7d20009040 comp rx=0 tx=0).stop 2024-06-06T04:24:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:30.428+0000 7f7d2c147700 1 -- 172.21.15.86:0/1123209342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d241025e0 msgr2=0x7f7d24197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:24:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:30.428+0000 7f7d2c147700 1 --2- 172.21.15.86:0/1123209342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d241025e0 0x7f7d24197a70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7d140098e0 tx=0x7f7d14009f10 comp rx=0 tx=0).stop 2024-06-06T04:24:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:30.428+0000 7f7d2c147700 1 -- 172.21.15.86:0/1123209342 shutdown_connections 2024-06-06T04:24:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:30.428+0000 7f7d2c147700 1 --2- 172.21.15.86:0/1123209342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d24102f80 0x7f7d2419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:30.428+0000 7f7d2c147700 1 --2- 172.21.15.86:0/1123209342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d241013e0 0x7f7d24197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:30.428+0000 7f7d2c147700 1 --2- 172.21.15.86:0/1123209342 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7d100a2ca0 0x7f7d100a5160 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:30.429+0000 7f7d2c147700 1 --2- 172.21.15.86:0/1123209342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d241025e0 0x7f7d24197a70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:30.429+0000 7f7d2c147700 1 -- 172.21.15.86:0/1123209342 >> 172.21.15.86:0/1123209342 conn(0x7f7d240fc980 msgr2=0x7f7d241059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:24:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:30.429+0000 7f7d2c147700 1 -- 172.21.15.86:0/1123209342 shutdown_connections 2024-06-06T04:24:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:24:30.429+0000 7f7d2c147700 1 -- 172.21.15.86:0/1123209342 wait complete. 2024-06-06T04:24:30.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:30 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:30.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:30 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:30.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:30 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:30.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:30 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:24:30.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:30 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:30.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:30 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:30.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:30 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:30.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:30 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:30.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:30 smithi086 ceph-mon[36053]: from='osd.7 [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T04:24:30.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:30 smithi086 ceph-mon[36053]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T04:24:30.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:30 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:30.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:30 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:30.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:30 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:30.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:30 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:24:30.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:30 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:30.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:30 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:30.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:30 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:30.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:30 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:30.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:30 smithi154 ceph-mon[36428]: from='osd.7 [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T04:24:30.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:30 smithi154 ceph-mon[36428]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T04:24:30.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:30 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:30.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:30 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:30.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:30 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:30.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:30 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:24:30.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:30 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:30.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:30 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:30.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:30 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:30.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:30 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:30.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:30 smithi045 ceph-mon[32299]: from='osd.7 [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T04:24:30.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:30 smithi045 ceph-mon[32299]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T04:24:31.154 DEBUG:teuthology.orchestra.run.smithi086:osd.7> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.7.service 2024-06-06T04:24:31.160 INFO:tasks.cephadm:Deploying osd.8 on smithi154 with /dev/vg_nvme/lv_2... 2024-06-06T04:24:31.160 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-06T04:24:31.409 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T04:24:31.791 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:31 smithi154 ceph-mon[36428]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:31.791 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:31 smithi154 ceph-mon[36428]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-06T04:24:31.791 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:31 smithi154 ceph-mon[36428]: from='osd.7 [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:24:31.791 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:31 smithi154 ceph-mon[36428]: osdmap e50: 8 total, 7 up, 8 in 2024-06-06T04:24:31.791 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:31 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T04:24:31.791 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:31 smithi154 ceph-mon[36428]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:24:31.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:31 smithi086 ceph-mon[36053]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:31.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:31 smithi086 ceph-mon[36053]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-06T04:24:31.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:31 smithi086 ceph-mon[36053]: from='osd.7 [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:24:31.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:31 smithi086 ceph-mon[36053]: osdmap e50: 8 total, 7 up, 8 in 2024-06-06T04:24:31.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:31 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T04:24:31.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:31 smithi086 ceph-mon[36053]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:24:31.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:31 smithi045 ceph-mon[32299]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:31.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:31 smithi045 ceph-mon[32299]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-06T04:24:31.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:31 smithi045 ceph-mon[32299]: from='osd.7 [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:24:31.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:31 smithi045 ceph-mon[32299]: osdmap e50: 8 total, 7 up, 8 in 2024-06-06T04:24:31.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:31 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T04:24:31.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:31 smithi045 ceph-mon[32299]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:24:33.110 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:24:32 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:24:32.810+0000 7f2b69ab1700 -1 osd.7 0 waiting for initial osdmap 2024-06-06T04:24:33.110 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:24:32 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:24:32.818+0000 7f2b6189c700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T04:24:33.454 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:33 smithi154 ceph-mon[36428]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:33.454 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:33 smithi154 ceph-mon[36428]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-06-06T04:24:33.454 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:33 smithi154 ceph-mon[36428]: osdmap e51: 8 total, 7 up, 8 in 2024-06-06T04:24:33.454 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:33 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T04:24:33.454 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:33 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T04:24:33.454 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:33 smithi154 ceph-mon[36428]: from='osd.7 ' entity='osd.7' 2024-06-06T04:24:33.454 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:33 smithi154 ceph-mon[36428]: osd.7 [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005] boot 2024-06-06T04:24:33.454 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:33 smithi154 ceph-mon[36428]: osdmap e52: 8 total, 8 up, 8 in 2024-06-06T04:24:33.455 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:33 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T04:24:33.471 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-06T04:24:33.471 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-06T04:24:33.471 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-06-06T04:24:33.471 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-06-06T04:24:33.471 INFO:teuthology.orchestra.run.smithi154.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0129552 s, 809 MB/s 2024-06-06T04:24:33.471 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-06-06T04:24:33.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:33 smithi045 ceph-mon[32299]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:33.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:33 smithi045 ceph-mon[32299]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-06-06T04:24:33.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:33 smithi045 ceph-mon[32299]: osdmap e51: 8 total, 7 up, 8 in 2024-06-06T04:24:33.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:33 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T04:24:33.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:33 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T04:24:33.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:33 smithi045 ceph-mon[32299]: from='osd.7 ' entity='osd.7' 2024-06-06T04:24:33.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:33 smithi045 ceph-mon[32299]: osd.7 [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005] boot 2024-06-06T04:24:33.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:33 smithi045 ceph-mon[32299]: osdmap e52: 8 total, 8 up, 8 in 2024-06-06T04:24:33.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:33 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T04:24:33.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:33 smithi086 ceph-mon[36053]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-06T04:24:33.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:33 smithi086 ceph-mon[36053]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-06-06T04:24:33.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:33 smithi086 ceph-mon[36053]: osdmap e51: 8 total, 7 up, 8 in 2024-06-06T04:24:33.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:33 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T04:24:33.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:33 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T04:24:33.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:33 smithi086 ceph-mon[36053]: from='osd.7 ' entity='osd.7' 2024-06-06T04:24:33.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:33 smithi086 ceph-mon[36053]: osd.7 [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005] boot 2024-06-06T04:24:33.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:33 smithi086 ceph-mon[36053]: osdmap e52: 8 total, 8 up, 8 in 2024-06-06T04:24:33.826 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:33 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T04:24:34.225 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi154:vg_nvme/lv_2 2024-06-06T04:24:34.470 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T04:24:34.593 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:34 smithi154 ceph-mon[36428]: purged_snaps scrub starts 2024-06-06T04:24:34.593 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:34 smithi154 ceph-mon[36428]: purged_snaps scrub ok 2024-06-06T04:24:34.593 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:34 smithi154 ceph-mon[36428]: osdmap e53: 8 total, 8 up, 8 in 2024-06-06T04:24:34.593 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:34.593 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:34.593 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:34.593 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:34.594 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:34.594 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:34.594 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:34.594 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:24:34.594 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:34 smithi086 ceph-mon[36053]: purged_snaps scrub starts 2024-06-06T04:24:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:34 smithi086 ceph-mon[36053]: purged_snaps scrub ok 2024-06-06T04:24:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:34 smithi086 ceph-mon[36053]: osdmap e53: 8 total, 8 up, 8 in 2024-06-06T04:24:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:34.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:34.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:34.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:34.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:24:34.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:34 smithi045 ceph-mon[32299]: purged_snaps scrub starts 2024-06-06T04:24:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:34 smithi045 ceph-mon[32299]: purged_snaps scrub ok 2024-06-06T04:24:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:34 smithi045 ceph-mon[32299]: osdmap e53: 8 total, 8 up, 8 in 2024-06-06T04:24:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:24:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:24:34.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:35.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:35 smithi045 ceph-mon[32299]: Detected new or changed devices on smithi086 2024-06-06T04:24:35.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:35 smithi045 ceph-mon[32299]: Adjusting osd_memory_target on smithi086 to 5696M 2024-06-06T04:24:35.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:35 smithi045 ceph-mon[32299]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:35.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:35 smithi045 ceph-mon[32299]: osdmap e54: 8 total, 8 up, 8 in 2024-06-06T04:24:36.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.102+0000 7f6910803700 1 -- 172.21.15.154:0/1868473674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f690c1013f0 msgr2=0x7f690c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:24:36.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.102+0000 7f6910803700 1 --2- 172.21.15.154:0/1868473674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f690c1013f0 0x7f690c101810 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009990 tx=0x7f68fc009ca0 comp rx=0 tx=0).stop 2024-06-06T04:24:36.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.103+0000 7f6910803700 1 -- 172.21.15.154:0/1868473674 shutdown_connections 2024-06-06T04:24:36.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.103+0000 7f6910803700 1 --2- 172.21.15.154:0/1868473674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f690c102f90 0x7f690c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:36.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.103+0000 7f6910803700 1 --2- 172.21.15.154:0/1868473674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f690c1013f0 0x7f690c101810 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:36.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.103+0000 7f6910803700 1 --2- 172.21.15.154:0/1868473674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690c1025f0 0x7f690c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:36.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.103+0000 7f6910803700 1 -- 172.21.15.154:0/1868473674 >> 172.21.15.154:0/1868473674 conn(0x7f690c0fc9b0 msgr2=0x7f690c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:24:36.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.103+0000 7f6910803700 1 -- 172.21.15.154:0/1868473674 shutdown_connections 2024-06-06T04:24:36.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.103+0000 7f6910803700 1 -- 172.21.15.154:0/1868473674 wait complete. 2024-06-06T04:24:36.104 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:35 smithi154 ceph-mon[36428]: Detected new or changed devices on smithi086 2024-06-06T04:24:36.105 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:35 smithi154 ceph-mon[36428]: Adjusting osd_memory_target on smithi086 to 5696M 2024-06-06T04:24:36.105 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:35 smithi154 ceph-mon[36428]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:36.105 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:35 smithi154 ceph-mon[36428]: osdmap e54: 8 total, 8 up, 8 in 2024-06-06T04:24:36.106 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.103+0000 7f6910803700 1 Processor -- start 2024-06-06T04:24:36.106 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.103+0000 7f6910803700 1 -- start start 2024-06-06T04:24:36.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f6910803700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690c1013f0 0x7f690c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:24:36.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f6910803700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f690c1025f0 0x7f690c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:24:36.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f6910803700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f690c102f90 0x7f690c19fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:24:36.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f6910803700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f690c10ce90 con 0x7f690c102f90 2024-06-06T04:24:36.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f6910803700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f690c10cd10 con 0x7f690c1013f0 2024-06-06T04:24:36.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f6910803700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f690c10d010 con 0x7f690c1025f0 2024-06-06T04:24:36.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f6909d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f690c1025f0 0x7f690c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:24:36.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f6909d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f690c1025f0 0x7f690c198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44064/0 (socket says 172.21.15.154:44064) 2024-06-06T04:24:36.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f6909d9b700 1 -- 172.21.15.154:0/1931992805 learned_addr learned my addr 172.21.15.154:0/1931992805 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-06-06T04:24:36.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f690ad9d700 1 --2- 172.21.15.154:0/1931992805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f690c102f90 0x7f690c19fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:24:36.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f690a59c700 1 --2- 172.21.15.154:0/1931992805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690c1013f0 0x7f690c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:24:36.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f6909d9b700 1 -- 172.21.15.154:0/1931992805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690c1013f0 msgr2=0x7f690c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:24:36.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f6909d9b700 1 --2- 172.21.15.154:0/1931992805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690c1013f0 0x7f690c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:36.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f6909d9b700 1 -- 172.21.15.154:0/1931992805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f690c102f90 msgr2=0x7f690c19fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:24:36.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f6909d9b700 1 --2- 172.21.15.154:0/1931992805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f690c102f90 0x7f690c19fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:36.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f6909d9b700 1 -- 172.21.15.154:0/1931992805 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690c1a0270 con 0x7f690c1025f0 2024-06-06T04:24:36.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f6909d9b700 1 --2- 172.21.15.154:0/1931992805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f690c1025f0 0x7f690c198b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f690000d9b0 tx=0x7f690000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:24:36.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.104+0000 7f68fb7fe700 1 -- 172.21.15.154:0/1931992805 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900017070 con 0x7f690c1025f0 2024-06-06T04:24:36.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.105+0000 7f6910803700 1 -- 172.21.15.154:0/1931992805 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690c1a0560 con 0x7f690c1025f0 2024-06-06T04:24:36.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.105+0000 7f68fb7fe700 1 -- 172.21.15.154:0/1931992805 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6900005020 con 0x7f690c1025f0 2024-06-06T04:24:36.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.105+0000 7f68fb7fe700 1 -- 172.21.15.154:0/1931992805 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900009a20 con 0x7f690c1025f0 2024-06-06T04:24:36.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.105+0000 7f6910803700 1 -- 172.21.15.154:0/1931992805 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f690c1a0aa0 con 0x7f690c1025f0 2024-06-06T04:24:36.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.106+0000 7f6910803700 1 -- 172.21.15.154:0/1931992805 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f690c069180 con 0x7f690c1025f0 2024-06-06T04:24:36.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:35 smithi086 ceph-mon[36053]: Detected new or changed devices on smithi086 2024-06-06T04:24:36.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:35 smithi086 ceph-mon[36053]: Adjusting osd_memory_target on smithi086 to 5696M 2024-06-06T04:24:36.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:35 smithi086 ceph-mon[36053]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:36.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:35 smithi086 ceph-mon[36053]: osdmap e54: 8 total, 8 up, 8 in 2024-06-06T04:24:36.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.107+0000 7f68fb7fe700 1 -- 172.21.15.154:0/1931992805 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f69000051d0 con 0x7f690c1025f0 2024-06-06T04:24:36.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.108+0000 7f68fb7fe700 1 --2- 172.21.15.154:0/1931992805 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f68f40a2c20 0x7f68f40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:24:36.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.108+0000 7f68fb7fe700 1 -- 172.21.15.154:0/1931992805 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7f69000ce2f0 con 0x7f690c1025f0 2024-06-06T04:24:36.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.108+0000 7f690a59c700 1 --2- 172.21.15.154:0/1931992805 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f68f40a2c20 0x7f68f40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:24:36.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.108+0000 7f690a59c700 1 --2- 172.21.15.154:0/1931992805 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f68f40a2c20 0x7f68f40a50e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0037f0 tx=0x7f68fc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:24:36.112 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.112+0000 7f68fb7fe700 1 -- 172.21.15.154:0/1931992805 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6900020020 con 0x7f690c1025f0 2024-06-06T04:24:36.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:36.317+0000 7f6910803700 1 -- 172.21.15.154:0/1931992805 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f690c0feb90 con 0x7f68f40a2c20 2024-06-06T04:24:37.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:36 smithi086 ceph-mon[36053]: osdmap e55: 8 total, 8 up, 8 in 2024-06-06T04:24:37.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:36 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:24:37.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:36 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:24:37.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:36 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:37.152 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:36 smithi154 ceph-mon[36428]: osdmap e55: 8 total, 8 up, 8 in 2024-06-06T04:24:37.152 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:36 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:24:37.152 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:36 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:24:37.152 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:36 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:37.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:36 smithi045 ceph-mon[32299]: osdmap e55: 8 total, 8 up, 8 in 2024-06-06T04:24:37.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:36 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:24:37.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:36 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:24:37.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:36 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:38.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:37 smithi086 ceph-mon[36053]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:38.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:37 smithi086 ceph-mon[36053]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:24:38.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:37 smithi086 ceph-mon[36053]: osdmap e56: 8 total, 8 up, 8 in 2024-06-06T04:24:38.172 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:37 smithi154 ceph-mon[36428]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:38.173 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:37 smithi154 ceph-mon[36428]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:24:38.173 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:37 smithi154 ceph-mon[36428]: osdmap e56: 8 total, 8 up, 8 in 2024-06-06T04:24:38.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:37 smithi045 ceph-mon[32299]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:38.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:37 smithi045 ceph-mon[32299]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:24:38.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:37 smithi045 ceph-mon[32299]: osdmap e56: 8 total, 8 up, 8 in 2024-06-06T04:24:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:39 smithi086 ceph-mon[36053]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:39 smithi086 ceph-mon[36053]: from='client.? 172.21.15.154:0/3723381821' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdf1dd1b-5392-4836-8e65-4c7c5b688d4a"}]: dispatch 2024-06-06T04:24:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:39 smithi086 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdf1dd1b-5392-4836-8e65-4c7c5b688d4a"}]: dispatch 2024-06-06T04:24:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:39 smithi086 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdf1dd1b-5392-4836-8e65-4c7c5b688d4a"}]': finished 2024-06-06T04:24:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:39 smithi086 ceph-mon[36053]: osdmap e57: 9 total, 8 up, 9 in 2024-06-06T04:24:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:39 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T04:24:40.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:39 smithi086 ceph-mon[36053]: from='client.? 172.21.15.154:0/290109537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:24:40.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:39 smithi154 ceph-mon[36428]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:40.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:39 smithi154 ceph-mon[36428]: from='client.? 172.21.15.154:0/3723381821' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdf1dd1b-5392-4836-8e65-4c7c5b688d4a"}]: dispatch 2024-06-06T04:24:40.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:39 smithi154 ceph-mon[36428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdf1dd1b-5392-4836-8e65-4c7c5b688d4a"}]: dispatch 2024-06-06T04:24:40.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:39 smithi154 ceph-mon[36428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdf1dd1b-5392-4836-8e65-4c7c5b688d4a"}]': finished 2024-06-06T04:24:40.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:39 smithi154 ceph-mon[36428]: osdmap e57: 9 total, 8 up, 9 in 2024-06-06T04:24:40.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:39 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T04:24:40.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:39 smithi154 ceph-mon[36428]: from='client.? 172.21.15.154:0/290109537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:24:40.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:39 smithi045 ceph-mon[32299]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:40.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:39 smithi045 ceph-mon[32299]: from='client.? 172.21.15.154:0/3723381821' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdf1dd1b-5392-4836-8e65-4c7c5b688d4a"}]: dispatch 2024-06-06T04:24:40.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:39 smithi045 ceph-mon[32299]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdf1dd1b-5392-4836-8e65-4c7c5b688d4a"}]: dispatch 2024-06-06T04:24:40.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:39 smithi045 ceph-mon[32299]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdf1dd1b-5392-4836-8e65-4c7c5b688d4a"}]': finished 2024-06-06T04:24:40.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:39 smithi045 ceph-mon[32299]: osdmap e57: 9 total, 8 up, 9 in 2024-06-06T04:24:40.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:39 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T04:24:40.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:39 smithi045 ceph-mon[32299]: from='client.? 172.21.15.154:0/290109537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:24:42.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:41 smithi086 ceph-mon[36053]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:42.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:41 smithi154 ceph-mon[36428]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:42.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:41 smithi045 ceph-mon[32299]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:44.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:43 smithi086 ceph-mon[36053]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:44.127 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:43 smithi154 ceph-mon[36428]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:44.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:43 smithi045 ceph-mon[32299]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:45.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:45 smithi154 ceph-mon[36428]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:46.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:45 smithi086 ceph-mon[36053]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:46.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:45 smithi045 ceph-mon[32299]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:47.144 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:46 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-06T04:24:47.144 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:46 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:47.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:46 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-06T04:24:47.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:46 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:47.282 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:46 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-06T04:24:47.282 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:46 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:48.131 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:47 smithi154 ceph-mon[36428]: Deploying daemon osd.8 on smithi154 2024-06-06T04:24:48.131 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:47 smithi154 ceph-mon[36428]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:48.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:47 smithi086 ceph-mon[36053]: Deploying daemon osd.8 on smithi154 2024-06-06T04:24:48.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:47 smithi086 ceph-mon[36053]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:48.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:47 smithi045 ceph-mon[32299]: Deploying daemon osd.8 on smithi154 2024-06-06T04:24:48.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:47 smithi045 ceph-mon[32299]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:50.138 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:49 smithi154 ceph-mon[36428]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:50.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:49 smithi045 ceph-mon[32299]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:50.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:49 smithi086 ceph-mon[36053]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:52.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:51 smithi154 ceph-mon[36428]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:52.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:51 smithi045 ceph-mon[32299]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:52.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:51 smithi086 ceph-mon[36053]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:53.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:52 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:53.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:52 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:53.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:52 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:53.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:52 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:53.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:52 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:53.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:52 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:53.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:52 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:53.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:52 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:53.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:52 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:54.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:53 smithi154 ceph-mon[36428]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:54.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:53 smithi045 ceph-mon[32299]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:54.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:53 smithi086 ceph-mon[36053]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:55.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:55.938+0000 7f68fb7fe700 1 -- 172.21.15.154:0/1931992805 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f690c0feb90 con 0x7f68f40a2c20 2024-06-06T04:24:55.939 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 8 on host 'smithi154' 2024-06-06T04:24:55.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:55.941+0000 7f6910803700 1 -- 172.21.15.154:0/1931992805 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f68f40a2c20 msgr2=0x7f68f40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:24:55.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:55.941+0000 7f6910803700 1 --2- 172.21.15.154:0/1931992805 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f68f40a2c20 0x7f68f40a50e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0037f0 tx=0x7f68fc005fd0 comp rx=0 tx=0).stop 2024-06-06T04:24:55.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:55.941+0000 7f6910803700 1 -- 172.21.15.154:0/1931992805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f690c1025f0 msgr2=0x7f690c198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:24:55.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:55.941+0000 7f6910803700 1 --2- 172.21.15.154:0/1931992805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f690c1025f0 0x7f690c198b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f690000d9b0 tx=0x7f690000dcc0 comp rx=0 tx=0).stop 2024-06-06T04:24:55.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:55.941+0000 7f6910803700 1 -- 172.21.15.154:0/1931992805 shutdown_connections 2024-06-06T04:24:55.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:55.941+0000 7f6910803700 1 --2- 172.21.15.154:0/1931992805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f690c102f90 0x7f690c19fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:55.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:55.941+0000 7f6910803700 1 --2- 172.21.15.154:0/1931992805 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f68f40a2c20 0x7f68f40a50e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:55.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:55.941+0000 7f6910803700 1 --2- 172.21.15.154:0/1931992805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f690c1013f0 0x7f690c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:55.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:55.941+0000 7f6910803700 1 --2- 172.21.15.154:0/1931992805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f690c1025f0 0x7f690c198b50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:24:55.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:55.941+0000 7f6910803700 1 -- 172.21.15.154:0/1931992805 >> 172.21.15.154:0/1931992805 conn(0x7f690c0fc9b0 msgr2=0x7f690c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:24:55.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:55.942+0000 7f6910803700 1 -- 172.21.15.154:0/1931992805 shutdown_connections 2024-06-06T04:24:55.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:24:55.942+0000 7f6910803700 1 -- 172.21.15.154:0/1931992805 wait complete. 2024-06-06T04:24:56.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:56 smithi154 ceph-mon[36428]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:56.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:56 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:56.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:56 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:56.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:56 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:56.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:56 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:24:56.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:56 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:56.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:56 smithi154 ceph-mon[36428]: from='osd.8 [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T04:24:56.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:56 smithi154 ceph-mon[36428]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T04:24:56.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:56 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:56.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:56 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:56.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:56 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:56.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:56 smithi086 ceph-mon[36053]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:56.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:56 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:56.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:56 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:56.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:56 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:56.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:56 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:24:56.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:56 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:56.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:56 smithi086 ceph-mon[36053]: from='osd.8 [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T04:24:56.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:56 smithi086 ceph-mon[36053]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T04:24:56.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:56 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:56.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:56 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:56.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:56 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:56.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:56 smithi045 ceph-mon[32299]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:56.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:56 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:56.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:56 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:56.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:56 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:24:56.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:56 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:24:56.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:56 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:56.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:56 smithi045 ceph-mon[32299]: from='osd.8 [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T04:24:56.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:56 smithi045 ceph-mon[32299]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T04:24:56.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:56 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:56.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:56 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:24:56.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:56 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:24:56.578 DEBUG:teuthology.orchestra.run.smithi154:osd.8> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.8.service 2024-06-06T04:24:56.580 INFO:tasks.cephadm:Deploying osd.9 on smithi045 with /dev/vg_nvme/lv_1... 2024-06-06T04:24:56.580 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-06T04:24:56.837 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:24:57.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:57 smithi154 ceph-mon[36428]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-06T04:24:57.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:57 smithi154 ceph-mon[36428]: osdmap e58: 9 total, 8 up, 9 in 2024-06-06T04:24:57.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:57 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T04:24:57.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:57 smithi154 ceph-mon[36428]: from='osd.8 [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:24:57.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:57 smithi154 ceph-mon[36428]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:24:57.181 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:24:57 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:24:57.079+0000 7ffb7ac4e700 -1 osd.8 0 waiting for initial osdmap 2024-06-06T04:24:57.181 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:24:57 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:24:57.092+0000 7ffb76a41700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T04:24:57.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:57 smithi045 ceph-mon[32299]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-06T04:24:57.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:57 smithi045 ceph-mon[32299]: osdmap e58: 9 total, 8 up, 9 in 2024-06-06T04:24:57.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:57 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T04:24:57.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:57 smithi045 ceph-mon[32299]: from='osd.8 [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:24:57.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:57 smithi045 ceph-mon[32299]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:24:57.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:57 smithi086 ceph-mon[36053]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-06T04:24:57.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:57 smithi086 ceph-mon[36053]: osdmap e58: 9 total, 8 up, 9 in 2024-06-06T04:24:57.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:57 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T04:24:57.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:57 smithi086 ceph-mon[36053]: from='osd.8 [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:24:57.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:57 smithi086 ceph-mon[36053]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:24:58.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:58 smithi045 ceph-mon[32299]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:58 smithi045 ceph-mon[32299]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-06-06T04:24:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:58 smithi045 ceph-mon[32299]: osdmap e59: 9 total, 8 up, 9 in 2024-06-06T04:24:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:58 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T04:24:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:58 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T04:24:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:58 smithi045 ceph-mon[32299]: osd.8 [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496] boot 2024-06-06T04:24:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:58 smithi045 ceph-mon[32299]: osdmap e60: 9 total, 9 up, 9 in 2024-06-06T04:24:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:58 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T04:24:58.264 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:58 smithi154 ceph-mon[36428]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:58.264 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:58 smithi154 ceph-mon[36428]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-06-06T04:24:58.264 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:58 smithi154 ceph-mon[36428]: osdmap e59: 9 total, 8 up, 9 in 2024-06-06T04:24:58.264 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:58 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T04:24:58.264 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:58 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T04:24:58.264 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:58 smithi154 ceph-mon[36428]: osd.8 [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496] boot 2024-06-06T04:24:58.265 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:58 smithi154 ceph-mon[36428]: osdmap e60: 9 total, 9 up, 9 in 2024-06-06T04:24:58.265 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:58 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T04:24:58.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:58 smithi086 ceph-mon[36053]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-06T04:24:58.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:58 smithi086 ceph-mon[36053]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-06-06T04:24:58.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:58 smithi086 ceph-mon[36053]: osdmap e59: 9 total, 8 up, 9 in 2024-06-06T04:24:58.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:58 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T04:24:58.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:58 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T04:24:58.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:58 smithi086 ceph-mon[36053]: osd.8 [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496] boot 2024-06-06T04:24:58.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:58 smithi086 ceph-mon[36053]: osdmap e60: 9 total, 9 up, 9 in 2024-06-06T04:24:58.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:58 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T04:24:58.985 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-06T04:24:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-06T04:24:58.986 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-06-06T04:24:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-06-06T04:24:58.986 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00861592 s, 1.2 GB/s 2024-06-06T04:24:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-06-06T04:24:59.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:59 smithi045 ceph-mon[32299]: purged_snaps scrub starts 2024-06-06T04:24:59.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:59 smithi045 ceph-mon[32299]: purged_snaps scrub ok 2024-06-06T04:24:59.317 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:24:59 smithi045 ceph-mon[32299]: osdmap e61: 9 total, 9 up, 9 in 2024-06-06T04:24:59.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:59 smithi086 ceph-mon[36053]: purged_snaps scrub starts 2024-06-06T04:24:59.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:59 smithi086 ceph-mon[36053]: purged_snaps scrub ok 2024-06-06T04:24:59.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:24:59 smithi086 ceph-mon[36053]: osdmap e61: 9 total, 9 up, 9 in 2024-06-06T04:24:59.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:59 smithi154 ceph-mon[36428]: purged_snaps scrub starts 2024-06-06T04:24:59.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:59 smithi154 ceph-mon[36428]: purged_snaps scrub ok 2024-06-06T04:24:59.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:24:59 smithi154 ceph-mon[36428]: osdmap e61: 9 total, 9 up, 9 in 2024-06-06T04:24:59.776 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_1 2024-06-06T04:25:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:25:00.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:00 smithi086 ceph-mon[36053]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:00.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:00.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:00.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:00.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:00.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:00.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:00.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:00.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:25:00.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:00.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:00 smithi086 ceph-mon[36053]: osdmap e62: 9 total, 9 up, 9 in 2024-06-06T04:25:00.363 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:00 smithi045 ceph-mon[32299]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:00.363 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:00.363 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:00.363 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:00.363 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:00.364 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:00.364 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:00.364 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:00.364 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:25:00.364 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:00.364 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:00 smithi045 ceph-mon[32299]: osdmap e62: 9 total, 9 up, 9 in 2024-06-06T04:25:00.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:00 smithi154 ceph-mon[36428]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:25:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:00 smithi154 ceph-mon[36428]: osdmap e62: 9 total, 9 up, 9 in 2024-06-06T04:25:01.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:01 smithi045 ceph-mon[32299]: Detected new or changed devices on smithi154 2024-06-06T04:25:01.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:01 smithi045 ceph-mon[32299]: Adjusting osd_memory_target on smithi154 to 5696M 2024-06-06T04:25:01.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:01 smithi045 ceph-mon[32299]: osdmap e63: 9 total, 9 up, 9 in 2024-06-06T04:25:01.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:01 smithi086 ceph-mon[36053]: Detected new or changed devices on smithi154 2024-06-06T04:25:01.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:01 smithi086 ceph-mon[36053]: Adjusting osd_memory_target on smithi154 to 5696M 2024-06-06T04:25:01.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:01 smithi086 ceph-mon[36053]: osdmap e63: 9 total, 9 up, 9 in 2024-06-06T04:25:01.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:01 smithi154 ceph-mon[36428]: Detected new or changed devices on smithi154 2024-06-06T04:25:01.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:01 smithi154 ceph-mon[36428]: Adjusting osd_memory_target on smithi154 to 5696M 2024-06-06T04:25:01.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:01 smithi154 ceph-mon[36428]: osdmap e63: 9 total, 9 up, 9 in 2024-06-06T04:25:01.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.690+0000 7fb8e0cc0700 1 -- 172.21.15.45:0/2209770600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8dc1081e0 msgr2=0x7fb8dc10a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:25:01.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.690+0000 7fb8e0cc0700 1 --2- 172.21.15.45:0/2209770600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8dc1081e0 0x7fb8dc10a650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400b0a0 tx=0x7fb8d400b3b0 comp rx=0 tx=0).stop 2024-06-06T04:25:01.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.690+0000 7fb8e0cc0700 1 -- 172.21.15.45:0/2209770600 shutdown_connections 2024-06-06T04:25:01.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.690+0000 7fb8e0cc0700 1 --2- 172.21.15.45:0/2209770600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8dc1081e0 0x7fb8dc10a650 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:01.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.690+0000 7fb8e0cc0700 1 --2- 172.21.15.45:0/2209770600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8dc102f80 0x7fb8dc105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:01.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.690+0000 7fb8e0cc0700 1 --2- 172.21.15.45:0/2209770600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8dc1058b0 0x7fb8dc107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:01.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.690+0000 7fb8e0cc0700 1 -- 172.21.15.45:0/2209770600 >> 172.21.15.45:0/2209770600 conn(0x7fb8dc0fc990 msgr2=0x7fb8dc0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:25:01.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.690+0000 7fb8e0cc0700 1 -- 172.21.15.45:0/2209770600 shutdown_connections 2024-06-06T04:25:01.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.690+0000 7fb8e0cc0700 1 -- 172.21.15.45:0/2209770600 wait complete. 2024-06-06T04:25:01.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.690+0000 7fb8e0cc0700 1 Processor -- start 2024-06-06T04:25:01.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8e0cc0700 1 -- start start 2024-06-06T04:25:01.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8e0cc0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8dc102f80 0x7fb8dc1028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:25:01.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8e0cc0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8dc1058b0 0x7fb8dc100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:25:01.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8e0cc0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8dc1081e0 0x7fb8dc101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:25:01.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8e0cc0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc10df50 con 0x7fb8dc1081e0 2024-06-06T04:25:01.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8e0cc0700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc10ddd0 con 0x7fb8dc1058b0 2024-06-06T04:25:01.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8e0cc0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc10e0d0 con 0x7fb8dc102f80 2024-06-06T04:25:01.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8dad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8dc1081e0 0x7fb8dc101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:25:01.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8dad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8dc1081e0 0x7fb8dc101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59932/0 (socket says 172.21.15.45:59932) 2024-06-06T04:25:01.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8dad9d700 1 -- 172.21.15.45:0/3460964923 learned_addr learned my addr 172.21.15.45:0/3460964923 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:25:01.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8d1d9b700 1 --2- 172.21.15.45:0/3460964923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8dc1058b0 0x7fb8dc100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:25:01.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8da59c700 1 --2- 172.21.15.45:0/3460964923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8dc102f80 0x7fb8dc1028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:25:01.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8dad9d700 1 -- 172.21.15.45:0/3460964923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8dc102f80 msgr2=0x7fb8dc1028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:25:01.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8dad9d700 1 --2- 172.21.15.45:0/3460964923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8dc102f80 0x7fb8dc1028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:01.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8dad9d700 1 -- 172.21.15.45:0/3460964923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8dc1058b0 msgr2=0x7fb8dc100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:25:01.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8dad9d700 1 --2- 172.21.15.45:0/3460964923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8dc1058b0 0x7fb8dc100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:01.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8dad9d700 1 -- 172.21.15.45:0/3460964923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc101d20 con 0x7fb8dc1081e0 2024-06-06T04:25:01.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.691+0000 7fb8da59c700 1 --2- 172.21.15.45:0/3460964923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8dc102f80 0x7fb8dc1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:25:01.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.692+0000 7fb8dad9d700 1 --2- 172.21.15.45:0/3460964923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8dc1081e0 0x7fb8dc101470 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40062a0 tx=0x7fb8d4004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:25:01.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.692+0000 7fb8d3fff700 1 -- 172.21.15.45:0/3460964923 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4021070 con 0x7fb8dc1081e0 2024-06-06T04:25:01.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.692+0000 7fb8d3fff700 1 -- 172.21.15.45:0/3460964923 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8d4007c80 con 0x7fb8dc1081e0 2024-06-06T04:25:01.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.692+0000 7fb8e0cc0700 1 -- 172.21.15.45:0/3460964923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8dc1a26d0 con 0x7fb8dc1081e0 2024-06-06T04:25:01.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.692+0000 7fb8d3fff700 1 -- 172.21.15.45:0/3460964923 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d401d450 con 0x7fb8dc1081e0 2024-06-06T04:25:01.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.692+0000 7fb8e0cc0700 1 -- 172.21.15.45:0/3460964923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8dc1a2b30 con 0x7fb8dc1081e0 2024-06-06T04:25:01.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.693+0000 7fb8e0cc0700 1 -- 172.21.15.45:0/3460964923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8dc069180 con 0x7fb8dc1081e0 2024-06-06T04:25:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.694+0000 7fb8d3fff700 1 -- 172.21.15.45:0/3460964923 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb8d4004020 con 0x7fb8dc1081e0 2024-06-06T04:25:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.695+0000 7fb8d3fff700 1 --2- 172.21.15.45:0/3460964923 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb8c80a2c20 0x7fb8c80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:25:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.695+0000 7fb8da59c700 1 --2- 172.21.15.45:0/3460964923 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb8c80a2c20 0x7fb8c80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:25:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.695+0000 7fb8d3fff700 1 -- 172.21.15.45:0/3460964923 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7fb8d400f020 con 0x7fb8dc1081e0 2024-06-06T04:25:01.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.695+0000 7fb8da59c700 1 --2- 172.21.15.45:0/3460964923 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb8c80a2c20 0x7fb8c80a50e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc005fd0 tx=0x7fb8cc005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:25:01.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.700+0000 7fb8d3fff700 1 -- 172.21.15.45:0/3460964923 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb8d402a020 con 0x7fb8dc1081e0 2024-06-06T04:25:01.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:01.920+0000 7fb8e0cc0700 1 -- 172.21.15.45:0/3460964923 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb8dc0628d0 con 0x7fb8c80a2c20 2024-06-06T04:25:02.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:02 smithi045 ceph-mon[32299]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:02.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:02 smithi045 ceph-mon[32299]: osdmap e64: 9 total, 9 up, 9 in 2024-06-06T04:25:02.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:25:02.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:25:02.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:02.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:02 smithi086 ceph-mon[36053]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:02.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:02 smithi086 ceph-mon[36053]: osdmap e64: 9 total, 9 up, 9 in 2024-06-06T04:25:02.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:25:02.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:25:02.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:02.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:02 smithi154 ceph-mon[36428]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:02.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:02 smithi154 ceph-mon[36428]: osdmap e64: 9 total, 9 up, 9 in 2024-06-06T04:25:02.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:25:02.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:25:02.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:03.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:03 smithi086 ceph-mon[36053]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:25:03.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:03 smithi154 ceph-mon[36428]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:25:03.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:03 smithi045 ceph-mon[32299]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:25:04.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:04 smithi045 ceph-mon[32299]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:04.351 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:04 smithi154 ceph-mon[36428]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:04.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:04 smithi086 ceph-mon[36053]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:05 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/4227041578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acede92d-2cc1-4967-a357-7e5e201e99d7"}]: dispatch 2024-06-06T04:25:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:05 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/4227041578' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acede92d-2cc1-4967-a357-7e5e201e99d7"}]': finished 2024-06-06T04:25:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:05 smithi086 ceph-mon[36053]: osdmap e65: 10 total, 9 up, 10 in 2024-06-06T04:25:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:05.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:05 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/4227041578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acede92d-2cc1-4967-a357-7e5e201e99d7"}]: dispatch 2024-06-06T04:25:05.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:05 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/4227041578' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acede92d-2cc1-4967-a357-7e5e201e99d7"}]': finished 2024-06-06T04:25:05.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:05 smithi154 ceph-mon[36428]: osdmap e65: 10 total, 9 up, 10 in 2024-06-06T04:25:05.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:05.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:05 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/4227041578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acede92d-2cc1-4967-a357-7e5e201e99d7"}]: dispatch 2024-06-06T04:25:05.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:05 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/4227041578' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acede92d-2cc1-4967-a357-7e5e201e99d7"}]': finished 2024-06-06T04:25:05.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:05 smithi045 ceph-mon[32299]: osdmap e65: 10 total, 9 up, 10 in 2024-06-06T04:25:05.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:06.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:06 smithi086 ceph-mon[36053]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-06-06T04:25:06.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:06 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1936700077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:25:06.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:06 smithi154 ceph-mon[36428]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-06-06T04:25:06.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:06 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1936700077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:25:06.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:06 smithi045 ceph-mon[32299]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-06-06T04:25:06.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:06 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1936700077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:25:08.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:08 smithi086 ceph-mon[36053]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T04:25:08.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:08 smithi154 ceph-mon[36428]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T04:25:08.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:08 smithi045 ceph-mon[32299]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T04:25:10.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:10 smithi045 ceph-mon[32299]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-06-06T04:25:10.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:10 smithi154 ceph-mon[36428]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-06-06T04:25:10.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:10 smithi086 ceph-mon[36053]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-06-06T04:25:12.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:12 smithi154 ceph-mon[36428]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-06-06T04:25:12.471 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:12 smithi045 ceph-mon[32299]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-06-06T04:25:12.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:12 smithi086 ceph-mon[36053]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-06-06T04:25:13.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:13 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-06T04:25:13.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:13 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:13.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:13 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-06T04:25:13.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:13 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:13.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:13 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-06T04:25:13.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:13 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:14.271 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:14 smithi045 ceph-mon[32299]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T04:25:14.271 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:14 smithi045 ceph-mon[32299]: Deploying daemon osd.9 on smithi045 2024-06-06T04:25:14.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:14 smithi154 ceph-mon[36428]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T04:25:14.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:14 smithi154 ceph-mon[36428]: Deploying daemon osd.9 on smithi045 2024-06-06T04:25:14.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:14 smithi086 ceph-mon[36053]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T04:25:14.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:14 smithi086 ceph-mon[36053]: Deploying daemon osd.9 on smithi045 2024-06-06T04:25:16.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:16 smithi154 ceph-mon[36428]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:16.460 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:16 smithi045 ceph-mon[32299]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:16.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:16 smithi086 ceph-mon[36053]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:18.157 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:18 smithi045 ceph-mon[32299]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:18.411 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:18 smithi086 ceph-mon[36053]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:18.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:18 smithi154 ceph-mon[36428]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:19.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:19.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:19.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:25:19.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:19.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:19.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:25:19.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:19.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:19.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:25:20.474 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:20 smithi045 ceph-mon[32299]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:20.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:20 smithi086 ceph-mon[36053]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:20.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:20 smithi154 ceph-mon[36428]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:22.147 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 9 on host 'smithi045' 2024-06-06T04:25:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:22.146+0000 7fb8d3fff700 1 -- 172.21.15.45:0/3460964923 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb8dc0628d0 con 0x7fb8c80a2c20 2024-06-06T04:25:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:22.149+0000 7fb8e0cc0700 1 -- 172.21.15.45:0/3460964923 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb8c80a2c20 msgr2=0x7fb8c80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:25:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:22.149+0000 7fb8e0cc0700 1 --2- 172.21.15.45:0/3460964923 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb8c80a2c20 0x7fb8c80a50e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc005fd0 tx=0x7fb8cc005e30 comp rx=0 tx=0).stop 2024-06-06T04:25:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:22.149+0000 7fb8e0cc0700 1 -- 172.21.15.45:0/3460964923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8dc1081e0 msgr2=0x7fb8dc101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:25:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:22.149+0000 7fb8e0cc0700 1 --2- 172.21.15.45:0/3460964923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8dc1081e0 0x7fb8dc101470 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40062a0 tx=0x7fb8d4004e40 comp rx=0 tx=0).stop 2024-06-06T04:25:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:22.149+0000 7fb8e0cc0700 1 -- 172.21.15.45:0/3460964923 shutdown_connections 2024-06-06T04:25:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:22.149+0000 7fb8e0cc0700 1 --2- 172.21.15.45:0/3460964923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8dc1081e0 0x7fb8dc101470 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:22.149+0000 7fb8e0cc0700 1 --2- 172.21.15.45:0/3460964923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8dc102f80 0x7fb8dc1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:22.149+0000 7fb8e0cc0700 1 --2- 172.21.15.45:0/3460964923 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb8c80a2c20 0x7fb8c80a50e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:22.149+0000 7fb8e0cc0700 1 --2- 172.21.15.45:0/3460964923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8dc1058b0 0x7fb8dc100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:22.149+0000 7fb8e0cc0700 1 -- 172.21.15.45:0/3460964923 >> 172.21.15.45:0/3460964923 conn(0x7fb8dc0fc990 msgr2=0x7fb8dc1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:25:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:22.149+0000 7fb8e0cc0700 1 -- 172.21.15.45:0/3460964923 shutdown_connections 2024-06-06T04:25:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:25:22.149+0000 7fb8e0cc0700 1 -- 172.21.15.45:0/3460964923 wait complete. 2024-06-06T04:25:22.456 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:22 smithi045 ceph-mon[32299]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:22.456 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:22 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:22.456 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:22 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:22.456 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:22 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:22.456 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:22 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:25:22.457 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:22 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:22.457 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:22 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:25:22.457 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:22 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:22.457 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:22 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:22.457 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:22 smithi045 ceph-mon[32299]: from='osd.9 [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-06T04:25:22.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:22 smithi086 ceph-mon[36053]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:22.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:22 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:22.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:22 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:22.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:22 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:22.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:22 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:25:22.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:22 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:22.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:22 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:25:22.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:22 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:22.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:22 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:22.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:22 smithi086 ceph-mon[36053]: from='osd.9 [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-06T04:25:22.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:22 smithi154 ceph-mon[36428]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:22.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:22 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:22.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:22 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:22.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:22 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:22.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:22 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:25:22.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:22 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:22.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:22 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:25:22.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:22 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:22.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:22 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:22.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:22 smithi154 ceph-mon[36428]: from='osd.9 [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-06T04:25:22.763 DEBUG:teuthology.orchestra.run.smithi045:osd.9> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.9.service 2024-06-06T04:25:22.766 INFO:tasks.cephadm:Deploying osd.10 on smithi086 with /dev/vg_nvme/lv_1... 2024-06-06T04:25:22.766 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-06T04:25:23.009 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T04:25:24.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:24 smithi045 ceph-mon[32299]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:24.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:24 smithi045 ceph-mon[32299]: from='osd.9 [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-06T04:25:24.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:24 smithi045 ceph-mon[32299]: osdmap e66: 10 total, 9 up, 10 in 2024-06-06T04:25:24.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:24 smithi045 ceph-mon[32299]: from='osd.9 [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-06-06T04:25:24.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:24 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:24.228 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:25:24 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:25:24.112+0000 7f08908b0700 -1 osd.9 0 waiting for initial osdmap 2024-06-06T04:25:24.228 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:25:24 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:25:24.125+0000 7f088c6a3700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T04:25:24.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:24 smithi154 ceph-mon[36428]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:24.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:24 smithi154 ceph-mon[36428]: from='osd.9 [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-06T04:25:24.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:24 smithi154 ceph-mon[36428]: osdmap e66: 10 total, 9 up, 10 in 2024-06-06T04:25:24.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:24 smithi154 ceph-mon[36428]: from='osd.9 [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-06-06T04:25:24.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:24 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:24.484 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:24 smithi086 ceph-mon[36053]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:24.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:24 smithi086 ceph-mon[36053]: from='osd.9 [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-06T04:25:24.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:24 smithi086 ceph-mon[36053]: osdmap e66: 10 total, 9 up, 10 in 2024-06-06T04:25:24.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:24 smithi086 ceph-mon[36053]: from='osd.9 [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-06-06T04:25:24.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:24 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:25.073 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-06T04:25:25.073 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-06T04:25:25.073 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-06-06T04:25:25.073 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-06-06T04:25:25.073 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122734 s, 854 MB/s 2024-06-06T04:25:25.073 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-06-06T04:25:25.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:25 smithi086 ceph-mon[36053]: purged_snaps scrub starts 2024-06-06T04:25:25.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:25 smithi086 ceph-mon[36053]: purged_snaps scrub ok 2024-06-06T04:25:25.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:25 smithi086 ceph-mon[36053]: from='osd.9 [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-06-06T04:25:25.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:25 smithi086 ceph-mon[36053]: osdmap e67: 10 total, 9 up, 10 in 2024-06-06T04:25:25.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:25 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:25.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:25 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:25.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:25 smithi045 ceph-mon[32299]: purged_snaps scrub starts 2024-06-06T04:25:25.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:25 smithi045 ceph-mon[32299]: purged_snaps scrub ok 2024-06-06T04:25:25.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:25 smithi045 ceph-mon[32299]: from='osd.9 [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-06-06T04:25:25.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:25 smithi045 ceph-mon[32299]: osdmap e67: 10 total, 9 up, 10 in 2024-06-06T04:25:25.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:25 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:25.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:25 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:25.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:25 smithi154 ceph-mon[36428]: purged_snaps scrub starts 2024-06-06T04:25:25.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:25 smithi154 ceph-mon[36428]: purged_snaps scrub ok 2024-06-06T04:25:25.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:25 smithi154 ceph-mon[36428]: from='osd.9 [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-06-06T04:25:25.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:25 smithi154 ceph-mon[36428]: osdmap e67: 10 total, 9 up, 10 in 2024-06-06T04:25:25.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:25 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:25.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:25 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:25.771 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_1 2024-06-06T04:25:26.011 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T04:25:26.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:26 smithi045 ceph-mon[32299]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:26.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:26.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:26 smithi045 ceph-mon[32299]: osd.9 [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510] boot 2024-06-06T04:25:26.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:26 smithi045 ceph-mon[32299]: osdmap e68: 10 total, 10 up, 10 in 2024-06-06T04:25:26.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:26.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:26.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:26.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:26.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:26.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:26.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:26.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:26.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:26.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:25:26.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:26.395 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:26 smithi086 ceph-mon[36053]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:26.396 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:26.396 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:26 smithi086 ceph-mon[36053]: osd.9 [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510] boot 2024-06-06T04:25:26.396 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:26 smithi086 ceph-mon[36053]: osdmap e68: 10 total, 10 up, 10 in 2024-06-06T04:25:26.396 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:26.396 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:26.396 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:26.396 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:26.396 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:26.396 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:26.396 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:26.396 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:26.396 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:26.396 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:25:26.396 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:26.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:26 smithi154 ceph-mon[36428]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-06T04:25:26.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:26.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:26 smithi154 ceph-mon[36428]: osd.9 [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510] boot 2024-06-06T04:25:26.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:26 smithi154 ceph-mon[36428]: osdmap e68: 10 total, 10 up, 10 in 2024-06-06T04:25:26.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T04:25:26.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:26.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:26.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:26.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:26.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:26.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:26.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:26.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:26.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:25:26.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:27.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:27 smithi154 ceph-mon[36428]: Detected new or changed devices on smithi045 2024-06-06T04:25:27.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:27 smithi154 ceph-mon[36428]: Adjusting osd_memory_target on smithi045 to 4273M 2024-06-06T04:25:27.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:27 smithi154 ceph-mon[36428]: osdmap e69: 10 total, 10 up, 10 in 2024-06-06T04:25:27.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:27 smithi045 ceph-mon[32299]: Detected new or changed devices on smithi045 2024-06-06T04:25:27.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:27 smithi045 ceph-mon[32299]: Adjusting osd_memory_target on smithi045 to 4273M 2024-06-06T04:25:27.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:27 smithi045 ceph-mon[32299]: osdmap e69: 10 total, 10 up, 10 in 2024-06-06T04:25:27.484 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:27 smithi086 ceph-mon[36053]: Detected new or changed devices on smithi045 2024-06-06T04:25:27.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:27 smithi086 ceph-mon[36053]: Adjusting osd_memory_target on smithi045 to 4273M 2024-06-06T04:25:27.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:27 smithi086 ceph-mon[36053]: osdmap e69: 10 total, 10 up, 10 in 2024-06-06T04:25:27.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.717+0000 7fcf2aca8700 1 -- 172.21.15.86:0/322127837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf241036f0 msgr2=0x7fcf24103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:25:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.717+0000 7fcf2aca8700 1 --2- 172.21.15.86:0/322127837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf241036f0 0x7fcf24103b50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf100099c0 tx=0x7fcf10009cd0 comp rx=0 tx=0).stop 2024-06-06T04:25:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.717+0000 7fcf2aca8700 1 -- 172.21.15.86:0/322127837 shutdown_connections 2024-06-06T04:25:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.717+0000 7fcf2aca8700 1 --2- 172.21.15.86:0/322127837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf24104090 0x7fcf2410a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.717+0000 7fcf2aca8700 1 --2- 172.21.15.86:0/322127837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf241024f0 0x7fcf24102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.717+0000 7fcf2aca8700 1 --2- 172.21.15.86:0/322127837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf241036f0 0x7fcf24103b50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.717+0000 7fcf2aca8700 1 -- 172.21.15.86:0/322127837 >> 172.21.15.86:0/322127837 conn(0x7fcf240fda90 msgr2=0x7fcf240ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:25:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.717+0000 7fcf2aca8700 1 -- 172.21.15.86:0/322127837 shutdown_connections 2024-06-06T04:25:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.717+0000 7fcf2aca8700 1 -- 172.21.15.86:0/322127837 wait complete. 2024-06-06T04:25:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.718+0000 7fcf2aca8700 1 Processor -- start 2024-06-06T04:25:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.718+0000 7fcf2aca8700 1 -- start start 2024-06-06T04:25:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.718+0000 7fcf2aca8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf241024f0 0x7fcf24198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:25:27.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.718+0000 7fcf2aca8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf241036f0 0x7fcf24198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:25:27.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.718+0000 7fcf2aca8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf24104090 0x7fcf2419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:25:27.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.718+0000 7fcf2aca8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2410df90 con 0x7fcf24104090 2024-06-06T04:25:27.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.718+0000 7fcf2aca8700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2410de10 con 0x7fcf241036f0 2024-06-06T04:25:27.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.718+0000 7fcf2aca8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2410e110 con 0x7fcf241024f0 2024-06-06T04:25:27.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.718+0000 7fcf23fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf241036f0 0x7fcf24198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:25:27.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.718+0000 7fcf23fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf241036f0 0x7fcf24198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53448/0 (socket says 172.21.15.86:53448) 2024-06-06T04:25:27.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.718+0000 7fcf23fff700 1 -- 172.21.15.86:0/663076010 learned_addr learned my addr 172.21.15.86:0/663076010 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-06-06T04:25:27.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.718+0000 7fcf29245700 1 --2- 172.21.15.86:0/663076010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf24104090 0x7fcf2419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:25:27.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.719+0000 7fcf28a44700 1 --2- 172.21.15.86:0/663076010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf241024f0 0x7fcf24198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:25:27.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.719+0000 7fcf23fff700 1 -- 172.21.15.86:0/663076010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf241024f0 msgr2=0x7fcf24198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:25:27.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.719+0000 7fcf23fff700 1 --2- 172.21.15.86:0/663076010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf241024f0 0x7fcf24198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:27.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.719+0000 7fcf23fff700 1 -- 172.21.15.86:0/663076010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf24104090 msgr2=0x7fcf2419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:25:27.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.719+0000 7fcf23fff700 1 --2- 172.21.15.86:0/663076010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf24104090 0x7fcf2419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:27.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.719+0000 7fcf23fff700 1 -- 172.21.15.86:0/663076010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf241a02c0 con 0x7fcf241036f0 2024-06-06T04:25:27.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.719+0000 7fcf29245700 1 --2- 172.21.15.86:0/663076010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf24104090 0x7fcf2419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:25:27.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.719+0000 7fcf23fff700 1 --2- 172.21.15.86:0/663076010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf241036f0 0x7fcf24198ba0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcf10006010 tx=0x7fcf100047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:25:27.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.719+0000 7fcf21ffb700 1 -- 172.21.15.86:0/663076010 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf10027070 con 0x7fcf241036f0 2024-06-06T04:25:27.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.719+0000 7fcf2aca8700 1 -- 172.21.15.86:0/663076010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf241a0550 con 0x7fcf241036f0 2024-06-06T04:25:27.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.719+0000 7fcf21ffb700 1 -- 172.21.15.86:0/663076010 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf100040d0 con 0x7fcf241036f0 2024-06-06T04:25:27.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.719+0000 7fcf21ffb700 1 -- 172.21.15.86:0/663076010 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf10022e60 con 0x7fcf241036f0 2024-06-06T04:25:27.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.719+0000 7fcf2aca8700 1 -- 172.21.15.86:0/663076010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf241a0a00 con 0x7fcf241036f0 2024-06-06T04:25:27.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.720+0000 7fcf2aca8700 1 -- 172.21.15.86:0/663076010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf24069180 con 0x7fcf241036f0 2024-06-06T04:25:27.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.721+0000 7fcf21ffb700 1 -- 172.21.15.86:0/663076010 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcf1002a070 con 0x7fcf241036f0 2024-06-06T04:25:27.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.722+0000 7fcf21ffb700 1 --2- 172.21.15.86:0/663076010 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fcf140a2c20 0x7fcf140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:25:27.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.722+0000 7fcf21ffb700 1 -- 172.21.15.86:0/663076010 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fcf100cf5f0 con 0x7fcf241036f0 2024-06-06T04:25:27.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.722+0000 7fcf28a44700 1 --2- 172.21.15.86:0/663076010 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fcf140a2c20 0x7fcf140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:25:27.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.722+0000 7fcf28a44700 1 --2- 172.21.15.86:0/663076010 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fcf140a2c20 0x7fcf140a50e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcf18005950 tx=0x7fcf180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:25:27.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.726+0000 7fcf21ffb700 1 -- 172.21.15.86:0/663076010 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcf10013070 con 0x7fcf241036f0 2024-06-06T04:25:27.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:27.929+0000 7fcf2aca8700 1 -- 172.21.15.86:0/663076010 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fcf240628d0 con 0x7fcf140a2c20 2024-06-06T04:25:28.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:28 smithi086 ceph-mon[36053]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:28.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:28 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:25:28.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:28 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:25:28.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:28 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:28.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:28 smithi154 ceph-mon[36428]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:28.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:28 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:25:28.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:28 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:25:28.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:28 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:28.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:28 smithi045 ceph-mon[32299]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:28.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:28 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:25:28.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:28 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:25:28.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:28 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:29.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:29 smithi154 ceph-mon[36428]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:25:29.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:29 smithi045 ceph-mon[32299]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:25:29.574 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:29 smithi086 ceph-mon[36053]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:25:30.174 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:30 smithi086 ceph-mon[36053]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:30.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:30 smithi154 ceph-mon[36428]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:30.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:30 smithi045 ceph-mon[32299]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:31.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:31 smithi154 ceph-mon[36428]: from='client.? 172.21.15.86:0/1078186097' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2d0c488-1892-4da4-adbf-952c22df0ce6"}]: dispatch 2024-06-06T04:25:31.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:31 smithi154 ceph-mon[36428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2d0c488-1892-4da4-adbf-952c22df0ce6"}]: dispatch 2024-06-06T04:25:31.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:31 smithi154 ceph-mon[36428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2d0c488-1892-4da4-adbf-952c22df0ce6"}]': finished 2024-06-06T04:25:31.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:31 smithi154 ceph-mon[36428]: osdmap e70: 11 total, 10 up, 11 in 2024-06-06T04:25:31.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:31 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:31.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:31 smithi045 ceph-mon[32299]: from='client.? 172.21.15.86:0/1078186097' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2d0c488-1892-4da4-adbf-952c22df0ce6"}]: dispatch 2024-06-06T04:25:31.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:31 smithi045 ceph-mon[32299]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2d0c488-1892-4da4-adbf-952c22df0ce6"}]: dispatch 2024-06-06T04:25:31.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:31 smithi045 ceph-mon[32299]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2d0c488-1892-4da4-adbf-952c22df0ce6"}]': finished 2024-06-06T04:25:31.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:31 smithi045 ceph-mon[32299]: osdmap e70: 11 total, 10 up, 11 in 2024-06-06T04:25:31.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:31 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:31.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:31 smithi086 ceph-mon[36053]: from='client.? 172.21.15.86:0/1078186097' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2d0c488-1892-4da4-adbf-952c22df0ce6"}]: dispatch 2024-06-06T04:25:31.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:31 smithi086 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2d0c488-1892-4da4-adbf-952c22df0ce6"}]: dispatch 2024-06-06T04:25:31.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:31 smithi086 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2d0c488-1892-4da4-adbf-952c22df0ce6"}]': finished 2024-06-06T04:25:31.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:31 smithi086 ceph-mon[36053]: osdmap e70: 11 total, 10 up, 11 in 2024-06-06T04:25:31.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:31 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:32.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:32 smithi154 ceph-mon[36428]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:32.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:32 smithi154 ceph-mon[36428]: from='client.? 172.21.15.86:0/4205386157' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:25:32.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:32 smithi045 ceph-mon[32299]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:32.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:32 smithi045 ceph-mon[32299]: from='client.? 172.21.15.86:0/4205386157' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:25:32.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:32 smithi086 ceph-mon[36053]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:32.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:32 smithi086 ceph-mon[36053]: from='client.? 172.21.15.86:0/4205386157' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:25:34.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:34 smithi154 ceph-mon[36428]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:34.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:34 smithi045 ceph-mon[32299]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:34.556 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:34 smithi086 ceph-mon[36053]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:36.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:36 smithi154 ceph-mon[36428]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:36.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:36 smithi086 ceph-mon[36053]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:36.468 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:36 smithi045 ceph-mon[32299]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:38.329 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:38 smithi086 ceph-mon[36053]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:38.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:38 smithi154 ceph-mon[36428]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:38.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:38 smithi045 ceph-mon[32299]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:39.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:39 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-06T04:25:39.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:39 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:39.440 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:39 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-06T04:25:39.440 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:39 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:39.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:39 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-06T04:25:39.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:39 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:40.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:40 smithi154 ceph-mon[36428]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:40.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:40 smithi154 ceph-mon[36428]: Deploying daemon osd.10 on smithi086 2024-06-06T04:25:40.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:40 smithi045 ceph-mon[32299]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:40.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:40 smithi045 ceph-mon[32299]: Deploying daemon osd.10 on smithi086 2024-06-06T04:25:40.500 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:40 smithi086 ceph-mon[36053]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:40.500 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:40 smithi086 ceph-mon[36053]: Deploying daemon osd.10 on smithi086 2024-06-06T04:25:42.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:42 smithi086 ceph-mon[36053]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:42.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:42 smithi154 ceph-mon[36428]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:42.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:42 smithi045 ceph-mon[32299]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:44.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:44 smithi154 ceph-mon[36428]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:44.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:44 smithi045 ceph-mon[32299]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:44.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:44 smithi086 ceph-mon[36053]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:45.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:45 smithi086 ceph-mon[36053]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:45.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:45 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:45.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:45 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:45.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:45 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:25:45.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:45 smithi154 ceph-mon[36428]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:45.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:45 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:45.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:45 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:45.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:45 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:25:45.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:45 smithi045 ceph-mon[32299]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:45.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:45 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:45.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:45 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:45.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:45 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:25:47.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:47 smithi154 ceph-mon[36428]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:47.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:47 smithi045 ceph-mon[32299]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:47.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:47 smithi086 ceph-mon[36053]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:48.299 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 10 on host 'smithi086' 2024-06-06T04:25:48.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:48.298+0000 7fcf21ffb700 1 -- 172.21.15.86:0/663076010 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fcf240628d0 con 0x7fcf140a2c20 2024-06-06T04:25:48.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:48.301+0000 7fcf2aca8700 1 -- 172.21.15.86:0/663076010 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fcf140a2c20 msgr2=0x7fcf140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:25:48.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:48.301+0000 7fcf2aca8700 1 --2- 172.21.15.86:0/663076010 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fcf140a2c20 0x7fcf140a50e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcf18005950 tx=0x7fcf180058e0 comp rx=0 tx=0).stop 2024-06-06T04:25:48.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:48.301+0000 7fcf2aca8700 1 -- 172.21.15.86:0/663076010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf241036f0 msgr2=0x7fcf24198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:25:48.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:48.301+0000 7fcf2aca8700 1 --2- 172.21.15.86:0/663076010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf241036f0 0x7fcf24198ba0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcf10006010 tx=0x7fcf100047b0 comp rx=0 tx=0).stop 2024-06-06T04:25:48.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:48.301+0000 7fcf2aca8700 1 -- 172.21.15.86:0/663076010 shutdown_connections 2024-06-06T04:25:48.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:48.301+0000 7fcf2aca8700 1 --2- 172.21.15.86:0/663076010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf24104090 0x7fcf2419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:48.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:48.301+0000 7fcf2aca8700 1 --2- 172.21.15.86:0/663076010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf241024f0 0x7fcf24198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:48.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:48.301+0000 7fcf2aca8700 1 --2- 172.21.15.86:0/663076010 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fcf140a2c20 0x7fcf140a50e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:48.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:48.301+0000 7fcf2aca8700 1 --2- 172.21.15.86:0/663076010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf241036f0 0x7fcf24198ba0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:48.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:48.301+0000 7fcf2aca8700 1 -- 172.21.15.86:0/663076010 >> 172.21.15.86:0/663076010 conn(0x7fcf240fda90 msgr2=0x7fcf24106ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:25:48.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:48.301+0000 7fcf2aca8700 1 -- 172.21.15.86:0/663076010 shutdown_connections 2024-06-06T04:25:48.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-06-06T04:25:48.301+0000 7fcf2aca8700 1 -- 172.21.15.86:0/663076010 wait complete. 2024-06-06T04:25:48.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:48 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:48.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:48 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:48.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:48 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:48.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:48 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:25:48.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:48 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:48.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:48 smithi086 ceph-mon[36053]: from='osd.10 [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T04:25:48.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:48 smithi086 ceph-mon[36053]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T04:25:48.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:48 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:48.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:48 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:25:48.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:48 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:48.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:48 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:48.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:48 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:48.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:48 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:48.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:48 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:25:48.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:48 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:48.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:48 smithi154 ceph-mon[36428]: from='osd.10 [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T04:25:48.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:48 smithi154 ceph-mon[36428]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T04:25:48.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:48 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:48.682 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:48 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:25:48.682 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:48 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:48.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:48 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:48.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:48 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:48.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:48 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:48.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:48 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:25:48.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:48 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:48.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:48 smithi045 ceph-mon[32299]: from='osd.10 [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T04:25:48.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:48 smithi045 ceph-mon[32299]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T04:25:48.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:48 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:48.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:48 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:25:48.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:48 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:48.973 DEBUG:teuthology.orchestra.run.smithi086:osd.10> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.10.service 2024-06-06T04:25:48.976 INFO:tasks.cephadm:Deploying osd.11 on smithi154 with /dev/vg_nvme/lv_1... 2024-06-06T04:25:48.976 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-06T04:25:49.224 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T04:25:49.468 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:25:49 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:25:49.455+0000 7f24520ea700 -1 osd.10 0 waiting for initial osdmap 2024-06-06T04:25:49.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:49 smithi086 ceph-mon[36053]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:49.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:49 smithi086 ceph-mon[36053]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-06T04:25:49.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:49 smithi086 ceph-mon[36053]: osdmap e71: 11 total, 10 up, 11 in 2024-06-06T04:25:49.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:49 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:49.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:49 smithi086 ceph-mon[36053]: from='osd.10 [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:25:49.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:49 smithi086 ceph-mon[36053]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:25:49.652 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:49 smithi154 ceph-mon[36428]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:49.652 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:49 smithi154 ceph-mon[36428]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-06T04:25:49.652 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:49 smithi154 ceph-mon[36428]: osdmap e71: 11 total, 10 up, 11 in 2024-06-06T04:25:49.653 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:49 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:49.653 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:49 smithi154 ceph-mon[36428]: from='osd.10 [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:25:49.653 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:49 smithi154 ceph-mon[36428]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:25:49.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:49 smithi045 ceph-mon[32299]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:49.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:49 smithi045 ceph-mon[32299]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-06T04:25:49.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:49 smithi045 ceph-mon[32299]: osdmap e71: 11 total, 10 up, 11 in 2024-06-06T04:25:49.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:49 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:49.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:49 smithi045 ceph-mon[32299]: from='osd.10 [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:25:49.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:49 smithi045 ceph-mon[32299]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-06-06T04:25:49.809 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:25:49 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:25:49.474+0000 7f244dedd700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T04:25:50.552 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:50 smithi086 ceph-mon[36053]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-06-06T04:25:50.552 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:50 smithi086 ceph-mon[36053]: osdmap e72: 11 total, 10 up, 11 in 2024-06-06T04:25:50.552 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:50 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:50.552 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:50 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:50.659 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:50 smithi154 ceph-mon[36428]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-06-06T04:25:50.659 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:50 smithi154 ceph-mon[36428]: osdmap e72: 11 total, 10 up, 11 in 2024-06-06T04:25:50.659 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:50 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:50.659 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:50 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:50.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:50 smithi045 ceph-mon[32299]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-06-06T04:25:50.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:50 smithi045 ceph-mon[32299]: osdmap e72: 11 total, 10 up, 11 in 2024-06-06T04:25:50.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:50 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:50.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:50 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-06T04:25:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-06T04:25:51.356 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-06-06T04:25:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-06-06T04:25:51.356 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122378 s, 857 MB/s 2024-06-06T04:25:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-06-06T04:25:51.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:51 smithi086 ceph-mon[36053]: purged_snaps scrub starts 2024-06-06T04:25:51.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:51 smithi086 ceph-mon[36053]: purged_snaps scrub ok 2024-06-06T04:25:51.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:51 smithi086 ceph-mon[36053]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:51.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:51 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:51.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:51 smithi086 ceph-mon[36053]: osd.10 [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877] boot 2024-06-06T04:25:51.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:51 smithi086 ceph-mon[36053]: osdmap e73: 11 total, 11 up, 11 in 2024-06-06T04:25:51.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:51 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:51.661 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:51 smithi154 ceph-mon[36428]: purged_snaps scrub starts 2024-06-06T04:25:51.661 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:51 smithi154 ceph-mon[36428]: purged_snaps scrub ok 2024-06-06T04:25:51.661 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:51 smithi154 ceph-mon[36428]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:51.661 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:51 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:51.661 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:51 smithi154 ceph-mon[36428]: osd.10 [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877] boot 2024-06-06T04:25:51.661 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:51 smithi154 ceph-mon[36428]: osdmap e73: 11 total, 11 up, 11 in 2024-06-06T04:25:51.661 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:51 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:51.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:51 smithi045 ceph-mon[32299]: purged_snaps scrub starts 2024-06-06T04:25:51.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:51 smithi045 ceph-mon[32299]: purged_snaps scrub ok 2024-06-06T04:25:51.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:51 smithi045 ceph-mon[32299]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-06-06T04:25:51.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:51 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:51.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:51 smithi045 ceph-mon[32299]: osd.10 [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877] boot 2024-06-06T04:25:51.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:51 smithi045 ceph-mon[32299]: osdmap e73: 11 total, 11 up, 11 in 2024-06-06T04:25:51.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:51 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T04:25:52.028 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi154:vg_nvme/lv_1 2024-06-06T04:25:52.278 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T04:25:52.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:52 smithi154 ceph-mon[36428]: osdmap e74: 11 total, 11 up, 11 in 2024-06-06T04:25:52.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:52 smithi154 ceph-mon[36428]: Detected new or changed devices on smithi086 2024-06-06T04:25:52.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:52 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:52.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:52 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:52.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:52 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:52.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:52 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:52.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:52 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:52.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:52 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:52.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:52 smithi154 ceph-mon[36428]: Adjusting osd_memory_target on smithi086 to 4272M 2024-06-06T04:25:52.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:52 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:52.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:52 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:52.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:52 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:25:52.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:52 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:52.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:52 smithi086 ceph-mon[36053]: osdmap e74: 11 total, 11 up, 11 in 2024-06-06T04:25:52.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:52 smithi086 ceph-mon[36053]: Detected new or changed devices on smithi086 2024-06-06T04:25:52.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:52 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:52.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:52 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:52.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:52 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:52.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:52 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:52.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:52 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:52.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:52 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:52.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:52 smithi086 ceph-mon[36053]: Adjusting osd_memory_target on smithi086 to 4272M 2024-06-06T04:25:52.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:52 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:52.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:52 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:52.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:52 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:25:52.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:52 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:52.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:52 smithi045 ceph-mon[32299]: osdmap e74: 11 total, 11 up, 11 in 2024-06-06T04:25:52.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:52 smithi045 ceph-mon[32299]: Detected new or changed devices on smithi086 2024-06-06T04:25:52.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:52 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:52.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:52 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:52.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:52 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:52.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:52 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:52.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:52 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:52.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:52 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:25:52.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:52 smithi045 ceph-mon[32299]: Adjusting osd_memory_target on smithi086 to 4272M 2024-06-06T04:25:52.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:52 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:52.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:52 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:52.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:52 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:25:52.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:52 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:25:53.610 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:53 smithi154 ceph-mon[36428]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:25:53.611 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:53 smithi154 ceph-mon[36428]: osdmap e75: 11 total, 11 up, 11 in 2024-06-06T04:25:53.611 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:53 smithi154 ceph-mon[36428]: osdmap e76: 11 total, 11 up, 11 in 2024-06-06T04:25:53.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.833+0000 7f7e36ad4700 1 -- 172.21.15.154:0/2178903238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e30102050 msgr2=0x7f7e30109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:25:53.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.833+0000 7f7e36ad4700 1 --2- 172.21.15.154:0/2178903238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e30102050 0x7f7e30109910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009990 tx=0x7f7e28009ca0 comp rx=0 tx=0).stop 2024-06-06T04:25:53.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.833+0000 7f7e36ad4700 1 -- 172.21.15.154:0/2178903238 shutdown_connections 2024-06-06T04:25:53.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.833+0000 7f7e36ad4700 1 --2- 172.21.15.154:0/2178903238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e30102050 0x7f7e30109910 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:53.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.833+0000 7f7e36ad4700 1 --2- 172.21.15.154:0/2178903238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e30102a00 0x7f7e30102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:53.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.833+0000 7f7e36ad4700 1 --2- 172.21.15.154:0/2178903238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e301016b0 0x7f7e30101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:53.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.833+0000 7f7e36ad4700 1 -- 172.21.15.154:0/2178903238 >> 172.21.15.154:0/2178903238 conn(0x7f7e300fc140 msgr2=0x7f7e300fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:25:53.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.833+0000 7f7e36ad4700 1 -- 172.21.15.154:0/2178903238 shutdown_connections 2024-06-06T04:25:53.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.833+0000 7f7e36ad4700 1 -- 172.21.15.154:0/2178903238 wait complete. 2024-06-06T04:25:53.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e36ad4700 1 Processor -- start 2024-06-06T04:25:53.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e36ad4700 1 -- start start 2024-06-06T04:25:53.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e36ad4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e301016b0 0x7f7e30101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:25:53.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e36ad4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e30102050 0x7f7e3019d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:25:53.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e36ad4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e30102a00 0x7f7e3019f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:25:53.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e36ad4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3010cea0 con 0x7f7e30102050 2024-06-06T04:25:53.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e36ad4700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3010cd20 con 0x7f7e301016b0 2024-06-06T04:25:53.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e36ad4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3010d020 con 0x7f7e30102a00 2024-06-06T04:25:53.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e35071700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e30102a00 0x7f7e3019f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:25:53.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e35071700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e30102a00 0x7f7e3019f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53690/0 (socket says 172.21.15.154:53690) 2024-06-06T04:25:53.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e35071700 1 -- 172.21.15.154:0/1437913223 learned_addr learned my addr 172.21.15.154:0/1437913223 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-06-06T04:25:53.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e35071700 1 -- 172.21.15.154:0/1437913223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e301016b0 msgr2=0x7f7e30101100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:25:53.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e34870700 1 --2- 172.21.15.154:0/1437913223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e301016b0 0x7f7e30101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:25:53.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e35071700 1 --2- 172.21.15.154:0/1437913223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e301016b0 0x7f7e30101100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:53.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e35071700 1 -- 172.21.15.154:0/1437913223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e30102050 msgr2=0x7f7e3019d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:25:53.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e27fff700 1 --2- 172.21.15.154:0/1437913223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e30102050 0x7f7e3019d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:25:53.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e35071700 1 --2- 172.21.15.154:0/1437913223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e30102050 0x7f7e3019d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:25:53.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.834+0000 7f7e35071700 1 -- 172.21.15.154:0/1437913223 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e301a0220 con 0x7f7e30102a00 2024-06-06T04:25:53.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.835+0000 7f7e35071700 1 --2- 172.21.15.154:0/1437913223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e30102a00 0x7f7e3019f950 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800b2e0 tx=0x7f7e2800f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:25:53.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.835+0000 7f7e25ffb700 1 -- 172.21.15.154:0/1437913223 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800be80 con 0x7f7e30102a00 2024-06-06T04:25:53.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.835+0000 7f7e25ffb700 1 -- 172.21.15.154:0/1437913223 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e28004ce0 con 0x7f7e30102a00 2024-06-06T04:25:53.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.835+0000 7f7e36ad4700 1 -- 172.21.15.154:0/1437913223 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e301a04b0 con 0x7f7e30102a00 2024-06-06T04:25:53.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.835+0000 7f7e25ffb700 1 -- 172.21.15.154:0/1437913223 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e280173f0 con 0x7f7e30102a00 2024-06-06T04:25:53.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.835+0000 7f7e36ad4700 1 -- 172.21.15.154:0/1437913223 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e301a08a0 con 0x7f7e30102a00 2024-06-06T04:25:53.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.836+0000 7f7e36ad4700 1 -- 172.21.15.154:0/1437913223 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e3002dca0 con 0x7f7e30102a00 2024-06-06T04:25:53.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.837+0000 7f7e25ffb700 1 -- 172.21.15.154:0/1437913223 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7e28028060 con 0x7f7e30102a00 2024-06-06T04:25:53.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.838+0000 7f7e25ffb700 1 --2- 172.21.15.154:0/1437913223 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7e180a2c20 0x7f7e180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:25:53.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.838+0000 7f7e25ffb700 1 -- 172.21.15.154:0/1437913223 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f7e280cfa30 con 0x7f7e30102a00 2024-06-06T04:25:53.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.838+0000 7f7e34870700 1 --2- 172.21.15.154:0/1437913223 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7e180a2c20 0x7f7e180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:25:53.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.838+0000 7f7e34870700 1 --2- 172.21.15.154:0/1437913223 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7e180a2c20 0x7f7e180a50e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e3006a9c0 tx=0x7f7e2c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:25:53.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:53.841+0000 7f7e25ffb700 1 -- 172.21.15.154:0/1437913223 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7e2805c850 con 0x7f7e30102a00 2024-06-06T04:25:53.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:53 smithi086 ceph-mon[36053]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:25:53.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:53 smithi086 ceph-mon[36053]: osdmap e75: 11 total, 11 up, 11 in 2024-06-06T04:25:53.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:53 smithi086 ceph-mon[36053]: osdmap e76: 11 total, 11 up, 11 in 2024-06-06T04:25:53.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:53 smithi045 ceph-mon[32299]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:25:53.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:53 smithi045 ceph-mon[32299]: osdmap e75: 11 total, 11 up, 11 in 2024-06-06T04:25:53.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:53 smithi045 ceph-mon[32299]: osdmap e76: 11 total, 11 up, 11 in 2024-06-06T04:25:54.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:25:54.050+0000 7f7e36ad4700 1 -- 172.21.15.154:0/1437913223 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7e300628d0 con 0x7f7e180a2c20 2024-06-06T04:25:54.919 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:54 smithi154 ceph-mon[36428]: osdmap e77: 11 total, 11 up, 11 in 2024-06-06T04:25:54.920 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:54 smithi154 ceph-mon[36428]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:25:54.920 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:25:54.920 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:25:54.920 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:55.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:54 smithi086 ceph-mon[36053]: osdmap e77: 11 total, 11 up, 11 in 2024-06-06T04:25:55.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:54 smithi086 ceph-mon[36053]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:25:55.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:54 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:25:55.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:54 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:25:55.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:54 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:55.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:54 smithi045 ceph-mon[32299]: osdmap e77: 11 total, 11 up, 11 in 2024-06-06T04:25:55.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:54 smithi045 ceph-mon[32299]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:25:55.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T04:25:55.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T04:25:55.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:25:56.083 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:55 smithi154 ceph-mon[36428]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-06-06T04:25:56.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:55 smithi086 ceph-mon[36053]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-06-06T04:25:56.132 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:55 smithi045 ceph-mon[32299]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-06-06T04:25:58.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:57 smithi086 ceph-mon[36053]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-06-06T04:25:58.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:57 smithi086 ceph-mon[36053]: from='client.? 172.21.15.154:0/970621571' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b349c4c-b672-480b-ac6a-8594dab0eaca"}]: dispatch 2024-06-06T04:25:58.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:57 smithi086 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b349c4c-b672-480b-ac6a-8594dab0eaca"}]: dispatch 2024-06-06T04:25:58.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:57 smithi086 ceph-mon[36053]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b349c4c-b672-480b-ac6a-8594dab0eaca"}]': finished 2024-06-06T04:25:58.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:57 smithi086 ceph-mon[36053]: osdmap e78: 12 total, 11 up, 12 in 2024-06-06T04:25:58.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:57 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:25:58.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:57 smithi086 ceph-mon[36053]: from='client.? 172.21.15.154:0/326976324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:25:58.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:57 smithi154 ceph-mon[36428]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-06-06T04:25:58.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:57 smithi154 ceph-mon[36428]: from='client.? 172.21.15.154:0/970621571' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b349c4c-b672-480b-ac6a-8594dab0eaca"}]: dispatch 2024-06-06T04:25:58.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:57 smithi154 ceph-mon[36428]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b349c4c-b672-480b-ac6a-8594dab0eaca"}]: dispatch 2024-06-06T04:25:58.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:57 smithi154 ceph-mon[36428]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b349c4c-b672-480b-ac6a-8594dab0eaca"}]': finished 2024-06-06T04:25:58.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:57 smithi154 ceph-mon[36428]: osdmap e78: 12 total, 11 up, 12 in 2024-06-06T04:25:58.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:57 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:25:58.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:57 smithi154 ceph-mon[36428]: from='client.? 172.21.15.154:0/326976324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:25:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:57 smithi045 ceph-mon[32299]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-06-06T04:25:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:57 smithi045 ceph-mon[32299]: from='client.? 172.21.15.154:0/970621571' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b349c4c-b672-480b-ac6a-8594dab0eaca"}]: dispatch 2024-06-06T04:25:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:57 smithi045 ceph-mon[32299]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b349c4c-b672-480b-ac6a-8594dab0eaca"}]: dispatch 2024-06-06T04:25:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:57 smithi045 ceph-mon[32299]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b349c4c-b672-480b-ac6a-8594dab0eaca"}]': finished 2024-06-06T04:25:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:57 smithi045 ceph-mon[32299]: osdmap e78: 12 total, 11 up, 12 in 2024-06-06T04:25:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:57 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:25:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:57 smithi045 ceph-mon[32299]: from='client.? 172.21.15.154:0/326976324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T04:26:00.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:25:59 smithi086 ceph-mon[36053]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-06-06T04:26:00.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:25:59 smithi154 ceph-mon[36428]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-06-06T04:26:00.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:25:59 smithi045 ceph-mon[32299]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-06-06T04:26:02.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:01 smithi154 ceph-mon[36428]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-06-06T04:26:02.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:01 smithi045 ceph-mon[32299]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-06-06T04:26:02.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:01 smithi086 ceph-mon[36053]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-06-06T04:26:04.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:03 smithi154 ceph-mon[36428]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:04.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:03 smithi045 ceph-mon[32299]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:04.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:03 smithi086 ceph-mon[36053]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:05.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-06T04:26:05.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:05.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-06T04:26:05.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-06T04:26:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:05 smithi045 ceph-mon[32299]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:05 smithi045 ceph-mon[32299]: Deploying daemon osd.11 on smithi154 2024-06-06T04:26:06.161 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:05 smithi154 ceph-mon[36428]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:06.161 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:05 smithi154 ceph-mon[36428]: Deploying daemon osd.11 on smithi154 2024-06-06T04:26:06.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:05 smithi086 ceph-mon[36053]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:06.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:05 smithi086 ceph-mon[36053]: Deploying daemon osd.11 on smithi154 2024-06-06T04:26:08.097 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:07 smithi154 ceph-mon[36428]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:08.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:07 smithi045 ceph-mon[32299]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:08.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:07 smithi086 ceph-mon[36053]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:10.059 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:09 smithi154 ceph-mon[36428]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:10.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:09 smithi045 ceph-mon[32299]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:10.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:09 smithi086 ceph-mon[36053]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:11.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:11 smithi154 ceph-mon[36428]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:11.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:11.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:11.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:26:11.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:11 smithi045 ceph-mon[32299]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:11.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:11.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:11.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:26:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:11 smithi086 ceph-mon[36053]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:26:13.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:13 smithi154 ceph-mon[36428]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:13.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:13 smithi045 ceph-mon[32299]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:13.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:13 smithi086 ceph-mon[36053]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:13.918 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 11 on host 'smithi154' 2024-06-06T04:26:13.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:13.917+0000 7f7e25ffb700 1 -- 172.21.15.154:0/1437913223 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7e300628d0 con 0x7f7e180a2c20 2024-06-06T04:26:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:13.920+0000 7f7e36ad4700 1 -- 172.21.15.154:0/1437913223 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7e180a2c20 msgr2=0x7f7e180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:13.920+0000 7f7e36ad4700 1 --2- 172.21.15.154:0/1437913223 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7e180a2c20 0x7f7e180a50e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e3006a9c0 tx=0x7f7e2c00b040 comp rx=0 tx=0).stop 2024-06-06T04:26:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:13.920+0000 7f7e36ad4700 1 -- 172.21.15.154:0/1437913223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e30102a00 msgr2=0x7f7e3019f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:13.920+0000 7f7e36ad4700 1 --2- 172.21.15.154:0/1437913223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e30102a00 0x7f7e3019f950 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800b2e0 tx=0x7f7e2800f740 comp rx=0 tx=0).stop 2024-06-06T04:26:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:13.920+0000 7f7e36ad4700 1 -- 172.21.15.154:0/1437913223 shutdown_connections 2024-06-06T04:26:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:13.920+0000 7f7e36ad4700 1 --2- 172.21.15.154:0/1437913223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e30102a00 0x7f7e3019f950 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:13.920+0000 7f7e36ad4700 1 --2- 172.21.15.154:0/1437913223 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7e180a2c20 0x7f7e180a50e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:13.920+0000 7f7e36ad4700 1 --2- 172.21.15.154:0/1437913223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e301016b0 0x7f7e30101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:13.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:13.920+0000 7f7e36ad4700 1 --2- 172.21.15.154:0/1437913223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e30102050 0x7f7e3019d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:13.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:13.920+0000 7f7e36ad4700 1 -- 172.21.15.154:0/1437913223 >> 172.21.15.154:0/1437913223 conn(0x7f7e300fc140 msgr2=0x7f7e30105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:13.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:13.920+0000 7f7e36ad4700 1 -- 172.21.15.154:0/1437913223 shutdown_connections 2024-06-06T04:26:13.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:13.920+0000 7f7e36ad4700 1 -- 172.21.15.154:0/1437913223 wait complete. 2024-06-06T04:26:14.624 DEBUG:teuthology.orchestra.run.smithi154:osd.11> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.11.service 2024-06-06T04:26:14.626 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-06-06T04:26:14.627 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-06T04:26:14.906 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:26:15.174 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:15.174 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:15.174 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:14 smithi045 ceph-mon[32299]: from='osd.11 [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T04:26:15.174 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:14 smithi045 ceph-mon[32299]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T04:26:15.174 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:15.174 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:14 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:15.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:15.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:15.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:14 smithi154 ceph-mon[36428]: from='osd.11 [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T04:26:15.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:14 smithi154 ceph-mon[36428]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T04:26:15.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:15.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:14 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:15.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:15.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:15.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:14 smithi086 ceph-mon[36053]: from='osd.11 [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T04:26:15.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:14 smithi086 ceph-mon[36053]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T04:26:15.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:15.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:14 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:15.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:15 smithi154 ceph-mon[36428]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:15.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:15 smithi154 ceph-mon[36428]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-06T04:26:15.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:15 smithi154 ceph-mon[36428]: osdmap e79: 12 total, 11 up, 12 in 2024-06-06T04:26:15.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:15 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:26:15.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:15 smithi154 ceph-mon[36428]: from='osd.11 [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:26:15.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:15 smithi154 ceph-mon[36428]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:26:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:15 smithi045 ceph-mon[32299]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:15 smithi045 ceph-mon[32299]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-06T04:26:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:15 smithi045 ceph-mon[32299]: osdmap e79: 12 total, 11 up, 12 in 2024-06-06T04:26:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:15 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:26:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:15 smithi045 ceph-mon[32299]: from='osd.11 [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:26:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:15 smithi045 ceph-mon[32299]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:26:16.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:15 smithi086 ceph-mon[36053]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:16.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:15 smithi086 ceph-mon[36053]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-06T04:26:16.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:15 smithi086 ceph-mon[36053]: osdmap e79: 12 total, 11 up, 12 in 2024-06-06T04:26:16.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:15 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:26:16.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:15 smithi086 ceph-mon[36053]: from='osd.11 [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:26:16.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:15 smithi086 ceph-mon[36053]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-06-06T04:26:16.624 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:26:16 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:26:16.387+0000 7fa916f4d700 -1 osd.11 0 waiting for initial osdmap 2024-06-06T04:26:16.624 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:26:16 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:26:16.404+0000 7fa90ed38700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T04:26:16.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.937+0000 7f0fbb53b700 1 -- 172.21.15.45:0/281027643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb41036e0 msgr2=0x7f0fb4103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:16.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.937+0000 7f0fbb53b700 1 --2- 172.21.15.45:0/281027643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb41036e0 0x7f0fb4103b40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80099c0 tx=0x7f0fa8009cd0 comp rx=0 tx=0).stop 2024-06-06T04:26:16.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.937+0000 7f0fbb53b700 1 -- 172.21.15.45:0/281027643 shutdown_connections 2024-06-06T04:26:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.937+0000 7f0fbb53b700 1 --2- 172.21.15.45:0/281027643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0fb4104080 0x7f0fb410a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.937+0000 7f0fbb53b700 1 --2- 172.21.15.45:0/281027643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb41024e0 0x7f0fb4102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.937+0000 7f0fbb53b700 1 --2- 172.21.15.45:0/281027643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb41036e0 0x7f0fb4103b40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.937+0000 7f0fbb53b700 1 -- 172.21.15.45:0/281027643 >> 172.21.15.45:0/281027643 conn(0x7f0fb40fda40 msgr2=0x7f0fb40ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.937+0000 7f0fbb53b700 1 -- 172.21.15.45:0/281027643 shutdown_connections 2024-06-06T04:26:16.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.937+0000 7f0fbb53b700 1 -- 172.21.15.45:0/281027643 wait complete. 2024-06-06T04:26:16.939 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:16 smithi045 ceph-mon[32299]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-06-06T04:26:16.939 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:16 smithi045 ceph-mon[32299]: osdmap e80: 12 total, 11 up, 12 in 2024-06-06T04:26:16.939 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:16 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:26:16.939 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:16 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:26:16.939 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:16 smithi045 ceph-mon[32299]: from='osd.11 ' entity='osd.11' 2024-06-06T04:26:16.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.938+0000 7f0fbb53b700 1 Processor -- start 2024-06-06T04:26:16.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.938+0000 7f0fbb53b700 1 -- start start 2024-06-06T04:26:16.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.938+0000 7f0fbb53b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb41024e0 0x7f0fb419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:16.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.938+0000 7f0fbb53b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb41036e0 0x7f0fb419f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:16.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.938+0000 7f0fbb53b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0fb4104080 0x7f0fb41a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:16.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.938+0000 7f0fbb53b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb410df80 con 0x7f0fb41024e0 2024-06-06T04:26:16.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.938+0000 7f0fbb53b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb410de00 con 0x7f0fb41036e0 2024-06-06T04:26:16.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.939+0000 7f0fbb53b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb410e100 con 0x7f0fb4104080 2024-06-06T04:26:16.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.939+0000 7f0fb92d7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb41024e0 0x7f0fb419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:16.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.939+0000 7f0fb92d7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb41024e0 0x7f0fb419ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60054/0 (socket says 172.21.15.45:60054) 2024-06-06T04:26:16.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.939+0000 7f0fb92d7700 1 -- 172.21.15.45:0/2044344298 learned_addr learned my addr 172.21.15.45:0/2044344298 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:26:16.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.939+0000 7f0fb9ad8700 1 --2- 172.21.15.45:0/2044344298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0fb4104080 0x7f0fb41a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:16.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.939+0000 7f0fb8ad6700 1 --2- 172.21.15.45:0/2044344298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb41036e0 0x7f0fb419f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:16.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.939+0000 7f0fb92d7700 1 -- 172.21.15.45:0/2044344298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0fb4104080 msgr2=0x7f0fb41a61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:16.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.939+0000 7f0fb92d7700 1 --2- 172.21.15.45:0/2044344298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0fb4104080 0x7f0fb41a61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:16.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.939+0000 7f0fb92d7700 1 -- 172.21.15.45:0/2044344298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb41036e0 msgr2=0x7f0fb419f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:16.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.939+0000 7f0fb92d7700 1 --2- 172.21.15.45:0/2044344298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb41036e0 0x7f0fb419f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:16.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.939+0000 7f0fb92d7700 1 -- 172.21.15.45:0/2044344298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb41a6880 con 0x7f0fb41024e0 2024-06-06T04:26:16.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.939+0000 7f0fb92d7700 1 --2- 172.21.15.45:0/2044344298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb41024e0 0x7f0fb419ec80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000d980 tx=0x7f0fb000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:16.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.939+0000 7f0fa67fc700 1 -- 172.21.15.45:0/2044344298 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0017030 con 0x7f0fb41024e0 2024-06-06T04:26:16.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.939+0000 7f0fa67fc700 1 -- 172.21.15.45:0/2044344298 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fb00044d0 con 0x7f0fb41024e0 2024-06-06T04:26:16.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.939+0000 7f0fbb53b700 1 -- 172.21.15.45:0/2044344298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb41a6b70 con 0x7f0fb41024e0 2024-06-06T04:26:16.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.939+0000 7f0fa67fc700 1 -- 172.21.15.45:0/2044344298 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0003970 con 0x7f0fb41024e0 2024-06-06T04:26:16.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.940+0000 7f0fbb53b700 1 -- 172.21.15.45:0/2044344298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb41a7100 con 0x7f0fb41024e0 2024-06-06T04:26:16.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.941+0000 7f0fbb53b700 1 -- 172.21.15.45:0/2044344298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb4069180 con 0x7f0fb41024e0 2024-06-06T04:26:16.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.942+0000 7f0fa67fc700 1 -- 172.21.15.45:0/2044344298 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0fb001a070 con 0x7f0fb41024e0 2024-06-06T04:26:16.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.942+0000 7f0fa67fc700 1 --2- 172.21.15.45:0/2044344298 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0fa00a2ca0 0x7f0fa00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:16.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.942+0000 7f0fb8ad6700 1 --2- 172.21.15.45:0/2044344298 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0fa00a2ca0 0x7f0fa00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:16.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.942+0000 7f0fa67fc700 1 -- 172.21.15.45:0/2044344298 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7f0fb00ce2f0 con 0x7f0fb41024e0 2024-06-06T04:26:16.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.943+0000 7f0fb8ad6700 1 --2- 172.21.15.45:0/2044344298 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0fa00a2ca0 0x7f0fa00a5160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8000c00 tx=0x7f0fa8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:16.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:16.947+0000 7f0fa67fc700 1 -- 172.21.15.45:0/2044344298 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0fb00232f0 con 0x7f0fb41024e0 2024-06-06T04:26:17.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:17.153+0000 7f0fbb53b700 1 -- 172.21.15.45:0/2044344298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0fb402dca0 con 0x7f0fb41024e0 2024-06-06T04:26:17.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:17.153+0000 7f0fa67fc700 1 -- 172.21.15.45:0/2044344298 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f0fb009b810 con 0x7f0fb41024e0 2024-06-06T04:26:17.154 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:26:17.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:17.155+0000 7f0fbb53b700 1 -- 172.21.15.45:0/2044344298 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0fa00a2ca0 msgr2=0x7f0fa00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:17.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:17.155+0000 7f0fbb53b700 1 --2- 172.21.15.45:0/2044344298 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0fa00a2ca0 0x7f0fa00a5160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8000c00 tx=0x7f0fa8005fb0 comp rx=0 tx=0).stop 2024-06-06T04:26:17.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:17.155+0000 7f0fbb53b700 1 -- 172.21.15.45:0/2044344298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb41024e0 msgr2=0x7f0fb419ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:17.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:17.155+0000 7f0fbb53b700 1 --2- 172.21.15.45:0/2044344298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb41024e0 0x7f0fb419ec80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000d980 tx=0x7f0fb000dc90 comp rx=0 tx=0).stop 2024-06-06T04:26:17.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:17.155+0000 7f0fbb53b700 1 -- 172.21.15.45:0/2044344298 shutdown_connections 2024-06-06T04:26:17.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:17.155+0000 7f0fbb53b700 1 --2- 172.21.15.45:0/2044344298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0fb4104080 0x7f0fb41a61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:17.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:17.155+0000 7f0fbb53b700 1 --2- 172.21.15.45:0/2044344298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fb41024e0 0x7f0fb419ec80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:17.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:17.155+0000 7f0fbb53b700 1 --2- 172.21.15.45:0/2044344298 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0fa00a2ca0 0x7f0fa00a5160 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:17.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:17.155+0000 7f0fbb53b700 1 --2- 172.21.15.45:0/2044344298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb41036e0 0x7f0fb419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:17.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:17.155+0000 7f0fbb53b700 1 -- 172.21.15.45:0/2044344298 >> 172.21.15.45:0/2044344298 conn(0x7f0fb40fda40 msgr2=0x7f0fb4104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:17.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:17.156+0000 7f0fbb53b700 1 -- 172.21.15.45:0/2044344298 shutdown_connections 2024-06-06T04:26:17.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:17.156+0000 7f0fbb53b700 1 -- 172.21.15.45:0/2044344298 wait complete. 2024-06-06T04:26:17.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:16 smithi154 ceph-mon[36428]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-06-06T04:26:17.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:16 smithi154 ceph-mon[36428]: osdmap e80: 12 total, 11 up, 12 in 2024-06-06T04:26:17.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:16 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:26:17.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:16 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:26:17.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:16 smithi154 ceph-mon[36428]: from='osd.11 ' entity='osd.11' 2024-06-06T04:26:17.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:16 smithi086 ceph-mon[36053]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-06-06T04:26:17.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:16 smithi086 ceph-mon[36053]: osdmap e80: 12 total, 11 up, 12 in 2024-06-06T04:26:17.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:16 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:26:17.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:16 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:26:17.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:16 smithi086 ceph-mon[36053]: from='osd.11 ' entity='osd.11' 2024-06-06T04:26:17.884 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1717647950,"num_in_osds":12,"osd_in_since":1717647956,"num_remapped_pgs":0} 2024-06-06T04:26:18.148 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:17 smithi154 ceph-mon[36428]: purged_snaps scrub starts 2024-06-06T04:26:18.148 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:17 smithi154 ceph-mon[36428]: purged_snaps scrub ok 2024-06-06T04:26:18.149 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:17 smithi154 ceph-mon[36428]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:18.149 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:17 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:26:18.149 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:17 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2044344298' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T04:26:18.149 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:17 smithi154 ceph-mon[36428]: osd.11 [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007] boot 2024-06-06T04:26:18.149 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:17 smithi154 ceph-mon[36428]: osdmap e81: 12 total, 12 up, 12 in 2024-06-06T04:26:18.149 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:17 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:26:18.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:17 smithi045 ceph-mon[32299]: purged_snaps scrub starts 2024-06-06T04:26:18.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:17 smithi045 ceph-mon[32299]: purged_snaps scrub ok 2024-06-06T04:26:18.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:17 smithi045 ceph-mon[32299]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:18.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:26:18.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:17 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2044344298' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T04:26:18.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:17 smithi045 ceph-mon[32299]: osd.11 [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007] boot 2024-06-06T04:26:18.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:17 smithi045 ceph-mon[32299]: osdmap e81: 12 total, 12 up, 12 in 2024-06-06T04:26:18.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:26:18.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:17 smithi086 ceph-mon[36053]: purged_snaps scrub starts 2024-06-06T04:26:18.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:17 smithi086 ceph-mon[36053]: purged_snaps scrub ok 2024-06-06T04:26:18.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:17 smithi086 ceph-mon[36053]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-06T04:26:18.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:17 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:26:18.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:17 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2044344298' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T04:26:18.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:17 smithi086 ceph-mon[36053]: osd.11 [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007] boot 2024-06-06T04:26:18.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:17 smithi086 ceph-mon[36053]: osdmap e81: 12 total, 12 up, 12 in 2024-06-06T04:26:18.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:17 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T04:26:18.885 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-06T04:26:19.135 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:26:19.518 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: Detected new or changed devices on smithi154 2024-06-06T04:26:19.518 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:19.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:19.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:26:19.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:26:19.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:26:19.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:26:19.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: Adjusting osd_memory_target on smithi154 to 4272M 2024-06-06T04:26:19.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:19.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:19.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:26:19.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:19.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:19.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:26:19.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:19.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:26:19.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:19 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:19.528 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: Detected new or changed devices on smithi154 2024-06-06T04:26:19.528 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:19.528 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:19.528 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:26:19.528 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:26:19.528 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:26:19.528 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:26:19.528 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: Adjusting osd_memory_target on smithi154 to 4272M 2024-06-06T04:26:19.528 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:19.528 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:19.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:26:19.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:19.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:19.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:26:19.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:19.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:26:19.529 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:19 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:19.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: Detected new or changed devices on smithi154 2024-06-06T04:26:19.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:19.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:19.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:26:19.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:26:19.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:26:19.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T04:26:19.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: Adjusting osd_memory_target on smithi154 to 4272M 2024-06-06T04:26:19.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:19.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:19.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:26:19.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:19.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:19.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:26:19.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:19.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:26:19.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:19 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:20.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:20 smithi045 ceph-mon[32299]: osdmap e82: 12 total, 12 up, 12 in 2024-06-06T04:26:20.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:20 smithi086 ceph-mon[36053]: osdmap e82: 12 total, 12 up, 12 in 2024-06-06T04:26:20.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:20 smithi154 ceph-mon[36428]: osdmap e82: 12 total, 12 up, 12 in 2024-06-06T04:26:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.935+0000 7f4e33e52700 1 -- 172.21.15.45:0/4166659473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e2c104090 msgr2=0x7f4e2c10a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.935+0000 7f4e33e52700 1 --2- 172.21.15.45:0/4166659473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e2c104090 0x7f4e2c10a940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400b0d0 tx=0x7f4e2400b3e0 comp rx=0 tx=0).stop 2024-06-06T04:26:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.935+0000 7f4e33e52700 1 -- 172.21.15.45:0/4166659473 shutdown_connections 2024-06-06T04:26:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.935+0000 7f4e33e52700 1 --2- 172.21.15.45:0/4166659473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e2c104090 0x7f4e2c10a940 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.935+0000 7f4e33e52700 1 --2- 172.21.15.45:0/4166659473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e2c1024f0 0x7f4e2c102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.935+0000 7f4e33e52700 1 --2- 172.21.15.45:0/4166659473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e2c1036f0 0x7f4e2c103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.935+0000 7f4e33e52700 1 -- 172.21.15.45:0/4166659473 >> 172.21.15.45:0/4166659473 conn(0x7f4e2c0fda90 msgr2=0x7f4e2c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.935+0000 7f4e33e52700 1 -- 172.21.15.45:0/4166659473 shutdown_connections 2024-06-06T04:26:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.935+0000 7f4e33e52700 1 -- 172.21.15.45:0/4166659473 wait complete. 2024-06-06T04:26:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e33e52700 1 Processor -- start 2024-06-06T04:26:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e33e52700 1 -- start start 2024-06-06T04:26:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e33e52700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e2c1024f0 0x7f4e2c076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e33e52700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e2c1036f0 0x7f4e2c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e33e52700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e2c104090 0x7f4e2c074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e33e52700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c10df70 con 0x7f4e2c104090 2024-06-06T04:26:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e33e52700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c10ddf0 con 0x7f4e2c1024f0 2024-06-06T04:26:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e33e52700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c10e0f0 con 0x7f4e2c1036f0 2024-06-06T04:26:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e323ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e2c104090 0x7f4e2c074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e323ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e2c104090 0x7f4e2c074dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42484/0 (socket says 172.21.15.45:42484) 2024-06-06T04:26:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e323ef700 1 -- 172.21.15.45:0/2660887981 learned_addr learned my addr 172.21.15.45:0/2660887981 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:26:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e313ed700 1 --2- 172.21.15.45:0/2660887981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e2c1036f0 0x7f4e2c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e31bee700 1 --2- 172.21.15.45:0/2660887981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e2c1024f0 0x7f4e2c076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e323ef700 1 -- 172.21.15.45:0/2660887981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e2c1036f0 msgr2=0x7f4e2c074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e323ef700 1 --2- 172.21.15.45:0/2660887981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e2c1036f0 0x7f4e2c074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e323ef700 1 -- 172.21.15.45:0/2660887981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e2c1024f0 msgr2=0x7f4e2c076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e323ef700 1 --2- 172.21.15.45:0/2660887981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e2c1024f0 0x7f4e2c076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e323ef700 1 -- 172.21.15.45:0/2660887981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c075670 con 0x7f4e2c104090 2024-06-06T04:26:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.936+0000 7f4e31bee700 1 --2- 172.21.15.45:0/2660887981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e2c1024f0 0x7f4e2c076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:26:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.937+0000 7f4e323ef700 1 --2- 172.21.15.45:0/2660887981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e2c104090 0x7f4e2c074dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009040 tx=0x7f4e24004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.937+0000 7f4e22ffd700 1 -- 172.21.15.45:0/2660887981 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2401f030 con 0x7f4e2c104090 2024-06-06T04:26:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.937+0000 7f4e22ffd700 1 -- 172.21.15.45:0/2660887981 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e24003a00 con 0x7f4e2c104090 2024-06-06T04:26:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.937+0000 7f4e33e52700 1 -- 172.21.15.45:0/2660887981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2c1a26b0 con 0x7f4e2c104090 2024-06-06T04:26:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.937+0000 7f4e22ffd700 1 -- 172.21.15.45:0/2660887981 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e240047c0 con 0x7f4e2c104090 2024-06-06T04:26:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.937+0000 7f4e33e52700 1 -- 172.21.15.45:0/2660887981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2c1a2b00 con 0x7f4e2c104090 2024-06-06T04:26:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.938+0000 7f4e33e52700 1 -- 172.21.15.45:0/2660887981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e2c069180 con 0x7f4e2c104090 2024-06-06T04:26:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.939+0000 7f4e22ffd700 1 -- 172.21.15.45:0/2660887981 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4e24003bb0 con 0x7f4e2c104090 2024-06-06T04:26:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.940+0000 7f4e22ffd700 1 --2- 172.21.15.45:0/2660887981 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4e180a2ca0 0x7f4e180a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.940+0000 7f4e31bee700 1 --2- 172.21.15.45:0/2660887981 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4e180a2ca0 0x7f4e180a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.940+0000 7f4e22ffd700 1 -- 172.21.15.45:0/2660887981 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4e240cfc80 con 0x7f4e2c104090 2024-06-06T04:26:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.940+0000 7f4e31bee700 1 --2- 172.21.15.45:0/2660887981 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4e180a2ca0 0x7f4e180a5160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e28005fd0 tx=0x7f4e28005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:20.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:20.944+0000 7f4e22ffd700 1 -- 172.21.15.45:0/2660887981 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4e2409d110 con 0x7f4e2c104090 2024-06-06T04:26:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:21.149+0000 7f4e33e52700 1 -- 172.21.15.45:0/2660887981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4e2c0630a0 con 0x7f4e2c104090 2024-06-06T04:26:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:21.150+0000 7f4e22ffd700 1 -- 172.21.15.45:0/2660887981 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f4e2409ca10 con 0x7f4e2c104090 2024-06-06T04:26:21.150 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:26:21.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:21.152+0000 7f4e33e52700 1 -- 172.21.15.45:0/2660887981 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4e180a2ca0 msgr2=0x7f4e180a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:21.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:21.152+0000 7f4e33e52700 1 --2- 172.21.15.45:0/2660887981 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4e180a2ca0 0x7f4e180a5160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e28005fd0 tx=0x7f4e28005f00 comp rx=0 tx=0).stop 2024-06-06T04:26:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:21.152+0000 7f4e33e52700 1 -- 172.21.15.45:0/2660887981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e2c104090 msgr2=0x7f4e2c074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:21.152+0000 7f4e33e52700 1 --2- 172.21.15.45:0/2660887981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e2c104090 0x7f4e2c074dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009040 tx=0x7f4e24004d50 comp rx=0 tx=0).stop 2024-06-06T04:26:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:21.152+0000 7f4e33e52700 1 -- 172.21.15.45:0/2660887981 shutdown_connections 2024-06-06T04:26:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:21.152+0000 7f4e33e52700 1 --2- 172.21.15.45:0/2660887981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e2c104090 0x7f4e2c074dc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:21.152+0000 7f4e33e52700 1 --2- 172.21.15.45:0/2660887981 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4e180a2ca0 0x7f4e180a5160 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:21.152+0000 7f4e33e52700 1 --2- 172.21.15.45:0/2660887981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e2c1024f0 0x7f4e2c076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:21.152+0000 7f4e33e52700 1 --2- 172.21.15.45:0/2660887981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e2c1036f0 0x7f4e2c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:21.152+0000 7f4e33e52700 1 -- 172.21.15.45:0/2660887981 >> 172.21.15.45:0/2660887981 conn(0x7f4e2c0fda90 msgr2=0x7f4e2c1069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:21.152+0000 7f4e33e52700 1 -- 172.21.15.45:0/2660887981 shutdown_connections 2024-06-06T04:26:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:21.152+0000 7f4e33e52700 1 -- 172.21.15.45:0/2660887981 wait complete. 2024-06-06T04:26:21.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:21 smithi045 ceph-mon[32299]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:21.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:21 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2660887981' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T04:26:21.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:21 smithi086 ceph-mon[36053]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:21.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:21 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2660887981' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T04:26:21.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:21 smithi154 ceph-mon[36428]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:21.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:21 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2660887981' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T04:26:21.774 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1717647977,"num_in_osds":12,"osd_in_since":1717647956,"num_remapped_pgs":0} 2024-06-06T04:26:21.775 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T04:26:22.020 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:26:23.609 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:23 smithi045 ceph-mon[32299]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:23.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:23 smithi154 ceph-mon[36428]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.834+0000 7f1a258bd700 1 -- 172.21.15.45:0/2118121141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2006b5a0 msgr2=0x7f1a2010f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.834+0000 7f1a258bd700 1 --2- 172.21.15.45:0/2118121141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2006b5a0 0x7f1a2010f340 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1a180099c0 tx=0x7f1a18009cd0 comp rx=0 tx=0).stop 2024-06-06T04:26:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.834+0000 7f1a258bd700 1 -- 172.21.15.45:0/2118121141 shutdown_connections 2024-06-06T04:26:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.834+0000 7f1a258bd700 1 --2- 172.21.15.45:0/2118121141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2006b5a0 0x7f1a2010f340 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.834+0000 7f1a258bd700 1 --2- 172.21.15.45:0/2118121141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a201085b0 0x7f1a20108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.834+0000 7f1a258bd700 1 --2- 172.21.15.45:0/2118121141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a2006abe0 0x7f1a2006b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.834+0000 7f1a258bd700 1 -- 172.21.15.45:0/2118121141 >> 172.21.15.45:0/2118121141 conn(0x7f1a20078250 msgr2=0x7f1a20078660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.834+0000 7f1a258bd700 1 -- 172.21.15.45:0/2118121141 shutdown_connections 2024-06-06T04:26:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.834+0000 7f1a258bd700 1 -- 172.21.15.45:0/2118121141 wait complete. 2024-06-06T04:26:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.835+0000 7f1a258bd700 1 Processor -- start 2024-06-06T04:26:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.835+0000 7f1a258bd700 1 -- start start 2024-06-06T04:26:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.835+0000 7f1a258bd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a2006abe0 0x7f1a2019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.835+0000 7f1a258bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2006b5a0 0x7f1a2019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.835+0000 7f1a258bd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a201085b0 0x7f1a20198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.835+0000 7f1a258bd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1a201126d0 con 0x7f1a2006b5a0 2024-06-06T04:26:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.835+0000 7f1a258bd700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a20112550 con 0x7f1a201085b0 2024-06-06T04:26:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.835+0000 7f1a258bd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1a20112850 con 0x7f1a2006abe0 2024-06-06T04:26:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.835+0000 7f1a1e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2006b5a0 0x7f1a2019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.835+0000 7f1a1e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2006b5a0 0x7f1a2019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42502/0 (socket says 172.21.15.45:42502) 2024-06-06T04:26:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.835+0000 7f1a1e7fc700 1 -- 172.21.15.45:0/1465025723 learned_addr learned my addr 172.21.15.45:0/1465025723 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:26:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.835+0000 7f1a1f7fe700 1 --2- 172.21.15.45:0/1465025723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a201085b0 0x7f1a20198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.835+0000 7f1a1effd700 1 --2- 172.21.15.45:0/1465025723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a2006abe0 0x7f1a2019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.836+0000 7f1a1e7fc700 1 -- 172.21.15.45:0/1465025723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a2006abe0 msgr2=0x7f1a2019e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.836+0000 7f1a1e7fc700 1 --2- 172.21.15.45:0/1465025723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a2006abe0 0x7f1a2019e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.836+0000 7f1a1e7fc700 1 -- 172.21.15.45:0/1465025723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a201085b0 msgr2=0x7f1a20198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.836+0000 7f1a1e7fc700 1 --2- 172.21.15.45:0/1465025723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a201085b0 0x7f1a20198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.836+0000 7f1a1e7fc700 1 -- 172.21.15.45:0/1465025723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a20199170 con 0x7f1a2006b5a0 2024-06-06T04:26:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.836+0000 7f1a1effd700 1 --2- 172.21.15.45:0/1465025723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a2006abe0 0x7f1a2019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:26:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.836+0000 7f1a1e7fc700 1 --2- 172.21.15.45:0/1465025723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2006b5a0 0x7f1a2019ed40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400c920 tx=0x7f1a1400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.836+0000 7f1a248bb700 1 -- 172.21.15.45:0/1465025723 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14007b00 con 0x7f1a2006b5a0 2024-06-06T04:26:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.836+0000 7f1a258bd700 1 -- 172.21.15.45:0/1465025723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a20199460 con 0x7f1a2006b5a0 2024-06-06T04:26:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.836+0000 7f1a248bb700 1 -- 172.21.15.45:0/1465025723 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a14004ce0 con 0x7f1a2006b5a0 2024-06-06T04:26:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.836+0000 7f1a248bb700 1 -- 172.21.15.45:0/1465025723 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14005550 con 0x7f1a2006b5a0 2024-06-06T04:26:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.836+0000 7f1a258bd700 1 -- 172.21.15.45:0/1465025723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a201a56d0 con 0x7f1a2006b5a0 2024-06-06T04:26:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.837+0000 7f1a258bd700 1 -- 172.21.15.45:0/1465025723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a20069180 con 0x7f1a2006b5a0 2024-06-06T04:26:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.838+0000 7f1a248bb700 1 -- 172.21.15.45:0/1465025723 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1a14004750 con 0x7f1a2006b5a0 2024-06-06T04:26:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.839+0000 7f1a248bb700 1 --2- 172.21.15.45:0/1465025723 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1a080a2c20 0x7f1a080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.839+0000 7f1a1effd700 1 --2- 172.21.15.45:0/1465025723 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1a080a2c20 0x7f1a080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.839+0000 7f1a248bb700 1 -- 172.21.15.45:0/1465025723 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1a140ceb60 con 0x7f1a2006b5a0 2024-06-06T04:26:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.840+0000 7f1a1effd700 1 --2- 172.21.15.45:0/1465025723 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1a080a2c20 0x7f1a080a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1a200ffe30 tx=0x7f1a10009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:23.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:23.843+0000 7f1a248bb700 1 -- 172.21.15.45:0/1465025723 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1a14014030 con 0x7f1a2006b5a0 2024-06-06T04:26:23.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:23 smithi086 ceph-mon[36053]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:24.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:24.050+0000 7f1a258bd700 1 -- 172.21.15.45:0/1465025723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a2002dca0 con 0x7f1a2006b5a0 2024-06-06T04:26:24.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:24.051+0000 7f1a248bb700 1 -- 172.21.15.45:0/1465025723 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18431 (secure 0 0 0) 0x7f1a1409bec0 con 0x7f1a2006b5a0 2024-06-06T04:26:24.052 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:26:24.053 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","created":"2024-06-06T04:19:02.626916+0000","modified":"2024-06-06T04:26:19.247434+0000","last_up_change":"2024-06-06T04:26:17.390429+0000","last_in_change":"2024-06-06T04:25:56.938388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T04:22:28.797137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d3cee4a-5025-45b3-b217-082af37e79db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6803","nonce":2111759250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6805","nonce":2111759250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6809","nonce":2111759250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6807","nonce":2111759250}]},"public_addr":"172.21.15.45:6803/2111759250","cluster_addr":"172.21.15.45:6805/2111759250","heartbeat_back_addr":"172.21.15.45:6809/2111759250","heartbeat_front_addr":"172.21.15.45:6807/2111759250","state":["exists","up"]},{"osd":1,"uuid":"75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6801","nonce":3732225287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6803","nonce":3732225287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6807","nonce":3732225287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6805","nonce":3732225287}]},"public_addr":"172.21.15.86:6801/3732225287","cluster_addr":"172.21.15.86:6803/3732225287","heartbeat_back_addr":"172.21.15.86:6807/3732225287","heartbeat_front_addr":"172.21.15.86:6805/3732225287","state":["exists","up"]},{"osd":2,"uuid":"bb20565f-f9ae-47e3-9604-c69f737cff4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6801","nonce":2898783039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6803","nonce":2898783039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6807","nonce":2898783039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6805","nonce":2898783039}]},"public_addr":"172.21.15.154:6801/2898783039","cluster_addr":"172.21.15.154:6803/2898783039","heartbeat_back_addr":"172.21.15.154:6807/2898783039","heartbeat_front_addr":"172.21.15.154:6805/2898783039","state":["exists","up"]},{"osd":3,"uuid":"e1160799-7811-4326-9792-7e6c7d189c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6811","nonce":3436753135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6813","nonce":3436753135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6817","nonce":3436753135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6815","nonce":3436753135}]},"public_addr":"172.21.15.45:6811/3436753135","cluster_addr":"172.21.15.45:6813/3436753135","heartbeat_back_addr":"172.21.15.45:6817/3436753135","heartbeat_front_addr":"172.21.15.45:6815/3436753135","state":["exists","up"]},{"osd":4,"uuid":"6b389bc8-e19a-47f3-83ff-7b782d146d0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6809","nonce":3030235796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6811","nonce":3030235796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6815","nonce":3030235796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6813","nonce":3030235796}]},"public_addr":"172.21.15.86:6809/3030235796","cluster_addr":"172.21.15.86:6811/3030235796","heartbeat_back_addr":"172.21.15.86:6815/3030235796","heartbeat_front_addr":"172.21.15.86:6813/3030235796","state":["exists","up"]},{"osd":5,"uuid":"069de7b2-89bf-4150-bed7-7b8832573dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6809","nonce":2222268646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6811","nonce":2222268646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6815","nonce":2222268646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6813","nonce":2222268646}]},"public_addr":"172.21.15.154:6809/2222268646","cluster_addr":"172.21.15.154:6811/2222268646","heartbeat_back_addr":"172.21.15.154:6815/2222268646","heartbeat_front_addr":"172.21.15.154:6813/2222268646","state":["exists","up"]},{"osd":6,"uuid":"4c70b627-eafa-4e27-ba2a-4d8e1e335962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6819","nonce":384734939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6821","nonce":384734939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6825","nonce":384734939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6823","nonce":384734939}]},"public_addr":"172.21.15.45:6819/384734939","cluster_addr":"172.21.15.45:6821/384734939","heartbeat_back_addr":"172.21.15.45:6825/384734939","heartbeat_front_addr":"172.21.15.45:6823/384734939","state":["exists","up"]},{"osd":7,"uuid":"3b09b875-be52-4be0-b116-7fd9b0c26705","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6817","nonce":1119791005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6819","nonce":1119791005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6823","nonce":1119791005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6821","nonce":1119791005}]},"public_addr":"172.21.15.86:6817/1119791005","cluster_addr":"172.21.15.86:6819/1119791005","heartbeat_back_addr":"172.21.15.86:6823/1119791005","heartbeat_front_addr":"172.21.15.86:6821/1119791005","state":["exists","up"]},{"osd":8,"uuid":"cdf1dd1b-5392-4836-8e65-4c7c5b688d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6817","nonce":3054812496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6819","nonce":3054812496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6823","nonce":3054812496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6821","nonce":3054812496}]},"public_addr":"172.21.15.154:6817/3054812496","cluster_addr":"172.21.15.154:6819/3054812496","heartbeat_back_addr":"172.21.15.154:6823/3054812496","heartbeat_front_addr":"172.21.15.154:6821/3054812496","state":["exists","up"]},{"osd":9,"uuid":"acede92d-2cc1-4967-a357-7e5e201e99d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6827","nonce":1360655510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6829","nonce":1360655510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6833","nonce":1360655510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6831","nonce":1360655510}]},"public_addr":"172.21.15.45:6827/1360655510","cluster_addr":"172.21.15.45:6829/1360655510","heartbeat_back_addr":"172.21.15.45:6833/1360655510","heartbeat_front_addr":"172.21.15.45:6831/1360655510","state":["exists","up"]},{"osd":10,"uuid":"a2d0c488-1892-4da4-adbf-952c22df0ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6825","nonce":1701691877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6827","nonce":1701691877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6831","nonce":1701691877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6829","nonce":1701691877}]},"public_addr":"172.21.15.86:6825/1701691877","cluster_addr":"172.21.15.86:6827/1701691877","heartbeat_back_addr":"172.21.15.86:6831/1701691877","heartbeat_front_addr":"172.21.15.86:6829/1701691877","state":["exists","up"]},{"osd":11,"uuid":"6b349c4c-b672-480b-ac6a-8594dab0eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6825","nonce":2065648007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6827","nonce":2065648007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6831","nonce":2065648007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6829","nonce":2065648007}]},"public_addr":"172.21.15.154:6825/2065648007","cluster_addr":"172.21.15.154:6827/2065648007","heartbeat_back_addr":"172.21.15.154:6831/2065648007","heartbeat_front_addr":"172.21.15.154:6829/2065648007","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:21:37.147394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:00.776617+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:24.786849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:50.487455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:14.869272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:39.815158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:05.972619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:31.410063+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:56.795119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:23.114884+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:49.256504+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:26:15.251967+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/36732849":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/1500318775":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6801/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6800/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/2041380391":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/2504003341":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/3798552599":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/498952822":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6800/1562511056":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6801/1562511056":"2024-06-07T04:19:37.026199+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T04:26:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:24.054+0000 7f1a258bd700 1 -- 172.21.15.45:0/1465025723 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1a080a2c20 msgr2=0x7f1a080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:24.054+0000 7f1a258bd700 1 --2- 172.21.15.45:0/1465025723 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1a080a2c20 0x7f1a080a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1a200ffe30 tx=0x7f1a10009570 comp rx=0 tx=0).stop 2024-06-06T04:26:24.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:24.054+0000 7f1a258bd700 1 -- 172.21.15.45:0/1465025723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2006b5a0 msgr2=0x7f1a2019ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:24.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:24.054+0000 7f1a258bd700 1 --2- 172.21.15.45:0/1465025723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2006b5a0 0x7f1a2019ed40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400c920 tx=0x7f1a1400cc30 comp rx=0 tx=0).stop 2024-06-06T04:26:24.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:24.054+0000 7f1a258bd700 1 -- 172.21.15.45:0/1465025723 shutdown_connections 2024-06-06T04:26:24.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:24.054+0000 7f1a258bd700 1 --2- 172.21.15.45:0/1465025723 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1a080a2c20 0x7f1a080a50e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:24.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:24.054+0000 7f1a258bd700 1 --2- 172.21.15.45:0/1465025723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a201085b0 0x7f1a20198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:24.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:24.054+0000 7f1a258bd700 1 --2- 172.21.15.45:0/1465025723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a2006abe0 0x7f1a2019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:24.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:24.054+0000 7f1a258bd700 1 --2- 172.21.15.45:0/1465025723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2006b5a0 0x7f1a2019ed40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:24.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:24.054+0000 7f1a258bd700 1 -- 172.21.15.45:0/1465025723 >> 172.21.15.45:0/1465025723 conn(0x7f1a20078250 msgr2=0x7f1a20100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:24.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:24.054+0000 7f1a258bd700 1 -- 172.21.15.45:0/1465025723 shutdown_connections 2024-06-06T04:26:24.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:24.054+0000 7f1a258bd700 1 -- 172.21.15.45:0/1465025723 wait complete. 2024-06-06T04:26:24.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:24 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1465025723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:26:24.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:24 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1465025723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:26:24.819 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-06T04:22:28.797137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-06T04:26:24.820 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-06T04:26:24.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:24 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1465025723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:26:25.075 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:26:25.449 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:25 smithi045 ceph-mon[32299]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:25.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:25 smithi154 ceph-mon[36428]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:25.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:25 smithi086 ceph-mon[36053]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:26.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.865+0000 7fc3fcd29700 1 -- 172.21.15.45:0/2757399022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f806ac60 msgr2=0x7fc3f806b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:26.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.865+0000 7fc3fcd29700 1 --2- 172.21.15.45:0/2757399022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f806ac60 0x7fc3f806b0e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0099c0 tx=0x7fc3ec009cd0 comp rx=0 tx=0).stop 2024-06-06T04:26:26.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.865+0000 7fc3fcd29700 1 -- 172.21.15.45:0/2757399022 shutdown_connections 2024-06-06T04:26:26.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.865+0000 7fc3fcd29700 1 --2- 172.21.15.45:0/2757399022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f806b620 0x7fc3f810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:26.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.865+0000 7fc3fcd29700 1 --2- 172.21.15.45:0/2757399022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3f8108580 0x7fc3f8108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:26.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.865+0000 7fc3fcd29700 1 --2- 172.21.15.45:0/2757399022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f806ac60 0x7fc3f806b0e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:26.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.865+0000 7fc3fcd29700 1 -- 172.21.15.45:0/2757399022 >> 172.21.15.45:0/2757399022 conn(0x7fc3f8078430 msgr2=0x7fc3f8078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:26.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.865+0000 7fc3fcd29700 1 -- 172.21.15.45:0/2757399022 shutdown_connections 2024-06-06T04:26:26.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.865+0000 7fc3fcd29700 1 -- 172.21.15.45:0/2757399022 wait complete. 2024-06-06T04:26:26.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3fcd29700 1 Processor -- start 2024-06-06T04:26:26.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3fcd29700 1 -- start start 2024-06-06T04:26:26.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3fcd29700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f806ac60 0x7fc3f819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:26.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3fcd29700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3f806b620 0x7fc3f819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:26.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3fcd29700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f8108580 0x7fc3f819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:26.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3fcd29700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8112680 con 0x7fc3f806ac60 2024-06-06T04:26:26.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3fcd29700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8112500 con 0x7fc3f8108580 2024-06-06T04:26:26.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3fcd29700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8112800 con 0x7fc3f806b620 2024-06-06T04:26:26.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3f659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f806ac60 0x7fc3f819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:26.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3f659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f806ac60 0x7fc3f819adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42522/0 (socket says 172.21.15.45:42522) 2024-06-06T04:26:26.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3f659c700 1 -- 172.21.15.45:0/1895210787 learned_addr learned my addr 172.21.15.45:0/1895210787 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:26:26.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3f5d9b700 1 --2- 172.21.15.45:0/1895210787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3f806b620 0x7fc3f819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:26.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3f6d9d700 1 --2- 172.21.15.45:0/1895210787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f8108580 0x7fc3f819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:26.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3f659c700 1 -- 172.21.15.45:0/1895210787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3f806b620 msgr2=0x7fc3f819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:26.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3f659c700 1 --2- 172.21.15.45:0/1895210787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3f806b620 0x7fc3f819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:26.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3f659c700 1 -- 172.21.15.45:0/1895210787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f8108580 msgr2=0x7fc3f819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:26.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3f659c700 1 --2- 172.21.15.45:0/1895210787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f8108580 0x7fc3f819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:26.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.866+0000 7fc3f659c700 1 -- 172.21.15.45:0/1895210787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f819fdc0 con 0x7fc3f806ac60 2024-06-06T04:26:26.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.867+0000 7fc3f659c700 1 --2- 172.21.15.45:0/1895210787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f806ac60 0x7fc3f819adb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800d950 tx=0x7fc3e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:26.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.867+0000 7fc3e77fe700 1 -- 172.21.15.45:0/1895210787 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8016070 con 0x7fc3f806ac60 2024-06-06T04:26:26.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.867+0000 7fc3fcd29700 1 -- 172.21.15.45:0/1895210787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f81a00b0 con 0x7fc3f806ac60 2024-06-06T04:26:26.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.867+0000 7fc3e77fe700 1 -- 172.21.15.45:0/1895210787 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3e800fd40 con 0x7fc3f806ac60 2024-06-06T04:26:26.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.867+0000 7fc3e77fe700 1 -- 172.21.15.45:0/1895210787 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e800e3b0 con 0x7fc3f806ac60 2024-06-06T04:26:26.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.867+0000 7fc3fcd29700 1 -- 172.21.15.45:0/1895210787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f81a7930 con 0x7fc3f806ac60 2024-06-06T04:26:26.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.868+0000 7fc3fcd29700 1 -- 172.21.15.45:0/1895210787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f8069180 con 0x7fc3f806ac60 2024-06-06T04:26:26.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.869+0000 7fc3e77fe700 1 -- 172.21.15.45:0/1895210787 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc3e8019070 con 0x7fc3f806ac60 2024-06-06T04:26:26.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.870+0000 7fc3e77fe700 1 --2- 172.21.15.45:0/1895210787 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc3e00a2c20 0x7fc3e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:26.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.870+0000 7fc3f5d9b700 1 --2- 172.21.15.45:0/1895210787 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc3e00a2c20 0x7fc3e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:26.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.870+0000 7fc3e77fe700 1 -- 172.21.15.45:0/1895210787 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc3e80ce6a0 con 0x7fc3f806ac60 2024-06-06T04:26:26.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.870+0000 7fc3f5d9b700 1 --2- 172.21.15.45:0/1895210787 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc3e00a2c20 0x7fc3e00a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc3f81033e0 tx=0x7fc3ec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:26.874+0000 7fc3e77fe700 1 -- 172.21.15.45:0/1895210787 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc3e809ba00 con 0x7fc3f806ac60 2024-06-06T04:26:27.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:27.076+0000 7fc3fcd29700 1 -- 172.21.15.45:0/1895210787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc3f81a7b60 con 0x7fc3f806ac60 2024-06-06T04:26:27.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:27.076+0000 7fc3e77fe700 1 -- 172.21.15.45:0/1895210787 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fc3e809b300 con 0x7fc3f806ac60 2024-06-06T04:26:27.077 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-06-06T04:26:27.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:27.078+0000 7fc3fcd29700 1 -- 172.21.15.45:0/1895210787 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc3e00a2c20 msgr2=0x7fc3e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:27.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:27.078+0000 7fc3fcd29700 1 --2- 172.21.15.45:0/1895210787 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc3e00a2c20 0x7fc3e00a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc3f81033e0 tx=0x7fc3ec005fb0 comp rx=0 tx=0).stop 2024-06-06T04:26:27.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:27.078+0000 7fc3fcd29700 1 -- 172.21.15.45:0/1895210787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f806ac60 msgr2=0x7fc3f819adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:27.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:27.078+0000 7fc3fcd29700 1 --2- 172.21.15.45:0/1895210787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f806ac60 0x7fc3f819adb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800d950 tx=0x7fc3e800dc60 comp rx=0 tx=0).stop 2024-06-06T04:26:27.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:27.078+0000 7fc3fcd29700 1 -- 172.21.15.45:0/1895210787 shutdown_connections 2024-06-06T04:26:27.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:27.078+0000 7fc3fcd29700 1 --2- 172.21.15.45:0/1895210787 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc3e00a2c20 0x7fc3e00a50e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:27.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:27.078+0000 7fc3fcd29700 1 --2- 172.21.15.45:0/1895210787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f8108580 0x7fc3f819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:27.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:27.078+0000 7fc3fcd29700 1 --2- 172.21.15.45:0/1895210787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f806ac60 0x7fc3f819adb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:27.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:27.078+0000 7fc3fcd29700 1 --2- 172.21.15.45:0/1895210787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3f806b620 0x7fc3f819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:27.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:27.078+0000 7fc3fcd29700 1 -- 172.21.15.45:0/1895210787 >> 172.21.15.45:0/1895210787 conn(0x7fc3f8078430 msgr2=0x7fc3f810d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:27.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:27.078+0000 7fc3fcd29700 1 -- 172.21.15.45:0/1895210787 shutdown_connections 2024-06-06T04:26:27.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:27.079+0000 7fc3fcd29700 1 -- 172.21.15.45:0/1895210787 wait complete. 2024-06-06T04:26:27.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:27 smithi154 ceph-mon[36428]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:27.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:27 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1895210787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T04:26:27.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:27 smithi045 ceph-mon[32299]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:27.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:27 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1895210787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T04:26:27.836 INFO:tasks.cephadm:Adding mds.a on smithi045 2024-06-06T04:26:27.836 INFO:tasks.cephadm:Adding mds.d on smithi045 2024-06-06T04:26:27.836 INFO:tasks.cephadm:Adding mds.g on smithi045 2024-06-06T04:26:27.836 INFO:tasks.cephadm:Adding mds.j on smithi045 2024-06-06T04:26:27.836 INFO:tasks.cephadm:Adding mds.b on smithi086 2024-06-06T04:26:27.836 INFO:tasks.cephadm:Adding mds.e on smithi086 2024-06-06T04:26:27.836 INFO:tasks.cephadm:Adding mds.h on smithi086 2024-06-06T04:26:27.836 INFO:tasks.cephadm:Adding mds.k on smithi086 2024-06-06T04:26:27.837 INFO:tasks.cephadm:Adding mds.c on smithi154 2024-06-06T04:26:27.837 INFO:tasks.cephadm:Adding mds.f on smithi154 2024-06-06T04:26:27.837 INFO:tasks.cephadm:Adding mds.i on smithi154 2024-06-06T04:26:27.837 INFO:tasks.cephadm:Adding mds.l on smithi154 2024-06-06T04:26:27.837 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l' 2024-06-06T04:26:27.858 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:27 smithi086 ceph-mon[36053]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:27 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1895210787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T04:26:28.088 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T04:26:29.505 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:29 smithi154 ceph-mon[36428]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:29.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:29 smithi045 ceph-mon[32299]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:29.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:29 smithi086 ceph-mon[36053]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:29.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.892+0000 7fbeed8ec700 1 -- 172.21.15.154:0/4204633667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbee8102ff0 msgr2=0x7fbee81053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:29.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.892+0000 7fbeed8ec700 1 --2- 172.21.15.154:0/4204633667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbee8102ff0 0x7fbee81053e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009990 tx=0x7fbed8009ca0 comp rx=0 tx=0).stop 2024-06-06T04:26:29.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.892+0000 7fbeed8ec700 1 -- 172.21.15.154:0/4204633667 shutdown_connections 2024-06-06T04:26:29.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.892+0000 7fbeed8ec700 1 --2- 172.21.15.154:0/4204633667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbee8108250 0x7fbee810a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:29.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.892+0000 7fbeed8ec700 1 --2- 172.21.15.154:0/4204633667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbee8102ff0 0x7fbee81053e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:29.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.892+0000 7fbeed8ec700 1 --2- 172.21.15.154:0/4204633667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbee8105920 0x7fbee8107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:29.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.892+0000 7fbeed8ec700 1 -- 172.21.15.154:0/4204633667 >> 172.21.15.154:0/4204633667 conn(0x7fbee80fc980 msgr2=0x7fbee80fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:29.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.892+0000 7fbeed8ec700 1 -- 172.21.15.154:0/4204633667 shutdown_connections 2024-06-06T04:26:29.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.892+0000 7fbeed8ec700 1 -- 172.21.15.154:0/4204633667 wait complete. 2024-06-06T04:26:29.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.892+0000 7fbeed8ec700 1 Processor -- start 2024-06-06T04:26:29.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.892+0000 7fbeed8ec700 1 -- start start 2024-06-06T04:26:29.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbeed8ec700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbee8102ff0 0x7fbee8102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbeed8ec700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbee8105920 0x7fbee8100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbeed8ec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbee8108250 0x7fbee81014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbeed8ec700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbee810dfa0 con 0x7fbee8108250 2024-06-06T04:26:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbeed8ec700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbee810de20 con 0x7fbee8105920 2024-06-06T04:26:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbeed8ec700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbee810e120 con 0x7fbee8102ff0 2024-06-06T04:26:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbee6ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbee8102ff0 0x7fbee8102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbee6ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbee8102ff0 0x7fbee8102950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48694/0 (socket says 172.21.15.154:48694) 2024-06-06T04:26:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbee6ffd700 1 -- 172.21.15.154:0/3480590975 learned_addr learned my addr 172.21.15.154:0/3480590975 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-06-06T04:26:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbee77fe700 1 --2- 172.21.15.154:0/3480590975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbee8108250 0x7fbee81014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbee6ffd700 1 -- 172.21.15.154:0/3480590975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbee8105920 msgr2=0x7fbee8100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbede7fc700 1 --2- 172.21.15.154:0/3480590975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbee8105920 0x7fbee8100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbee6ffd700 1 --2- 172.21.15.154:0/3480590975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbee8105920 0x7fbee8100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbee6ffd700 1 -- 172.21.15.154:0/3480590975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbee8108250 msgr2=0x7fbee81014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbee6ffd700 1 --2- 172.21.15.154:0/3480590975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbee8108250 0x7fbee81014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbee6ffd700 1 -- 172.21.15.154:0/3480590975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee8101d90 con 0x7fbee8102ff0 2024-06-06T04:26:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbee77fe700 1 --2- 172.21.15.154:0/3480590975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbee8108250 0x7fbee81014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:26:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbee6ffd700 1 --2- 172.21.15.154:0/3480590975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbee8102ff0 0x7fbee8102950 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbed800b5c0 tx=0x7fbed80047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.893+0000 7fbeec8ea700 1 -- 172.21.15.154:0/3480590975 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8027070 con 0x7fbee8102ff0 2024-06-06T04:26:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.894+0000 7fbeed8ec700 1 -- 172.21.15.154:0/3480590975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee81a26d0 con 0x7fbee8102ff0 2024-06-06T04:26:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.894+0000 7fbeec8ea700 1 -- 172.21.15.154:0/3480590975 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbed8022470 con 0x7fbee8102ff0 2024-06-06T04:26:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.894+0000 7fbeec8ea700 1 -- 172.21.15.154:0/3480590975 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed800f6c0 con 0x7fbee8102ff0 2024-06-06T04:26:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.894+0000 7fbeed8ec700 1 -- 172.21.15.154:0/3480590975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee81a2b20 con 0x7fbee8102ff0 2024-06-06T04:26:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.895+0000 7fbeed8ec700 1 -- 172.21.15.154:0/3480590975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee8069180 con 0x7fbee8102ff0 2024-06-06T04:26:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.896+0000 7fbeec8ea700 1 -- 172.21.15.154:0/3480590975 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbed802a070 con 0x7fbee8102ff0 2024-06-06T04:26:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.897+0000 7fbeec8ea700 1 --2- 172.21.15.154:0/3480590975 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbed40a2c20 0x7fbed40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.897+0000 7fbeec8ea700 1 -- 172.21.15.154:0/3480590975 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbed80cf970 con 0x7fbee8102ff0 2024-06-06T04:26:29.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.897+0000 7fbede7fc700 1 --2- 172.21.15.154:0/3480590975 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbed40a2c20 0x7fbed40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:29.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.897+0000 7fbede7fc700 1 --2- 172.21.15.154:0/3480590975 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbed40a2c20 0x7fbed40a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbed0005950 tx=0x7fbed00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:29.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:29.901+0000 7fbeec8ea700 1 -- 172.21.15.154:0/3480590975 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbed8013070 con 0x7fbee8102ff0 2024-06-06T04:26:30.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:30.105+0000 7fbeed8ec700 1 -- 172.21.15.154:0/3480590975 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l", "target": ["mon-mgr", ""]}) -- 0x7fbee80629d0 con 0x7fbed40a2c20 2024-06-06T04:26:30.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:30.113+0000 7fbeec8ea700 1 -- 172.21.15.154:0/3480590975 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fbee80629d0 con 0x7fbed40a2c20 2024-06-06T04:26:30.114 INFO:teuthology.orchestra.run.smithi154.stdout:Scheduled mds.all update... 2024-06-06T04:26:30.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:30.116+0000 7fbeed8ec700 1 -- 172.21.15.154:0/3480590975 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbed40a2c20 msgr2=0x7fbed40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:30.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:30.116+0000 7fbeed8ec700 1 --2- 172.21.15.154:0/3480590975 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbed40a2c20 0x7fbed40a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbed0005950 tx=0x7fbed00058e0 comp rx=0 tx=0).stop 2024-06-06T04:26:30.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:30.116+0000 7fbeed8ec700 1 -- 172.21.15.154:0/3480590975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbee8102ff0 msgr2=0x7fbee8102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:30.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:30.116+0000 7fbeed8ec700 1 --2- 172.21.15.154:0/3480590975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbee8102ff0 0x7fbee8102950 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbed800b5c0 tx=0x7fbed80047b0 comp rx=0 tx=0).stop 2024-06-06T04:26:30.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:30.116+0000 7fbeed8ec700 1 -- 172.21.15.154:0/3480590975 shutdown_connections 2024-06-06T04:26:30.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:30.116+0000 7fbeed8ec700 1 --2- 172.21.15.154:0/3480590975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbee8108250 0x7fbee81014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:30.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:30.116+0000 7fbeed8ec700 1 --2- 172.21.15.154:0/3480590975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbee8102ff0 0x7fbee8102950 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:30.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:30.116+0000 7fbeed8ec700 1 --2- 172.21.15.154:0/3480590975 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbed40a2c20 0x7fbed40a50e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:30.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:30.116+0000 7fbeed8ec700 1 --2- 172.21.15.154:0/3480590975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbee8105920 0x7fbee8100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:30.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:30.116+0000 7fbeed8ec700 1 -- 172.21.15.154:0/3480590975 >> 172.21.15.154:0/3480590975 conn(0x7fbee80fc980 msgr2=0x7fbee8103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:30.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:30.116+0000 7fbeed8ec700 1 -- 172.21.15.154:0/3480590975 shutdown_connections 2024-06-06T04:26:30.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-06-06T04:26:30.116+0000 7fbeed8ec700 1 -- 172.21.15.154:0/3480590975 wait complete. 2024-06-06T04:26:30.711 DEBUG:teuthology.orchestra.run.smithi045:mds.a> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.a.service 2024-06-06T04:26:30.714 DEBUG:teuthology.orchestra.run.smithi045:mds.d> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.d.service 2024-06-06T04:26:30.757 DEBUG:teuthology.orchestra.run.smithi045:mds.g> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.g.service 2024-06-06T04:26:30.801 DEBUG:teuthology.orchestra.run.smithi045:mds.j> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.j.service 2024-06-06T04:26:30.845 DEBUG:teuthology.orchestra.run.smithi086:mds.b> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.b.service 2024-06-06T04:26:30.848 DEBUG:teuthology.orchestra.run.smithi086:mds.e> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.e.service 2024-06-06T04:26:30.891 DEBUG:teuthology.orchestra.run.smithi086:mds.h> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.h.service 2024-06-06T04:26:30.934 DEBUG:teuthology.orchestra.run.smithi086:mds.k> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.k.service 2024-06-06T04:26:30.977 DEBUG:teuthology.orchestra.run.smithi154:mds.c> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.c.service 2024-06-06T04:26:30.980 DEBUG:teuthology.orchestra.run.smithi154:mds.f> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.f.service 2024-06-06T04:26:31.023 DEBUG:teuthology.orchestra.run.smithi154:mds.i> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.i.service 2024-06-06T04:26:31.067 DEBUG:teuthology.orchestra.run.smithi154:mds.l> sudo journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.l.service 2024-06-06T04:26:31.110 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-06-06T04:26:31.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:26:31.358 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:26:31.381 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:31 smithi154 ceph-mon[36428]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:26:31.381 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:31 smithi154 ceph-mon[36428]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l;count:12 2024-06-06T04:26:31.382 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:31 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:31.382 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:31 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:26:31.382 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:31 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:31.382 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:31 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:26:31.382 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:31 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:31.382 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:31 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:31.382 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:31 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:31.382 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:31 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:31.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:31 smithi045 ceph-mon[32299]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:26:31.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:31 smithi045 ceph-mon[32299]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l;count:12 2024-06-06T04:26:31.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:31 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:31.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:31 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:26:31.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:31 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:31.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:31 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:26:31.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:31 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:31.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:31 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:31.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:31 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:31.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:31 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:31.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:31 smithi086 ceph-mon[36053]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:26:31.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:31 smithi086 ceph-mon[36053]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l;count:12 2024-06-06T04:26:31.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:31 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:31.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:31 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:26:31.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:31 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:31.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:31 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:26:31.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:31 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:31.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:31 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:31.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:31 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:31.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:31 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:32.365 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:32 smithi154 ceph-mon[36428]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:32.366 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:32 smithi154 ceph-mon[36428]: Deploying daemon mds.l on smithi154 2024-06-06T04:26:32.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:32 smithi045 ceph-mon[32299]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:32.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:32 smithi045 ceph-mon[32299]: Deploying daemon mds.l on smithi154 2024-06-06T04:26:32.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:32 smithi086 ceph-mon[36053]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:32.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:32 smithi086 ceph-mon[36053]: Deploying daemon mds.l on smithi154 2024-06-06T04:26:32.931 INFO:journalctl@ceph.mds.l.smithi154.stdout:Jun 06 04:26:32 smithi154 systemd[1]: Starting Ceph mds.l for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:26:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.157+0000 7efcbbb8a700 1 -- 172.21.15.45:0/2652821993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcb4106370 msgr2=0x7efcb4106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.157+0000 7efcbbb8a700 1 --2- 172.21.15.45:0/2652821993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcb4106370 0x7efcb4106750 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efcb000b0a0 tx=0x7efcb000b3b0 comp rx=0 tx=0).stop 2024-06-06T04:26:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.157+0000 7efcbbb8a700 1 -- 172.21.15.45:0/2652821993 shutdown_connections 2024-06-06T04:26:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.157+0000 7efcbbb8a700 1 --2- 172.21.15.45:0/2652821993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcb4102550 0x7efcb410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.157+0000 7efcbbb8a700 1 --2- 172.21.15.45:0/2652821993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcb4106370 0x7efcb4106750 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.157+0000 7efcbbb8a700 1 --2- 172.21.15.45:0/2652821993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb4101b90 0x7efcb4102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.157+0000 7efcbbb8a700 1 -- 172.21.15.45:0/2652821993 >> 172.21.15.45:0/2652821993 conn(0x7efcb4077a00 msgr2=0x7efcb40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.157+0000 7efcbbb8a700 1 -- 172.21.15.45:0/2652821993 shutdown_connections 2024-06-06T04:26:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.157+0000 7efcbbb8a700 1 -- 172.21.15.45:0/2652821993 wait complete. 2024-06-06T04:26:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.158+0000 7efcbbb8a700 1 Processor -- start 2024-06-06T04:26:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.158+0000 7efcbbb8a700 1 -- start start 2024-06-06T04:26:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.158+0000 7efcbbb8a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcb4101b90 0x7efcb419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.158+0000 7efcbbb8a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb4102550 0x7efcb419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.158+0000 7efcbbb8a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcb4106370 0x7efcb41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.158+0000 7efcbbb8a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4111510 con 0x7efcb4101b90 2024-06-06T04:26:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.158+0000 7efcbbb8a700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4111390 con 0x7efcb4106370 2024-06-06T04:26:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.158+0000 7efcbbb8a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4111690 con 0x7efcb4102550 2024-06-06T04:26:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.158+0000 7efcb9926700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcb4101b90 0x7efcb419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.158+0000 7efcb9926700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcb4101b90 0x7efcb419e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34268/0 (socket says 172.21.15.45:34268) 2024-06-06T04:26:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.158+0000 7efcb9926700 1 -- 172.21.15.45:0/982684063 learned_addr learned my addr 172.21.15.45:0/982684063 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:26:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.159+0000 7efcb9125700 1 --2- 172.21.15.45:0/982684063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb4102550 0x7efcb419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.159+0000 7efcba127700 1 --2- 172.21.15.45:0/982684063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcb4106370 0x7efcb41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.159+0000 7efcb9926700 1 -- 172.21.15.45:0/982684063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb4102550 msgr2=0x7efcb419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.159+0000 7efcb9926700 1 --2- 172.21.15.45:0/982684063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb4102550 0x7efcb419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.159+0000 7efcb9926700 1 -- 172.21.15.45:0/982684063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcb4106370 msgr2=0x7efcb41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.159+0000 7efcb9926700 1 --2- 172.21.15.45:0/982684063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcb4106370 0x7efcb41988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.159+0000 7efcb9926700 1 -- 172.21.15.45:0/982684063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb4199000 con 0x7efcb4101b90 2024-06-06T04:26:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.159+0000 7efcb9926700 1 --2- 172.21.15.45:0/982684063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcb4101b90 0x7efcb419e6e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efcb000b070 tx=0x7efcb0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.159+0000 7efcaaffd700 1 -- 172.21.15.45:0/982684063 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0021070 con 0x7efcb4101b90 2024-06-06T04:26:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.159+0000 7efcaaffd700 1 -- 172.21.15.45:0/982684063 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcb0007e30 con 0x7efcb4101b90 2024-06-06T04:26:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.159+0000 7efcbbb8a700 1 -- 172.21.15.45:0/982684063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb4199290 con 0x7efcb4101b90 2024-06-06T04:26:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.159+0000 7efcaaffd700 1 -- 172.21.15.45:0/982684063 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0004780 con 0x7efcb4101b90 2024-06-06T04:26:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.159+0000 7efcbbb8a700 1 -- 172.21.15.45:0/982684063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb41a55b0 con 0x7efcb4101b90 2024-06-06T04:26:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.160+0000 7efcbbb8a700 1 -- 172.21.15.45:0/982684063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb4069180 con 0x7efcb4101b90 2024-06-06T04:26:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.161+0000 7efcaaffd700 1 -- 172.21.15.45:0/982684063 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efcb0007980 con 0x7efcb4101b90 2024-06-06T04:26:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.162+0000 7efcaaffd700 1 --2- 172.21.15.45:0/982684063 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7efca00a2c20 0x7efca00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.163+0000 7efcb9125700 1 --2- 172.21.15.45:0/982684063 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7efca00a2c20 0x7efca00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.163+0000 7efcaaffd700 1 -- 172.21.15.45:0/982684063 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efcb000f020 con 0x7efcb4101b90 2024-06-06T04:26:33.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.163+0000 7efcb9125700 1 --2- 172.21.15.45:0/982684063 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7efca00a2c20 0x7efca00a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efcb40fffd0 tx=0x7efca4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.166+0000 7efcaaffd700 1 -- 172.21.15.45:0/982684063 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efcb002a020 con 0x7efcb4101b90 2024-06-06T04:26:33.306 INFO:journalctl@ceph.mds.l.smithi154.stdout:Jun 06 04:26:33 smithi154 podman[66055]: 2024-06-06 04:26:32.938889344 +0000 UTC m=+0.017011222 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:26:33.306 INFO:journalctl@ceph.mds.l.smithi154.stdout:Jun 06 04:26:33 smithi154 podman[66055]: 2024-06-06 04:26:33.051337514 +0000 UTC m=+0.129459368 container create 3ca7f1261d921251343152bd000696d58fdb3062548cf4a7baeb41431c2c92ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:26:33.306 INFO:journalctl@ceph.mds.l.smithi154.stdout:Jun 06 04:26:33 smithi154 podman[66055]: 2024-06-06 04:26:33.279712145 +0000 UTC m=+0.357834004 container init 3ca7f1261d921251343152bd000696d58fdb3062548cf4a7baeb41431c2c92ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-l, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T04:26:33.306 INFO:journalctl@ceph.mds.l.smithi154.stdout:Jun 06 04:26:33 smithi154 podman[66055]: 2024-06-06 04:26:33.282399316 +0000 UTC m=+0.360521187 container start 3ca7f1261d921251343152bd000696d58fdb3062548cf4a7baeb41431c2c92ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-l, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-3d89acc) 2024-06-06T04:26:33.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.419+0000 7efcbbb8a700 1 -- 172.21.15.45:0/982684063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcb41a5780 con 0x7efcb4101b90 2024-06-06T04:26:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.419+0000 7efcaaffd700 1 -- 172.21.15.45:0/982684063 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7efcb002e3e0 con 0x7efcb4101b90 2024-06-06T04:26:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:26:33.420 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-06T04:26:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.421+0000 7efcbbb8a700 1 -- 172.21.15.45:0/982684063 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7efca00a2c20 msgr2=0x7efca00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.421+0000 7efcbbb8a700 1 --2- 172.21.15.45:0/982684063 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7efca00a2c20 0x7efca00a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efcb40fffd0 tx=0x7efca4009260 comp rx=0 tx=0).stop 2024-06-06T04:26:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.421+0000 7efcbbb8a700 1 -- 172.21.15.45:0/982684063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcb4101b90 msgr2=0x7efcb419e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.421+0000 7efcbbb8a700 1 --2- 172.21.15.45:0/982684063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcb4101b90 0x7efcb419e6e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efcb000b070 tx=0x7efcb0004d00 comp rx=0 tx=0).stop 2024-06-06T04:26:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.421+0000 7efcbbb8a700 1 -- 172.21.15.45:0/982684063 shutdown_connections 2024-06-06T04:26:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.421+0000 7efcbbb8a700 1 --2- 172.21.15.45:0/982684063 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7efca00a2c20 0x7efca00a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.421+0000 7efcbbb8a700 1 --2- 172.21.15.45:0/982684063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcb4106370 0x7efcb41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.421+0000 7efcbbb8a700 1 --2- 172.21.15.45:0/982684063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcb4101b90 0x7efcb419e6e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.421+0000 7efcbbb8a700 1 --2- 172.21.15.45:0/982684063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb4102550 0x7efcb419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.421+0000 7efcbbb8a700 1 -- 172.21.15.45:0/982684063 >> 172.21.15.45:0/982684063 conn(0x7efcb4077a00 msgr2=0x7efcb4100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.422+0000 7efcbbb8a700 1 -- 172.21.15.45:0/982684063 shutdown_connections 2024-06-06T04:26:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:33.422+0000 7efcbbb8a700 1 -- 172.21.15.45:0/982684063 wait complete. 2024-06-06T04:26:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 2 2024-06-06T04:26:33.681 INFO:journalctl@ceph.mds.l.smithi154.stdout:Jun 06 04:26:33 smithi154 ceph-mds[66074]: -- 172.21.15.154:0/1336934327 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556510d6ba40 con 0x556510d3f000 2024-06-06T04:26:33.681 INFO:journalctl@ceph.mds.l.smithi154.stdout:Jun 06 04:26:33 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-l[66070]: starting mds.l at 2024-06-06T04:26:33.681 INFO:journalctl@ceph.mds.l.smithi154.stdout:Jun 06 04:26:33 smithi154 bash[66055]: 3ca7f1261d921251343152bd000696d58fdb3062548cf4a7baeb41431c2c92ab 2024-06-06T04:26:33.681 INFO:journalctl@ceph.mds.l.smithi154.stdout:Jun 06 04:26:33 smithi154 systemd[1]: Started Ceph mds.l for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:26:34.417 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:34 smithi086 ceph-mon[36053]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:34.418 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:34 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.154:6832/2851773245,v1:172.21.15.154:6833/2851773245] up:boot 2024-06-06T04:26:34.418 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:34 smithi086 ceph-mon[36053]: fsmap 1 up:standby 2024-06-06T04:26:34.418 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T04:26:34.418 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:34 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/982684063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:26:34.418 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:34.418 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:34.418 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:34.418 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:34.418 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:34.418 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:34 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:34.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:34 smithi154 ceph-mon[36428]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:34.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:34 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.154:6832/2851773245,v1:172.21.15.154:6833/2851773245] up:boot 2024-06-06T04:26:34.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:34 smithi154 ceph-mon[36428]: fsmap 1 up:standby 2024-06-06T04:26:34.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T04:26:34.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:34 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/982684063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:26:34.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:34.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:34.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:34.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:34.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:34.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:34 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:34.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:34 smithi045 ceph-mon[32299]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:34.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:34 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.154:6832/2851773245,v1:172.21.15.154:6833/2851773245] up:boot 2024-06-06T04:26:34.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:34 smithi045 ceph-mon[32299]: fsmap 1 up:standby 2024-06-06T04:26:34.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T04:26:34.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:34 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/982684063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:26:34.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:34.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:34.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:34.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:34.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:34.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:34 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:35.394 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:35 smithi086 ceph-mon[36053]: Deploying daemon mds.k on smithi086 2024-06-06T04:26:35.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:35 smithi154 ceph-mon[36428]: Deploying daemon mds.k on smithi086 2024-06-06T04:26:35.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:35 smithi045 ceph-mon[32299]: Deploying daemon mds.k on smithi086 2024-06-06T04:26:36.061 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jun 06 04:26:35 smithi086 systemd[1]: Starting Ceph mds.k for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:26:36.138 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:26:36.333 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:36 smithi086 ceph-mon[36053]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:36.334 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jun 06 04:26:36 smithi086 podman[64992]: 2024-06-06 04:26:35.964041515 +0000 UTC m=+0.017132545 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:26:36.334 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jun 06 04:26:36 smithi086 podman[64992]: 2024-06-06 04:26:36.076899905 +0000 UTC m=+0.129990918 container create 21cbfb7954d198c5ffd524c9c7aa944d4779695648704ee7124bb4afa94c9f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-k, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-06T04:26:36.334 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jun 06 04:26:36 smithi086 podman[64992]: 2024-06-06 04:26:36.307436117 +0000 UTC m=+0.360527126 container init 21cbfb7954d198c5ffd524c9c7aa944d4779695648704ee7124bb4afa94c9f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-k, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-06T04:26:36.334 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jun 06 04:26:36 smithi086 podman[64992]: 2024-06-06 04:26:36.310334638 +0000 UTC m=+0.363425648 container start 21cbfb7954d198c5ffd524c9c7aa944d4779695648704ee7124bb4afa94c9f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-k, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-06T04:26:36.334 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jun 06 04:26:36 smithi086 ceph-mds[65011]: -- 172.21.15.86:0/1049418367 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dda0ad7a40 con 0x55dda0aab000 2024-06-06T04:26:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:26:36.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:36 smithi154 ceph-mon[36428]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:36.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:36 smithi045 ceph-mon[32299]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:36.609 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jun 06 04:26:36 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-k[65007]: starting mds.k at 2024-06-06T04:26:36.609 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jun 06 04:26:36 smithi086 bash[64992]: 21cbfb7954d198c5ffd524c9c7aa944d4779695648704ee7124bb4afa94c9f85 2024-06-06T04:26:36.609 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jun 06 04:26:36 smithi086 systemd[1]: Started Ceph mds.k for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:26:37.615 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:37 smithi045 ceph-mon[32299]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:37.616 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:37 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:37.616 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:37 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:37.616 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:37 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:37.616 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:37 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:37.616 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:37 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:37.616 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:37 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:37.616 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:37 smithi045 ceph-mon[32299]: Deploying daemon mds.j on smithi045 2024-06-06T04:26:37.616 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:37 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.86:6832/452357683,v1:172.21.15.86:6833/452357683] up:boot 2024-06-06T04:26:37.616 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:37 smithi045 ceph-mon[32299]: fsmap 2 up:standby 2024-06-06T04:26:37.616 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:37 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T04:26:37.722 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:37 smithi086 ceph-mon[36053]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:37.722 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:37 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:37.722 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:37 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:37.722 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:37 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:37.722 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:37 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:37.722 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:37 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:37.722 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:37 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:37.722 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:37 smithi086 ceph-mon[36053]: Deploying daemon mds.j on smithi045 2024-06-06T04:26:37.722 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:37 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.86:6832/452357683,v1:172.21.15.86:6833/452357683] up:boot 2024-06-06T04:26:37.722 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:37 smithi086 ceph-mon[36053]: fsmap 2 up:standby 2024-06-06T04:26:37.722 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:37 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T04:26:37.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:37 smithi154 ceph-mon[36428]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:37.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:37 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:37.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:37 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:37.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:37 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:37.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:37 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:37.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:37 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:37.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:37 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:37.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:37 smithi154 ceph-mon[36428]: Deploying daemon mds.j on smithi045 2024-06-06T04:26:37.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:37 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.86:6832/452357683,v1:172.21.15.86:6833/452357683] up:boot 2024-06-06T04:26:37.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:37 smithi154 ceph-mon[36428]: fsmap 2 up:standby 2024-06-06T04:26:37.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:37 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T04:26:39.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.380+0000 7fe571eba700 1 -- 172.21.15.45:0/538807867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c1030f0 msgr2=0x7fe56c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:39.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.380+0000 7fe571eba700 1 --2- 172.21.15.45:0/538807867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c1030f0 0x7fe56c107110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe5600099c0 tx=0x7fe560009cd0 comp rx=0 tx=0).stop 2024-06-06T04:26:39.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.381+0000 7fe571eba700 1 -- 172.21.15.45:0/538807867 shutdown_connections 2024-06-06T04:26:39.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.381+0000 7fe571eba700 1 --2- 172.21.15.45:0/538807867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe56c107840 0x7fe56c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:39.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.381+0000 7fe571eba700 1 --2- 172.21.15.45:0/538807867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe56c102740 0x7fe56c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:39.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.381+0000 7fe571eba700 1 --2- 172.21.15.45:0/538807867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c1030f0 0x7fe56c107110 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:39.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.381+0000 7fe571eba700 1 -- 172.21.15.45:0/538807867 >> 172.21.15.45:0/538807867 conn(0x7fe56c0795f0 msgr2=0x7fe56c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:39.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.381+0000 7fe571eba700 1 -- 172.21.15.45:0/538807867 shutdown_connections 2024-06-06T04:26:39.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.381+0000 7fe571eba700 1 -- 172.21.15.45:0/538807867 wait complete. 2024-06-06T04:26:39.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.381+0000 7fe571eba700 1 Processor -- start 2024-06-06T04:26:39.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.382+0000 7fe571eba700 1 -- start start 2024-06-06T04:26:39.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.382+0000 7fe571eba700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe56c102740 0x7fe56c1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:39.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.382+0000 7fe571eba700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe56c1030f0 0x7fe56c1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:39.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.382+0000 7fe571eba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c107840 0x7fe56c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:39.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.382+0000 7fe571eba700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c112600 con 0x7fe56c107840 2024-06-06T04:26:39.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.382+0000 7fe571eba700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c112480 con 0x7fe56c102740 2024-06-06T04:26:39.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.382+0000 7fe571eba700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c112780 con 0x7fe56c1030f0 2024-06-06T04:26:39.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.382+0000 7fe56bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c107840 0x7fe56c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:39.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.382+0000 7fe56bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c107840 0x7fe56c19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48540/0 (socket says 172.21.15.45:48540) 2024-06-06T04:26:39.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.382+0000 7fe56bfff700 1 -- 172.21.15.45:0/3080657574 learned_addr learned my addr 172.21.15.45:0/3080657574 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:26:39.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.382+0000 7fe56affd700 1 --2- 172.21.15.45:0/3080657574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe56c1030f0 0x7fe56c1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:39.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.382+0000 7fe56b7fe700 1 --2- 172.21.15.45:0/3080657574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe56c102740 0x7fe56c1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:39.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.383+0000 7fe56bfff700 1 -- 172.21.15.45:0/3080657574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe56c1030f0 msgr2=0x7fe56c1a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:39.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.383+0000 7fe56bfff700 1 --2- 172.21.15.45:0/3080657574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe56c1030f0 0x7fe56c1a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:39.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.383+0000 7fe56bfff700 1 -- 172.21.15.45:0/3080657574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe56c102740 msgr2=0x7fe56c1a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:39.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.383+0000 7fe56bfff700 1 --2- 172.21.15.45:0/3080657574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe56c102740 0x7fe56c1a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:39.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.383+0000 7fe56bfff700 1 -- 172.21.15.45:0/3080657574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c19b1a0 con 0x7fe56c107840 2024-06-06T04:26:39.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.383+0000 7fe56bfff700 1 --2- 172.21.15.45:0/3080657574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c107840 0x7fe56c19aa40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe56400ef30 tx=0x7fe56400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:39.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.383+0000 7fe568ff9700 1 -- 172.21.15.45:0/3080657574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640092e0 con 0x7fe56c107840 2024-06-06T04:26:39.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.383+0000 7fe571eba700 1 -- 172.21.15.45:0/3080657574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56c19b430 con 0x7fe56c107840 2024-06-06T04:26:39.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.383+0000 7fe571eba700 1 -- 172.21.15.45:0/3080657574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe56c1a7750 con 0x7fe56c107840 2024-06-06T04:26:39.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.383+0000 7fe568ff9700 1 -- 172.21.15.45:0/3080657574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe564004bb0 con 0x7fe56c107840 2024-06-06T04:26:39.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.383+0000 7fe568ff9700 1 -- 172.21.15.45:0/3080657574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564004020 con 0x7fe56c107840 2024-06-06T04:26:39.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.384+0000 7fe571eba700 1 -- 172.21.15.45:0/3080657574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe558005330 con 0x7fe56c107840 2024-06-06T04:26:39.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.386+0000 7fe568ff9700 1 -- 172.21.15.45:0/3080657574 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe56401b030 con 0x7fe56c107840 2024-06-06T04:26:39.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.386+0000 7fe568ff9700 1 --2- 172.21.15.45:0/3080657574 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5540a2ca0 0x7fe5540a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:39.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.386+0000 7fe56b7fe700 1 --2- 172.21.15.45:0/3080657574 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5540a2ca0 0x7fe5540a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:39.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.386+0000 7fe568ff9700 1 -- 172.21.15.45:0/3080657574 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe5640ceee0 con 0x7fe56c107840 2024-06-06T04:26:39.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.387+0000 7fe56b7fe700 1 --2- 172.21.15.45:0/3080657574 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5540a2ca0 0x7fe5540a5160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe56c19bb80 tx=0x7fe55c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:39.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.394+0000 7fe568ff9700 1 -- 172.21.15.45:0/3080657574 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe564022020 con 0x7fe56c107840 2024-06-06T04:26:39.625 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:39 smithi045 ceph-mon[32299]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:39.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.663+0000 7fe571eba700 1 -- 172.21.15.45:0/3080657574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe558005620 con 0x7fe56c107840 2024-06-06T04:26:39.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.663+0000 7fe568ff9700 1 -- 172.21.15.45:0/3080657574 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7fe5640a2070 con 0x7fe56c107840 2024-06-06T04:26:39.664 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:26:39.664 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-06T04:26:39.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.665+0000 7fe571eba700 1 -- 172.21.15.45:0/3080657574 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5540a2ca0 msgr2=0x7fe5540a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:39.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.665+0000 7fe571eba700 1 --2- 172.21.15.45:0/3080657574 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5540a2ca0 0x7fe5540a5160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe56c19bb80 tx=0x7fe55c00a380 comp rx=0 tx=0).stop 2024-06-06T04:26:39.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.665+0000 7fe571eba700 1 -- 172.21.15.45:0/3080657574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c107840 msgr2=0x7fe56c19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:39.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.665+0000 7fe571eba700 1 --2- 172.21.15.45:0/3080657574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c107840 0x7fe56c19aa40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe56400ef30 tx=0x7fe56400c3a0 comp rx=0 tx=0).stop 2024-06-06T04:26:39.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.665+0000 7fe571eba700 1 -- 172.21.15.45:0/3080657574 shutdown_connections 2024-06-06T04:26:39.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.665+0000 7fe571eba700 1 --2- 172.21.15.45:0/3080657574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c107840 0x7fe56c19aa40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:39.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.665+0000 7fe571eba700 1 --2- 172.21.15.45:0/3080657574 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5540a2ca0 0x7fe5540a5160 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:39.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.665+0000 7fe571eba700 1 --2- 172.21.15.45:0/3080657574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe56c102740 0x7fe56c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:39.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.665+0000 7fe571eba700 1 --2- 172.21.15.45:0/3080657574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe56c1030f0 0x7fe56c1a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:39.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.665+0000 7fe571eba700 1 -- 172.21.15.45:0/3080657574 >> 172.21.15.45:0/3080657574 conn(0x7fe56c0795f0 msgr2=0x7fe56c101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:39.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.666+0000 7fe571eba700 1 -- 172.21.15.45:0/3080657574 shutdown_connections 2024-06-06T04:26:39.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:39.666+0000 7fe571eba700 1 -- 172.21.15.45:0/3080657574 wait complete. 2024-06-06T04:26:39.668 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 3 2024-06-06T04:26:39.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:39 smithi086 ceph-mon[36053]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:39.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:39 smithi154 ceph-mon[36428]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:40.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:40 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3080657574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:26:40.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:40 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3080657574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:26:40.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:40 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3080657574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:26:40.976 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jun 06 04:26:40 smithi045 systemd[1]: Started Ceph mds.j for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:26:42.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:41 smithi154 ceph-mon[36428]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:42.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:41 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:42.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:41 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:42.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:41 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:42.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:41 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:42.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:41 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:42.182 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:41 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:42.182 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:41 smithi154 ceph-mon[36428]: Deploying daemon mds.d on smithi045 2024-06-06T04:26:42.182 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:41 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.45:6834/1876588301,v1:172.21.15.45:6835/1876588301] up:boot 2024-06-06T04:26:42.182 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:41 smithi154 ceph-mon[36428]: fsmap 3 up:standby 2024-06-06T04:26:42.182 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:41 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T04:26:42.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:41 smithi045 ceph-mon[32299]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:42.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:42.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:42.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:42.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:42.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:42.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:42.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:41 smithi045 ceph-mon[32299]: Deploying daemon mds.d on smithi045 2024-06-06T04:26:42.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:41 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.45:6834/1876588301,v1:172.21.15.45:6835/1876588301] up:boot 2024-06-06T04:26:42.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:41 smithi045 ceph-mon[32299]: fsmap 3 up:standby 2024-06-06T04:26:42.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:41 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T04:26:42.327 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:26:42.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:41 smithi086 ceph-mon[36053]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:42.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:41 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:42.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:41 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:42.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:41 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:42.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:41 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:42.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:41 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:42.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:41 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:42.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:41 smithi086 ceph-mon[36053]: Deploying daemon mds.d on smithi045 2024-06-06T04:26:42.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:41 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.45:6834/1876588301,v1:172.21.15.45:6835/1876588301] up:boot 2024-06-06T04:26:42.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:41 smithi086 ceph-mon[36053]: fsmap 3 up:standby 2024-06-06T04:26:42.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:41 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T04:26:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:26:43.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:43 smithi045 ceph-mon[32299]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:44.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:43 smithi154 ceph-mon[36428]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:44.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:43 smithi086 ceph-mon[36053]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:44.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.710+0000 7fe5140b0700 1 -- 172.21.15.45:0/861891632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c107440 msgr2=0x7fe50c107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:44.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.710+0000 7fe5140b0700 1 --2- 172.21.15.45:0/861891632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c107440 0x7fe50c107820 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe508009990 tx=0x7fe508009ca0 comp rx=0 tx=0).stop 2024-06-06T04:26:44.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.710+0000 7fe5140b0700 1 -- 172.21.15.45:0/861891632 shutdown_connections 2024-06-06T04:26:44.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.710+0000 7fe5140b0700 1 --2- 172.21.15.45:0/861891632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe50c101d80 0x7fe50c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:44.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.710+0000 7fe5140b0700 1 --2- 172.21.15.45:0/861891632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c107440 0x7fe50c107820 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:44.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.710+0000 7fe5140b0700 1 --2- 172.21.15.45:0/861891632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe50c1013c0 0x7fe50c101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:44.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.710+0000 7fe5140b0700 1 -- 172.21.15.45:0/861891632 >> 172.21.15.45:0/861891632 conn(0x7fe50c077400 msgr2=0x7fe50c077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:44.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.711+0000 7fe5140b0700 1 -- 172.21.15.45:0/861891632 shutdown_connections 2024-06-06T04:26:44.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.711+0000 7fe5140b0700 1 -- 172.21.15.45:0/861891632 wait complete. 2024-06-06T04:26:44.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.711+0000 7fe5140b0700 1 Processor -- start 2024-06-06T04:26:44.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.711+0000 7fe5140b0700 1 -- start start 2024-06-06T04:26:44.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.711+0000 7fe5140b0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe50c1013c0 0x7fe50c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:44.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.711+0000 7fe5140b0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c101d80 0x7fe50c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:44.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.711+0000 7fe5140b0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe50c107440 0x7fe50c1a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:44.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe5140b0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1115b0 con 0x7fe50c101d80 2024-06-06T04:26:44.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe5140b0700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c111430 con 0x7fe50c1013c0 2024-06-06T04:26:44.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe5140b0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c111730 con 0x7fe50c107440 2024-06-06T04:26:44.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe51164b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c101d80 0x7fe50c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:44.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe51164b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c101d80 0x7fe50c199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48570/0 (socket says 172.21.15.45:48570) 2024-06-06T04:26:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe51164b700 1 -- 172.21.15.45:0/1537788353 learned_addr learned my addr 172.21.15.45:0/1537788353 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:26:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe51264d700 1 --2- 172.21.15.45:0/1537788353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe50c107440 0x7fe50c1a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe511e4c700 1 --2- 172.21.15.45:0/1537788353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe50c1013c0 0x7fe50c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe51164b700 1 -- 172.21.15.45:0/1537788353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe50c107440 msgr2=0x7fe50c1a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe51164b700 1 --2- 172.21.15.45:0/1537788353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe50c107440 0x7fe50c1a4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe51164b700 1 -- 172.21.15.45:0/1537788353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe50c1013c0 msgr2=0x7fe50c199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe51164b700 1 --2- 172.21.15.45:0/1537788353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe50c1013c0 0x7fe50c199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe51164b700 1 -- 172.21.15.45:0/1537788353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50c1a5010 con 0x7fe50c101d80 2024-06-06T04:26:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe51164b700 1 --2- 172.21.15.45:0/1537788353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c101d80 0x7fe50c199f90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00b530 tx=0x7fe4fc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe502ffd700 1 -- 172.21.15.45:0/1537788353 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc015020 con 0x7fe50c101d80 2024-06-06T04:26:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe502ffd700 1 -- 172.21.15.45:0/1537788353 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4fc011ce0 con 0x7fe50c101d80 2024-06-06T04:26:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe5140b0700 1 -- 172.21.15.45:0/1537788353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50c1a5240 con 0x7fe50c101d80 2024-06-06T04:26:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe502ffd700 1 -- 172.21.15.45:0/1537788353 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc010d60 con 0x7fe50c101d80 2024-06-06T04:26:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.712+0000 7fe5140b0700 1 -- 172.21.15.45:0/1537788353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe50c1a5750 con 0x7fe50c101d80 2024-06-06T04:26:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.713+0000 7fe5140b0700 1 -- 172.21.15.45:0/1537788353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50c109900 con 0x7fe50c101d80 2024-06-06T04:26:44.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.715+0000 7fe502ffd700 1 -- 172.21.15.45:0/1537788353 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe4fc018020 con 0x7fe50c101d80 2024-06-06T04:26:44.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.715+0000 7fe502ffd700 1 --2- 172.21.15.45:0/1537788353 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe4f80a2c20 0x7fe4f80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:44.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.716+0000 7fe511e4c700 1 --2- 172.21.15.45:0/1537788353 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe4f80a2c20 0x7fe4f80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:44.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.716+0000 7fe502ffd700 1 -- 172.21.15.45:0/1537788353 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe4fc0ce980 con 0x7fe50c101d80 2024-06-06T04:26:44.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.716+0000 7fe511e4c700 1 --2- 172.21.15.45:0/1537788353 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe4f80a2c20 0x7fe4f80a50e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe508009750 tx=0x7fe50800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:44.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.719+0000 7fe502ffd700 1 -- 172.21.15.45:0/1537788353 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe4fc01f020 con 0x7fe50c101d80 2024-06-06T04:26:44.944 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jun 06 04:26:44 smithi045 systemd[1]: Started Ceph mds.d for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:26:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.984+0000 7fe5140b0700 1 -- 172.21.15.45:0/1537788353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe50c102ab0 con 0x7fe50c101d80 2024-06-06T04:26:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.985+0000 7fe502ffd700 1 -- 172.21.15.45:0/1537788353 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7fe4fc09bd80 con 0x7fe50c101d80 2024-06-06T04:26:44.986 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:26:44.986 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-06T04:26:44.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.987+0000 7fe5140b0700 1 -- 172.21.15.45:0/1537788353 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe4f80a2c20 msgr2=0x7fe4f80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:44.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.987+0000 7fe5140b0700 1 --2- 172.21.15.45:0/1537788353 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe4f80a2c20 0x7fe4f80a50e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe508009750 tx=0x7fe50800b540 comp rx=0 tx=0).stop 2024-06-06T04:26:44.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.987+0000 7fe5140b0700 1 -- 172.21.15.45:0/1537788353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c101d80 msgr2=0x7fe50c199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:44.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.987+0000 7fe5140b0700 1 --2- 172.21.15.45:0/1537788353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c101d80 0x7fe50c199f90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00b530 tx=0x7fe4fc00b8f0 comp rx=0 tx=0).stop 2024-06-06T04:26:44.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.987+0000 7fe5140b0700 1 -- 172.21.15.45:0/1537788353 shutdown_connections 2024-06-06T04:26:44.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.987+0000 7fe5140b0700 1 --2- 172.21.15.45:0/1537788353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe50c107440 0x7fe50c1a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:44.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.987+0000 7fe5140b0700 1 --2- 172.21.15.45:0/1537788353 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe4f80a2c20 0x7fe4f80a50e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:44.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.987+0000 7fe5140b0700 1 --2- 172.21.15.45:0/1537788353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe50c1013c0 0x7fe50c199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:44.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.987+0000 7fe5140b0700 1 --2- 172.21.15.45:0/1537788353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c101d80 0x7fe50c199f90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:44.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.987+0000 7fe5140b0700 1 -- 172.21.15.45:0/1537788353 >> 172.21.15.45:0/1537788353 conn(0x7fe50c077400 msgr2=0x7fe50c1002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:44.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.987+0000 7fe5140b0700 1 -- 172.21.15.45:0/1537788353 shutdown_connections 2024-06-06T04:26:44.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:44.987+0000 7fe5140b0700 1 -- 172.21.15.45:0/1537788353 wait complete. 2024-06-06T04:26:44.989 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 5 2024-06-06T04:26:46.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:45 smithi086 ceph-mon[36053]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:46.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:45 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:46.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:45 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:46.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:45 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.45:6836/3532043360,v1:172.21.15.45:6837/3532043360] up:boot 2024-06-06T04:26:46.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:45 smithi086 ceph-mon[36053]: fsmap 4 up:standby 2024-06-06T04:26:46.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:45 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T04:26:46.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:45 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:46.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:45 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:46.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:45 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:46.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:45 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:46.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:45 smithi086 ceph-mon[36053]: Deploying daemon mds.e on smithi086 2024-06-06T04:26:46.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:45 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1537788353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:26:46.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:45 smithi154 ceph-mon[36428]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:46.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:45 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:46.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:45 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:46.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:45 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.45:6836/3532043360,v1:172.21.15.45:6837/3532043360] up:boot 2024-06-06T04:26:46.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:45 smithi154 ceph-mon[36428]: fsmap 4 up:standby 2024-06-06T04:26:46.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:45 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T04:26:46.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:45 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:46.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:45 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:46.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:45 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:46.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:45 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:46.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:45 smithi154 ceph-mon[36428]: Deploying daemon mds.e on smithi086 2024-06-06T04:26:46.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:45 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1537788353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:26:46.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:45 smithi045 ceph-mon[32299]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:46.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:45 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:46.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:45 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:46.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:45 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.45:6836/3532043360,v1:172.21.15.45:6837/3532043360] up:boot 2024-06-06T04:26:46.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:45 smithi045 ceph-mon[32299]: fsmap 4 up:standby 2024-06-06T04:26:46.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:45 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T04:26:46.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:45 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:46.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:45 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:46.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:45 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:46.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:45 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:46.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:45 smithi045 ceph-mon[32299]: Deploying daemon mds.e on smithi086 2024-06-06T04:26:46.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:45 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1537788353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:26:47.359 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:26:47 smithi086 systemd[1]: Starting Ceph mds.e for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:26:47.635 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:26:47.667 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:26:47 smithi086 podman[65305]: 2024-06-06 04:26:47.288352706 +0000 UTC m=+0.016660536 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:26:47.667 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:26:47 smithi086 podman[65305]: 2024-06-06 04:26:47.426615115 +0000 UTC m=+0.154922920 container create 3feb81881c0dff815d7ab28886432df5e29cb055023d9ea920506f497b871c27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-e, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-06T04:26:47.667 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:26:47 smithi086 podman[65305]: 2024-06-06 04:26:47.667771871 +0000 UTC m=+0.396079675 container init 3feb81881c0dff815d7ab28886432df5e29cb055023d9ea920506f497b871c27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-06T04:26:47.907 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:26:47.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:47 smithi045 ceph-mon[32299]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:47.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:47 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:47.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:47 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:47.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:47 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:47.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:47 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:47.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:47 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:47.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:47 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:48.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:47 smithi086 ceph-mon[36053]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:48.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:47 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:48.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:47 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:48.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:47 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:48.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:47 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:48.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:47 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:48.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:47 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:48.109 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:26:47 smithi086 podman[65305]: 2024-06-06 04:26:47.670678887 +0000 UTC m=+0.398986707 container start 3feb81881c0dff815d7ab28886432df5e29cb055023d9ea920506f497b871c27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-e, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-06T04:26:48.110 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:26:47 smithi086 ceph-mds[65324]: -- 172.21.15.86:0/894812036 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562ebd15fa40 con 0x562ebd132800 2024-06-06T04:26:48.110 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:26:47 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-e[65320]: starting mds.e at 2024-06-06T04:26:48.110 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:26:47 smithi086 bash[65305]: 3feb81881c0dff815d7ab28886432df5e29cb055023d9ea920506f497b871c27 2024-06-06T04:26:48.110 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:26:47 smithi086 systemd[1]: Started Ceph mds.e for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:26:48.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:47 smithi154 ceph-mon[36428]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:48.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:47 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:48.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:47 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:48.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:47 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:48.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:47 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:48.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:47 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:48.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:47 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:49.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:48 smithi086 ceph-mon[36053]: Deploying daemon mds.f on smithi154 2024-06-06T04:26:49.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:48 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.86:6834/4268654313,v1:172.21.15.86:6835/4268654313] up:boot 2024-06-06T04:26:49.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:48 smithi086 ceph-mon[36053]: fsmap 5 up:standby 2024-06-06T04:26:49.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:48 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T04:26:49.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:48 smithi154 ceph-mon[36428]: Deploying daemon mds.f on smithi154 2024-06-06T04:26:49.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:48 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.86:6834/4268654313,v1:172.21.15.86:6835/4268654313] up:boot 2024-06-06T04:26:49.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:48 smithi154 ceph-mon[36428]: fsmap 5 up:standby 2024-06-06T04:26:49.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:48 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T04:26:49.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:48 smithi045 ceph-mon[32299]: Deploying daemon mds.f on smithi154 2024-06-06T04:26:49.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:48 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.86:6834/4268654313,v1:172.21.15.86:6835/4268654313] up:boot 2024-06-06T04:26:49.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:48 smithi045 ceph-mon[32299]: fsmap 5 up:standby 2024-06-06T04:26:49.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:48 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T04:26:49.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.575+0000 7f92ef269700 1 -- 172.21.15.45:0/3088305017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92e806ab30 msgr2=0x7f92e810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:49.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.575+0000 7f92ef269700 1 --2- 172.21.15.45:0/3088305017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92e806ab30 0x7f92e810f320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f92e000b0a0 tx=0x7f92e000b3b0 comp rx=0 tx=0).stop 2024-06-06T04:26:49.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.575+0000 7f92ef269700 1 -- 172.21.15.45:0/3088305017 shutdown_connections 2024-06-06T04:26:49.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.575+0000 7f92ef269700 1 --2- 172.21.15.45:0/3088305017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92e806ab30 0x7f92e810f320 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:49.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.575+0000 7f92ef269700 1 --2- 172.21.15.45:0/3088305017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e8108500 0x7f92e81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:49.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.575+0000 7f92ef269700 1 --2- 172.21.15.45:0/3088305017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92e806a170 0x7f92e806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:49.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.575+0000 7f92ef269700 1 -- 172.21.15.45:0/3088305017 >> 172.21.15.45:0/3088305017 conn(0x7f92e8078340 msgr2=0x7f92e8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:49.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.576+0000 7f92ef269700 1 -- 172.21.15.45:0/3088305017 shutdown_connections 2024-06-06T04:26:49.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.576+0000 7f92ef269700 1 -- 172.21.15.45:0/3088305017 wait complete. 2024-06-06T04:26:49.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.577+0000 7f92ef269700 1 Processor -- start 2024-06-06T04:26:49.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.577+0000 7f92ef269700 1 -- start start 2024-06-06T04:26:49.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.577+0000 7f92ef269700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92e806a170 0x7f92e8101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:49.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.577+0000 7f92ef269700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e806ab30 0x7f92e81019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:49.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.577+0000 7f92ef269700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92e8108500 0x7f92e8101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:49.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.577+0000 7f92ef269700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f92e8112610 con 0x7f92e8108500 2024-06-06T04:26:49.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.577+0000 7f92ef269700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92e8112490 con 0x7f92e806ab30 2024-06-06T04:26:49.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.577+0000 7f92ef269700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f92e8112790 con 0x7f92e806a170 2024-06-06T04:26:49.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.577+0000 7f92ed005700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92e806a170 0x7f92e8101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:49.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.577+0000 7f92ed005700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92e806a170 0x7f92e8101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:38086/0 (socket says 172.21.15.45:38086) 2024-06-06T04:26:49.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.577+0000 7f92ed005700 1 -- 172.21.15.45:0/714666860 learned_addr learned my addr 172.21.15.45:0/714666860 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:26:49.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.577+0000 7f92ec804700 1 --2- 172.21.15.45:0/714666860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e806ab30 0x7f92e81019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:49.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.578+0000 7f92ed806700 1 --2- 172.21.15.45:0/714666860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92e8108500 0x7f92e8101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:49.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.578+0000 7f92ed005700 1 -- 172.21.15.45:0/714666860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e806ab30 msgr2=0x7f92e81019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:49.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.578+0000 7f92ed005700 1 --2- 172.21.15.45:0/714666860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e806ab30 0x7f92e81019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:49.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.578+0000 7f92ed005700 1 -- 172.21.15.45:0/714666860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92e8108500 msgr2=0x7f92e8101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:49.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.578+0000 7f92ed005700 1 --2- 172.21.15.45:0/714666860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92e8108500 0x7f92e8101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:49.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.578+0000 7f92ed005700 1 -- 172.21.15.45:0/714666860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e81a7010 con 0x7f92e806a170 2024-06-06T04:26:49.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.578+0000 7f92ed005700 1 --2- 172.21.15.45:0/714666860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92e806a170 0x7f92e8101460 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f92e400d950 tx=0x7f92e400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:49.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.578+0000 7f92de7fc700 1 -- 172.21.15.45:0/714666860 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4016070 con 0x7f92e806a170 2024-06-06T04:26:49.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.578+0000 7f92ef269700 1 -- 172.21.15.45:0/714666860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e81a7300 con 0x7f92e806a170 2024-06-06T04:26:49.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.578+0000 7f92de7fc700 1 -- 172.21.15.45:0/714666860 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92e400fd40 con 0x7f92e806a170 2024-06-06T04:26:49.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.578+0000 7f92de7fc700 1 -- 172.21.15.45:0/714666860 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e400e3b0 con 0x7f92e806a170 2024-06-06T04:26:49.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.578+0000 7f92ef269700 1 -- 172.21.15.45:0/714666860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e81a7890 con 0x7f92e806a170 2024-06-06T04:26:49.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.579+0000 7f92ef269700 1 -- 172.21.15.45:0/714666860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e8069180 con 0x7f92e806a170 2024-06-06T04:26:49.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.581+0000 7f92de7fc700 1 -- 172.21.15.45:0/714666860 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f92e4019070 con 0x7f92e806a170 2024-06-06T04:26:49.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.581+0000 7f92de7fc700 1 --2- 172.21.15.45:0/714666860 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f92d40a2ca0 0x7f92d40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:49.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.581+0000 7f92ec804700 1 --2- 172.21.15.45:0/714666860 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f92d40a2ca0 0x7f92d40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:49.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.581+0000 7f92de7fc700 1 -- 172.21.15.45:0/714666860 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f92e40ce540 con 0x7f92e806a170 2024-06-06T04:26:49.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.582+0000 7f92ec804700 1 --2- 172.21.15.45:0/714666860 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f92d40a2ca0 0x7f92d40a5160 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f92e80789b0 tx=0x7f92d800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:49.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.584+0000 7f92de7fc700 1 -- 172.21.15.45:0/714666860 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f92e409b940 con 0x7f92e806a170 2024-06-06T04:26:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.835+0000 7f92ef269700 1 -- 172.21.15.45:0/714666860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92e802dca0 con 0x7f92e806a170 2024-06-06T04:26:49.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.836+0000 7f92de7fc700 1 -- 172.21.15.45:0/714666860 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f92e409b240 con 0x7f92e806a170 2024-06-06T04:26:49.837 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:26:49.837 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-06-06T04:26:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.838+0000 7f92ef269700 1 -- 172.21.15.45:0/714666860 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f92d40a2ca0 msgr2=0x7f92d40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.838+0000 7f92ef269700 1 --2- 172.21.15.45:0/714666860 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f92d40a2ca0 0x7f92d40a5160 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f92e80789b0 tx=0x7f92d800b410 comp rx=0 tx=0).stop 2024-06-06T04:26:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.838+0000 7f92ef269700 1 -- 172.21.15.45:0/714666860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92e806a170 msgr2=0x7f92e8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.838+0000 7f92ef269700 1 --2- 172.21.15.45:0/714666860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92e806a170 0x7f92e8101460 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f92e400d950 tx=0x7f92e400dc60 comp rx=0 tx=0).stop 2024-06-06T04:26:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.838+0000 7f92ef269700 1 -- 172.21.15.45:0/714666860 shutdown_connections 2024-06-06T04:26:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.838+0000 7f92ef269700 1 --2- 172.21.15.45:0/714666860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92e8108500 0x7f92e8101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.838+0000 7f92ef269700 1 --2- 172.21.15.45:0/714666860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92e806a170 0x7f92e8101460 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.838+0000 7f92ef269700 1 --2- 172.21.15.45:0/714666860 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f92d40a2ca0 0x7f92d40a5160 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.838+0000 7f92ef269700 1 --2- 172.21.15.45:0/714666860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e806ab30 0x7f92e81019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.838+0000 7f92ef269700 1 -- 172.21.15.45:0/714666860 >> 172.21.15.45:0/714666860 conn(0x7f92e8078340 msgr2=0x7f92e8100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.839+0000 7f92ef269700 1 -- 172.21.15.45:0/714666860 shutdown_connections 2024-06-06T04:26:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:49.839+0000 7f92ef269700 1 -- 172.21.15.45:0/714666860 wait complete. 2024-06-06T04:26:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 6 2024-06-06T04:26:49.989 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:49 smithi086 ceph-mon[36053]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:49.989 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:49 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/714666860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:26:50.110 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:49 smithi154 ceph-mon[36428]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:50.110 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:49 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/714666860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:26:50.111 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:26:49 smithi154 systemd[1]: Starting Ceph mds.f for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:26:50.154 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:49 smithi045 ceph-mon[32299]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:50.155 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:49 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/714666860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:26:50.392 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:26:50 smithi154 podman[66358]: 2024-06-06 04:26:50.013709966 +0000 UTC m=+0.018069243 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:26:50.392 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:26:50 smithi154 podman[66358]: 2024-06-06 04:26:50.131262214 +0000 UTC m=+0.135621468 container create b1c620c0d3beb514b3b12e72d77e1a4c0e5a272d134ac48cebb41c58a23ffedf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-f, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-06-06T04:26:50.393 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:26:50 smithi154 podman[66358]: 2024-06-06 04:26:50.392680577 +0000 UTC m=+0.397039833 container init b1c620c0d3beb514b3b12e72d77e1a4c0e5a272d134ac48cebb41c58a23ffedf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-06T04:26:50.680 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:26:50 smithi154 podman[66358]: 2024-06-06 04:26:50.395601787 +0000 UTC m=+0.399961052 container start b1c620c0d3beb514b3b12e72d77e1a4c0e5a272d134ac48cebb41c58a23ffedf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-f, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-06T04:26:50.680 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:26:50 smithi154 ceph-mds[66377]: -- 172.21.15.154:0/3189298467 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c94546da40 con 0x55c945440c00 2024-06-06T04:26:50.680 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:26:50 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-f[66373]: starting mds.f at 2024-06-06T04:26:50.680 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:26:50 smithi154 bash[66358]: b1c620c0d3beb514b3b12e72d77e1a4c0e5a272d134ac48cebb41c58a23ffedf 2024-06-06T04:26:50.681 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:26:50 smithi154 systemd[1]: Started Ceph mds.f for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:26:51.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:51 smithi086 ceph-mon[36053]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:51.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:51 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:51.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:51 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:51.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:51 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:51.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:51 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:51.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:51 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:51.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:51 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:51.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:51 smithi086 ceph-mon[36053]: Deploying daemon mds.c on smithi154 2024-06-06T04:26:51.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:51 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.154:6834/954391764,v1:172.21.15.154:6835/954391764] up:boot 2024-06-06T04:26:51.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:51 smithi086 ceph-mon[36053]: fsmap 6 up:standby 2024-06-06T04:26:51.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:51 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T04:26:51.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:51 smithi154 ceph-mon[36428]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:51.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:51 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:51.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:51 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:51.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:51 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:51.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:51 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:51.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:51 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:51.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:51 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:51.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:51 smithi154 ceph-mon[36428]: Deploying daemon mds.c on smithi154 2024-06-06T04:26:51.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:51 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.154:6834/954391764,v1:172.21.15.154:6835/954391764] up:boot 2024-06-06T04:26:51.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:51 smithi154 ceph-mon[36428]: fsmap 6 up:standby 2024-06-06T04:26:51.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:51 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T04:26:51.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:51 smithi045 ceph-mon[32299]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:51.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:51 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:51.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:51 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:51.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:51 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:51.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:51 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:51.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:51 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:51.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:51 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:51.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:51 smithi045 ceph-mon[32299]: Deploying daemon mds.c on smithi154 2024-06-06T04:26:51.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:51 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.154:6834/954391764,v1:172.21.15.154:6835/954391764] up:boot 2024-06-06T04:26:51.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:51 smithi045 ceph-mon[32299]: fsmap 6 up:standby 2024-06-06T04:26:51.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:51 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T04:26:52.495 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:26:52.760 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:26:52.930 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:26:52 smithi154 systemd[1]: Starting Ceph mds.c for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:26:53.193 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:26:52 smithi154 podman[66658]: 2024-06-06 04:26:52.836561284 +0000 UTC m=+0.017203078 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:26:53.193 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:26:52 smithi154 podman[66658]: 2024-06-06 04:26:52.956506012 +0000 UTC m=+0.137147790 container create 4bf74394285bd57e88cacb0b868a62e7b360003f2f29166afa8c178df689616e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-c, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-06T04:26:53.193 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:26:53 smithi154 podman[66658]: 2024-06-06 04:26:53.193239322 +0000 UTC m=+0.373881110 container init 4bf74394285bd57e88cacb0b868a62e7b360003f2f29166afa8c178df689616e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-06T04:26:53.546 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:26:53 smithi154 podman[66658]: 2024-06-06 04:26:53.196208383 +0000 UTC m=+0.376850168 container start 4bf74394285bd57e88cacb0b868a62e7b360003f2f29166afa8c178df689616e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-c, ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-06T04:26:53.546 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:26:53 smithi154 ceph-mds[66678]: -- 172.21.15.154:0/2691083369 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557f65267a40 con 0x557f6523a800 2024-06-06T04:26:53.546 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:26:53 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-c[66674]: starting mds.c at 2024-06-06T04:26:53.546 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:26:53 smithi154 bash[66658]: 4bf74394285bd57e88cacb0b868a62e7b360003f2f29166afa8c178df689616e 2024-06-06T04:26:53.546 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:26:53 smithi154 systemd[1]: Started Ceph mds.c for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:26:53.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:53 smithi086 ceph-mon[36053]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:53.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:53 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:53.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:53 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:53.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:53 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:53.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:53 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:53.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:53 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:53.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:53 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:53.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:53 smithi045 ceph-mon[32299]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:53.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:53.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:53.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:53.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:53.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:53.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:53 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:53.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:53 smithi154 ceph-mon[36428]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:53.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:53 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:53.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:53 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:53.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:53 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:53.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:53 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:53.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:53 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:53.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:53 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:54.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.484+0000 7f4f6be56700 1 -- 172.21.15.45:0/1449895767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f64102770 msgr2=0x7f4f64102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:54.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.484+0000 7f4f6be56700 1 --2- 172.21.15.45:0/1449895767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f64102770 0x7f4f64102b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4f600099c0 tx=0x7f4f60009cd0 comp rx=0 tx=0).stop 2024-06-06T04:26:54.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.484+0000 7f4f6be56700 1 -- 172.21.15.45:0/1449895767 shutdown_connections 2024-06-06T04:26:54.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.484+0000 7f4f6be56700 1 --2- 172.21.15.45:0/1449895767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f64107860 0x7f4f6410f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:54.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.484+0000 7f4f6be56700 1 --2- 172.21.15.45:0/1449895767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f64102770 0x7f4f64102b50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:54.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.484+0000 7f4f6be56700 1 --2- 172.21.15.45:0/1449895767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f64103120 0x7f4f64107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.484+0000 7f4f6be56700 1 -- 172.21.15.45:0/1449895767 >> 172.21.15.45:0/1449895767 conn(0x7f4f64079600 msgr2=0x7f4f64079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.484+0000 7f4f6be56700 1 -- 172.21.15.45:0/1449895767 shutdown_connections 2024-06-06T04:26:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.484+0000 7f4f6be56700 1 -- 172.21.15.45:0/1449895767 wait complete. 2024-06-06T04:26:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.484+0000 7f4f6be56700 1 Processor -- start 2024-06-06T04:26:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.484+0000 7f4f6be56700 1 -- start start 2024-06-06T04:26:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f6be56700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f64102770 0x7f4f64074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f6be56700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f64103120 0x7f4f64074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f6be56700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f64107860 0x7f4f64075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f6be56700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f64112630 con 0x7f4f64102770 2024-06-06T04:26:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f6be56700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641124b0 con 0x7f4f64103120 2024-06-06T04:26:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f6be56700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641127b0 con 0x7f4f64107860 2024-06-06T04:26:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f69bf2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f64102770 0x7f4f64074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f69bf2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f64102770 0x7f4f64074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36224/0 (socket says 172.21.15.45:36224) 2024-06-06T04:26:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f69bf2700 1 -- 172.21.15.45:0/3569201970 learned_addr learned my addr 172.21.15.45:0/3569201970 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:26:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f693f1700 1 --2- 172.21.15.45:0/3569201970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f64103120 0x7f4f64074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f6a3f3700 1 --2- 172.21.15.45:0/3569201970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f64107860 0x7f4f64075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f69bf2700 1 -- 172.21.15.45:0/3569201970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f64107860 msgr2=0x7f4f64075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f69bf2700 1 --2- 172.21.15.45:0/3569201970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f64107860 0x7f4f64075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f69bf2700 1 -- 172.21.15.45:0/3569201970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f64103120 msgr2=0x7f4f64074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f69bf2700 1 --2- 172.21.15.45:0/3569201970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f64103120 0x7f4f64074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f69bf2700 1 -- 172.21.15.45:0/3569201970 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f641a6f60 con 0x7f4f64102770 2024-06-06T04:26:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f69bf2700 1 --2- 172.21.15.45:0/3569201970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f64102770 0x7f4f64074790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4f60006010 tx=0x7f4f600055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f5affd700 1 -- 172.21.15.45:0/3569201970 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60026070 con 0x7f4f64102770 2024-06-06T04:26:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f5affd700 1 -- 172.21.15.45:0/3569201970 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f6000f460 con 0x7f4f64102770 2024-06-06T04:26:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f6be56700 1 -- 172.21.15.45:0/3569201970 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f641a71f0 con 0x7f4f64102770 2024-06-06T04:26:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f5affd700 1 -- 172.21.15.45:0/3569201970 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60009e60 con 0x7f4f64102770 2024-06-06T04:26:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.485+0000 7f4f6be56700 1 -- 172.21.15.45:0/3569201970 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f641a76a0 con 0x7f4f64102770 2024-06-06T04:26:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.486+0000 7f4f6be56700 1 -- 172.21.15.45:0/3569201970 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f6410aaa0 con 0x7f4f64102770 2024-06-06T04:26:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.488+0000 7f4f5affd700 1 -- 172.21.15.45:0/3569201970 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4f60029070 con 0x7f4f64102770 2024-06-06T04:26:54.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.488+0000 7f4f5affd700 1 --2- 172.21.15.45:0/3569201970 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4f500a2ca0 0x7f4f500a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:54.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.488+0000 7f4f693f1700 1 --2- 172.21.15.45:0/3569201970 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4f500a2ca0 0x7f4f500a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:54.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.488+0000 7f4f5affd700 1 -- 172.21.15.45:0/3569201970 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4f600cf6a0 con 0x7f4f64102770 2024-06-06T04:26:54.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.489+0000 7f4f693f1700 1 --2- 172.21.15.45:0/3569201970 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4f500a2ca0 0x7f4f500a5160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4f64102070 tx=0x7f4f5400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:54.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.492+0000 7f4f5affd700 1 -- 172.21.15.45:0/3569201970 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4f60030020 con 0x7f4f64102770 2024-06-06T04:26:54.547 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:54 smithi086 ceph-mon[36053]: Deploying daemon mds.h on smithi086 2024-06-06T04:26:54.547 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:54 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] up:boot 2024-06-06T04:26:54.547 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:54 smithi086 ceph-mon[36053]: fsmap 7 up:standby 2024-06-06T04:26:54.548 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:54 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T04:26:54.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.746+0000 7f4f6be56700 1 -- 172.21.15.45:0/3569201970 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f6410a480 con 0x7f4f64102770 2024-06-06T04:26:54.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:54 smithi045 ceph-mon[32299]: Deploying daemon mds.h on smithi086 2024-06-06T04:26:54.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:54 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] up:boot 2024-06-06T04:26:54.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:54 smithi045 ceph-mon[32299]: fsmap 7 up:standby 2024-06-06T04:26:54.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:54 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T04:26:54.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.747+0000 7f4f5affd700 1 -- 172.21.15.45:0/3569201970 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7f4f60013070 con 0x7f4f64102770 2024-06-06T04:26:54.749 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:26:54.749 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-06-06T04:26:54.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.749+0000 7f4f6be56700 1 -- 172.21.15.45:0/3569201970 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4f500a2ca0 msgr2=0x7f4f500a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.749+0000 7f4f6be56700 1 --2- 172.21.15.45:0/3569201970 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4f500a2ca0 0x7f4f500a5160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4f64102070 tx=0x7f4f5400a380 comp rx=0 tx=0).stop 2024-06-06T04:26:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.749+0000 7f4f6be56700 1 -- 172.21.15.45:0/3569201970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f64102770 msgr2=0x7f4f64074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.749+0000 7f4f6be56700 1 --2- 172.21.15.45:0/3569201970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f64102770 0x7f4f64074790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4f60006010 tx=0x7f4f600055f0 comp rx=0 tx=0).stop 2024-06-06T04:26:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.749+0000 7f4f6be56700 1 -- 172.21.15.45:0/3569201970 shutdown_connections 2024-06-06T04:26:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.749+0000 7f4f6be56700 1 --2- 172.21.15.45:0/3569201970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f64107860 0x7f4f64075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.749+0000 7f4f6be56700 1 --2- 172.21.15.45:0/3569201970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f64102770 0x7f4f64074790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.749+0000 7f4f6be56700 1 --2- 172.21.15.45:0/3569201970 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4f500a2ca0 0x7f4f500a5160 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.749+0000 7f4f6be56700 1 --2- 172.21.15.45:0/3569201970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f64103120 0x7f4f64074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.749+0000 7f4f6be56700 1 -- 172.21.15.45:0/3569201970 >> 172.21.15.45:0/3569201970 conn(0x7f4f64079600 msgr2=0x7f4f64100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.749+0000 7f4f6be56700 1 -- 172.21.15.45:0/3569201970 shutdown_connections 2024-06-06T04:26:54.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:54.749+0000 7f4f6be56700 1 -- 172.21.15.45:0/3569201970 wait complete. 2024-06-06T04:26:54.752 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-06-06T04:26:54.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:54 smithi154 ceph-mon[36428]: Deploying daemon mds.h on smithi086 2024-06-06T04:26:54.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:54 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] up:boot 2024-06-06T04:26:54.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:54 smithi154 ceph-mon[36428]: fsmap 7 up:standby 2024-06-06T04:26:54.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:54 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T04:26:55.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:55 smithi045 ceph-mon[32299]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:55.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:55 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3569201970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:26:55.819 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:26:55 smithi086 systemd[1]: Starting Ceph mds.h for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:26:55.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:55 smithi086 ceph-mon[36053]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:55.819 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:55 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3569201970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:26:55.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:55 smithi154 ceph-mon[36428]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:55.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:55 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3569201970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:26:56.109 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:26:55 smithi086 podman[65614]: 2024-06-06 04:26:55.721372786 +0000 UTC m=+0.016521211 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:26:56.109 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:26:55 smithi086 podman[65614]: 2024-06-06 04:26:55.856401771 +0000 UTC m=+0.151550194 container create 8c8c48e2f25525a2518f284780f63b1bea8e517bc5ea25815aee1d26fa941a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-h, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-06T04:26:56.609 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:26:56 smithi086 podman[65614]: 2024-06-06 04:26:56.137152787 +0000 UTC m=+0.432301211 container init 8c8c48e2f25525a2518f284780f63b1bea8e517bc5ea25815aee1d26fa941a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-h, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-06T04:26:56.609 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:26:56 smithi086 podman[65614]: 2024-06-06 04:26:56.1399439 +0000 UTC m=+0.435092325 container start 8c8c48e2f25525a2518f284780f63b1bea8e517bc5ea25815aee1d26fa941a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-h, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:26:56.609 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:26:56 smithi086 ceph-mds[65634]: -- 172.21.15.86:0/2588525066 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f842ecba40 con 0x55f842e9ec00 2024-06-06T04:26:56.609 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:26:56 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-h[65630]: starting mds.h at 2024-06-06T04:26:56.610 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:26:56 smithi086 bash[65614]: 8c8c48e2f25525a2518f284780f63b1bea8e517bc5ea25815aee1d26fa941a0f 2024-06-06T04:26:56.610 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:26:56 smithi086 systemd[1]: Started Ceph mds.h for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:26:57.398 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:26:57.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:57 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:57.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:57 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:57.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:57 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:57.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:57 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:57.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:57 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:57.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:57 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:57.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:57 smithi086 ceph-mon[36053]: Deploying daemon mds.i on smithi154 2024-06-06T04:26:57.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:57 smithi086 ceph-mon[36053]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:57.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:57 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.86:6836/414874753,v1:172.21.15.86:6837/414874753] up:boot 2024-06-06T04:26:57.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:57 smithi086 ceph-mon[36053]: fsmap 8 up:standby 2024-06-06T04:26:57.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:26:57 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T04:26:57.662 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:26:57.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:57 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:57.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:57 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:57.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:57 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:57.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:57 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:57.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:57 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:57.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:57 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:57.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:57 smithi154 ceph-mon[36428]: Deploying daemon mds.i on smithi154 2024-06-06T04:26:57.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:57 smithi154 ceph-mon[36428]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:57.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:57 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.86:6836/414874753,v1:172.21.15.86:6837/414874753] up:boot 2024-06-06T04:26:57.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:57 smithi154 ceph-mon[36428]: fsmap 8 up:standby 2024-06-06T04:26:57.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:26:57 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T04:26:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:57 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:57 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:57 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:26:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:57 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:26:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:57 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:26:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:57 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:26:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:57 smithi045 ceph-mon[32299]: Deploying daemon mds.i on smithi154 2024-06-06T04:26:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:57 smithi045 ceph-mon[32299]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:26:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:57 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.86:6836/414874753,v1:172.21.15.86:6837/414874753] up:boot 2024-06-06T04:26:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:57 smithi045 ceph-mon[32299]: fsmap 8 up:standby 2024-06-06T04:26:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:26:57 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T04:26:58.641 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:26:58 smithi154 systemd[1]: Starting Ceph mds.i for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:26:58.893 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:26:58 smithi154 podman[66959]: 2024-06-06 04:26:58.543484773 +0000 UTC m=+0.017112162 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:26:58.893 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:26:58 smithi154 podman[66959]: 2024-06-06 04:26:58.655843369 +0000 UTC m=+0.129470734 container create 456a88d250d145856dac083bceccd96cd029848a5321e57853bbf160cc9b55e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-06T04:26:58.893 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:26:58 smithi154 podman[66959]: 2024-06-06 04:26:58.867363609 +0000 UTC m=+0.340990983 container init 456a88d250d145856dac083bceccd96cd029848a5321e57853bbf160cc9b55e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True) 2024-06-06T04:26:58.894 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:26:58 smithi154 podman[66959]: 2024-06-06 04:26:58.870270354 +0000 UTC m=+0.343897740 container start 456a88d250d145856dac083bceccd96cd029848a5321e57853bbf160cc9b55e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-06T04:26:59.180 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:26:58 smithi154 ceph-mds[66978]: -- 172.21.15.154:0/3057525053 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5634be14ba40 con 0x5634be11e800 2024-06-06T04:26:59.180 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:26:58 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-i[66974]: starting mds.i at 2024-06-06T04:26:59.180 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:26:58 smithi154 bash[66959]: 456a88d250d145856dac083bceccd96cd029848a5321e57853bbf160cc9b55e5 2024-06-06T04:26:59.180 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:26:58 smithi154 systemd[1]: Started Ceph mds.i for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:26:59.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.372+0000 7f9d2406f700 1 -- 172.21.15.45:0/674839781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d1c107400 msgr2=0x7f9d1c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:59.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.372+0000 7f9d2406f700 1 --2- 172.21.15.45:0/674839781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d1c107400 0x7f9d1c1077e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009990 tx=0x7f9d18009ca0 comp rx=0 tx=0).stop 2024-06-06T04:26:59.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.372+0000 7f9d2406f700 1 -- 172.21.15.45:0/674839781 shutdown_connections 2024-06-06T04:26:59.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.372+0000 7f9d2406f700 1 --2- 172.21.15.45:0/674839781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c101d50 0x7f9d1c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:59.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.372+0000 7f9d2406f700 1 --2- 172.21.15.45:0/674839781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d1c107400 0x7f9d1c1077e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:59.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.372+0000 7f9d2406f700 1 --2- 172.21.15.45:0/674839781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d1c101390 0x7f9d1c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:59.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.372+0000 7f9d2406f700 1 -- 172.21.15.45:0/674839781 >> 172.21.15.45:0/674839781 conn(0x7f9d1c078340 msgr2=0x7f9d1c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:59.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.372+0000 7f9d2406f700 1 -- 172.21.15.45:0/674839781 shutdown_connections 2024-06-06T04:26:59.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.372+0000 7f9d2406f700 1 -- 172.21.15.45:0/674839781 wait complete. 2024-06-06T04:26:59.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.372+0000 7f9d2406f700 1 Processor -- start 2024-06-06T04:26:59.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.372+0000 7f9d2406f700 1 -- start start 2024-06-06T04:26:59.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d2406f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d1c101390 0x7f9d1c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:59.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d2406f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c101d50 0x7f9d1c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:59.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d2406f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d1c107400 0x7f9d1c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:59.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d2406f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c111520 con 0x7f9d1c101390 2024-06-06T04:26:59.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d2406f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1113a0 con 0x7f9d1c107400 2024-06-06T04:26:59.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d2406f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1116a0 con 0x7f9d1c101d50 2024-06-06T04:26:59.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d21e0b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d1c101390 0x7f9d1c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:59.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d21e0b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d1c101390 0x7f9d1c19e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41064/0 (socket says 172.21.15.45:41064) 2024-06-06T04:26:59.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d21e0b700 1 -- 172.21.15.45:0/1329074902 learned_addr learned my addr 172.21.15.45:0/1329074902 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:26:59.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d2260c700 1 --2- 172.21.15.45:0/1329074902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d1c107400 0x7f9d1c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:59.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d2160a700 1 --2- 172.21.15.45:0/1329074902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c101d50 0x7f9d1c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:59.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d21e0b700 1 -- 172.21.15.45:0/1329074902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c101d50 msgr2=0x7f9d1c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:59.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d21e0b700 1 --2- 172.21.15.45:0/1329074902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c101d50 0x7f9d1c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:59.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d21e0b700 1 -- 172.21.15.45:0/1329074902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d1c107400 msgr2=0x7f9d1c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:59.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d21e0b700 1 --2- 172.21.15.45:0/1329074902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d1c107400 0x7f9d1c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:59.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d21e0b700 1 -- 172.21.15.45:0/1329074902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c199110 con 0x7f9d1c101390 2024-06-06T04:26:59.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d2160a700 1 --2- 172.21.15.45:0/1329074902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c101d50 0x7f9d1c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:26:59.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d2260c700 1 --2- 172.21.15.45:0/1329074902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d1c107400 0x7f9d1c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:26:59.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d21e0b700 1 --2- 172.21.15.45:0/1329074902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d1c101390 0x7f9d1c19e7f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9d18000c00 tx=0x7f9d18003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:59.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.373+0000 7f9d12ffd700 1 -- 172.21.15.45:0/1329074902 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18027070 con 0x7f9d1c101390 2024-06-06T04:26:59.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.374+0000 7f9d12ffd700 1 -- 172.21.15.45:0/1329074902 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d1800bdc0 con 0x7f9d1c101390 2024-06-06T04:26:59.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.374+0000 7f9d2406f700 1 -- 172.21.15.45:0/1329074902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1c1993a0 con 0x7f9d1c101390 2024-06-06T04:26:59.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.374+0000 7f9d12ffd700 1 -- 172.21.15.45:0/1329074902 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1800f460 con 0x7f9d1c101390 2024-06-06T04:26:59.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.374+0000 7f9d2406f700 1 -- 172.21.15.45:0/1329074902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1c1a56c0 con 0x7f9d1c101390 2024-06-06T04:26:59.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.375+0000 7f9d2406f700 1 -- 172.21.15.45:0/1329074902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1c069180 con 0x7f9d1c101390 2024-06-06T04:26:59.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.376+0000 7f9d12ffd700 1 -- 172.21.15.45:0/1329074902 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9d1802a070 con 0x7f9d1c101390 2024-06-06T04:26:59.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.377+0000 7f9d12ffd700 1 --2- 172.21.15.45:0/1329074902 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9d080a2ca0 0x7f9d080a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:26:59.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.377+0000 7f9d2160a700 1 --2- 172.21.15.45:0/1329074902 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9d080a2ca0 0x7f9d080a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:26:59.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.377+0000 7f9d12ffd700 1 -- 172.21.15.45:0/1329074902 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9d180cfac0 con 0x7f9d1c101390 2024-06-06T04:26:59.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.378+0000 7f9d2160a700 1 --2- 172.21.15.45:0/1329074902 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9d080a2ca0 0x7f9d080a5160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c005950 tx=0x7f9d0c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:26:59.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.382+0000 7f9d12ffd700 1 -- 172.21.15.45:0/1329074902 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9d18013070 con 0x7f9d1c101390 2024-06-06T04:26:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.649+0000 7f9d2406f700 1 -- 172.21.15.45:0/1329074902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d1c1a5890 con 0x7f9d1c101390 2024-06-06T04:26:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.650+0000 7f9d12ffd700 1 -- 172.21.15.45:0/1329074902 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7995 (secure 0 0 0) 0x7f9d1809ceb0 con 0x7f9d1c101390 2024-06-06T04:26:59.650 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:26:59.650 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-06T04:26:59.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.652+0000 7f9d2406f700 1 -- 172.21.15.45:0/1329074902 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9d080a2ca0 msgr2=0x7f9d080a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.652+0000 7f9d2406f700 1 --2- 172.21.15.45:0/1329074902 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9d080a2ca0 0x7f9d080a5160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c005950 tx=0x7f9d0c0058e0 comp rx=0 tx=0).stop 2024-06-06T04:26:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.652+0000 7f9d2406f700 1 -- 172.21.15.45:0/1329074902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d1c101390 msgr2=0x7f9d1c19e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:26:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.652+0000 7f9d2406f700 1 --2- 172.21.15.45:0/1329074902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d1c101390 0x7f9d1c19e7f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9d18000c00 tx=0x7f9d18003b30 comp rx=0 tx=0).stop 2024-06-06T04:26:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.652+0000 7f9d2406f700 1 -- 172.21.15.45:0/1329074902 shutdown_connections 2024-06-06T04:26:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.652+0000 7f9d2406f700 1 --2- 172.21.15.45:0/1329074902 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9d080a2ca0 0x7f9d080a5160 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.652+0000 7f9d2406f700 1 --2- 172.21.15.45:0/1329074902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d1c107400 0x7f9d1c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.652+0000 7f9d2406f700 1 --2- 172.21.15.45:0/1329074902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d1c101390 0x7f9d1c19e7f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.652+0000 7f9d2406f700 1 --2- 172.21.15.45:0/1329074902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c101d50 0x7f9d1c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:26:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.652+0000 7f9d2406f700 1 -- 172.21.15.45:0/1329074902 >> 172.21.15.45:0/1329074902 conn(0x7f9d1c078340 msgr2=0x7f9d1c1057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:26:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.652+0000 7f9d2406f700 1 -- 172.21.15.45:0/1329074902 shutdown_connections 2024-06-06T04:26:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:26:59.652+0000 7f9d2406f700 1 -- 172.21.15.45:0/1329074902 wait complete. 2024-06-06T04:26:59.654 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 10 2024-06-06T04:27:00.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:00 smithi086 ceph-mon[36053]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:00.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:00.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:00.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:00.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:27:00.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:27:00.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:27:00.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:00 smithi086 ceph-mon[36053]: Deploying daemon mds.g on smithi045 2024-06-06T04:27:00.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:00 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] up:boot 2024-06-06T04:27:00.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:00 smithi086 ceph-mon[36053]: fsmap 9 up:standby 2024-06-06T04:27:00.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T04:27:00.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:00 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1329074902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:27:00.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:00 smithi154 ceph-mon[36428]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:00.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:00.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:00.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:27:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:27:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:27:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:00 smithi154 ceph-mon[36428]: Deploying daemon mds.g on smithi045 2024-06-06T04:27:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:00 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] up:boot 2024-06-06T04:27:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:00 smithi154 ceph-mon[36428]: fsmap 9 up:standby 2024-06-06T04:27:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T04:27:00.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:00 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1329074902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:27:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:00 smithi045 ceph-mon[32299]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:27:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:27:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:27:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:00 smithi045 ceph-mon[32299]: Deploying daemon mds.g on smithi045 2024-06-06T04:27:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:00 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] up:boot 2024-06-06T04:27:00.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:00 smithi045 ceph-mon[32299]: fsmap 9 up:standby 2024-06-06T04:27:00.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T04:27:00.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:00 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1329074902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:27:01.977 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jun 06 04:27:01 smithi045 systemd[1]: Starting Ceph mds.g for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:27:02.266 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:02 smithi045 ceph-mon[32299]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:02.266 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jun 06 04:27:02 smithi045 podman[74208]: 2024-06-06 04:27:01.905790157 +0000 UTC m=+0.017688441 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:27:02.266 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jun 06 04:27:02 smithi045 podman[74208]: 2024-06-06 04:27:02.030481758 +0000 UTC m=+0.142380039 container create 011a566e7a776b3c2fbb06e67717e55484c4bc52c4bd6c769409c6c842942104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-g, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T04:27:02.266 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jun 06 04:27:02 smithi045 podman[74208]: 2024-06-06 04:27:02.266656638 +0000 UTC m=+0.378554925 container init 011a566e7a776b3c2fbb06e67717e55484c4bc52c4bd6c769409c6c842942104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:27:02.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:02 smithi086 ceph-mon[36053]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:02.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:02 smithi154 ceph-mon[36428]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:02.726 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jun 06 04:27:02 smithi045 podman[74208]: 2024-06-06 04:27:02.269670277 +0000 UTC m=+0.381568566 container start 011a566e7a776b3c2fbb06e67717e55484c4bc52c4bd6c769409c6c842942104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-06T04:27:02.727 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jun 06 04:27:02 smithi045 ceph-mds[74227]: -- 172.21.15.45:0/2776400174 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c217e9fa40 con 0x55c217e72c00 2024-06-06T04:27:02.727 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jun 06 04:27:02 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-g[74223]: starting mds.g at 2024-06-06T04:27:02.727 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jun 06 04:27:02 smithi045 bash[74208]: 011a566e7a776b3c2fbb06e67717e55484c4bc52c4bd6c769409c6c842942104 2024-06-06T04:27:02.727 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jun 06 04:27:02 smithi045 systemd[1]: Started Ceph mds.g for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:27:02.934 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:27:03.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:03 smithi154 ceph-mon[36428]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:03.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:27:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:27:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:27:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:03 smithi154 ceph-mon[36428]: Deploying daemon mds.a on smithi045 2024-06-06T04:27:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:03 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.45:6838/2563890632,v1:172.21.15.45:6839/2563890632] up:boot 2024-06-06T04:27:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:03 smithi154 ceph-mon[36428]: fsmap 10 up:standby 2024-06-06T04:27:03.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T04:27:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:03 smithi045 ceph-mon[32299]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:27:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:27:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:27:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:03 smithi045 ceph-mon[32299]: Deploying daemon mds.a on smithi045 2024-06-06T04:27:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:03 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.45:6838/2563890632,v1:172.21.15.45:6839/2563890632] up:boot 2024-06-06T04:27:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:03 smithi045 ceph-mon[32299]: fsmap 10 up:standby 2024-06-06T04:27:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T04:27:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:03 smithi086 ceph-mon[36053]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:27:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:27:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:27:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:03 smithi086 ceph-mon[36053]: Deploying daemon mds.a on smithi045 2024-06-06T04:27:03.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:03 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.45:6838/2563890632,v1:172.21.15.45:6839/2563890632] up:boot 2024-06-06T04:27:03.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:03 smithi086 ceph-mon[36053]: fsmap 10 up:standby 2024-06-06T04:27:03.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T04:27:04.100 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:04.665 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jun 06 04:27:04 smithi045 systemd[1]: Starting Ceph mds.a for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:27:05.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:05 smithi154 ceph-mon[36428]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:05.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:05 smithi045 ceph-mon[32299]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:05.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:05 smithi086 ceph-mon[36053]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:05.998 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jun 06 04:27:05 smithi045 podman[74619]: 2024-06-06 04:27:05.641989994 +0000 UTC m=+0.123555694 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:27:05.998 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jun 06 04:27:05 smithi045 podman[74619]: 2024-06-06 04:27:05.881209007 +0000 UTC m=+0.362774686 container create 4c72eb5e276405d355a2069cf0b8998adb9441642b3ff87020fb3fb8a16dc573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-06T04:27:06.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.206+0000 7fe1c3ecb700 1 -- 172.21.15.45:0/542571486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc107400 msgr2=0x7fe1bc1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:06.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.206+0000 7fe1c3ecb700 1 --2- 172.21.15.45:0/542571486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc107400 0x7fe1bc1077e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009990 tx=0x7fe1b8009ca0 comp rx=0 tx=0).stop 2024-06-06T04:27:06.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.206+0000 7fe1c3ecb700 1 -- 172.21.15.45:0/542571486 shutdown_connections 2024-06-06T04:27:06.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.206+0000 7fe1c3ecb700 1 --2- 172.21.15.45:0/542571486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1bc101d50 0x7fe1bc10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.206+0000 7fe1c3ecb700 1 --2- 172.21.15.45:0/542571486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc107400 0x7fe1bc1077e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.206+0000 7fe1c3ecb700 1 --2- 172.21.15.45:0/542571486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc101390 0x7fe1bc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.206+0000 7fe1c3ecb700 1 -- 172.21.15.45:0/542571486 >> 172.21.15.45:0/542571486 conn(0x7fe1bc078340 msgr2=0x7fe1bc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.206+0000 7fe1c3ecb700 1 -- 172.21.15.45:0/542571486 shutdown_connections 2024-06-06T04:27:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.206+0000 7fe1c3ecb700 1 -- 172.21.15.45:0/542571486 wait complete. 2024-06-06T04:27:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.207+0000 7fe1c3ecb700 1 Processor -- start 2024-06-06T04:27:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.207+0000 7fe1c3ecb700 1 -- start start 2024-06-06T04:27:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.207+0000 7fe1c3ecb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc101390 0x7fe1bc19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.207+0000 7fe1c3ecb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc101d50 0x7fe1bc19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.207+0000 7fe1c3ecb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1bc107400 0x7fe1bc197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.207+0000 7fe1c3ecb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc111550 con 0x7fe1bc101390 2024-06-06T04:27:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.207+0000 7fe1c3ecb700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc1113d0 con 0x7fe1bc101d50 2024-06-06T04:27:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.207+0000 7fe1c3ecb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc1116d0 con 0x7fe1bc107400 2024-06-06T04:27:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.207+0000 7fe1c1c67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc101390 0x7fe1bc19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.207+0000 7fe1c1c67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc101390 0x7fe1bc19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41096/0 (socket says 172.21.15.45:41096) 2024-06-06T04:27:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.207+0000 7fe1c1c67700 1 -- 172.21.15.45:0/3586295063 learned_addr learned my addr 172.21.15.45:0/3586295063 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.207+0000 7fe1c2468700 1 --2- 172.21.15.45:0/3586295063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1bc107400 0x7fe1bc197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.208+0000 7fe1c1466700 1 --2- 172.21.15.45:0/3586295063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc101d50 0x7fe1bc19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.208+0000 7fe1c1c67700 1 -- 172.21.15.45:0/3586295063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1bc107400 msgr2=0x7fe1bc197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.208+0000 7fe1c1c67700 1 --2- 172.21.15.45:0/3586295063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1bc107400 0x7fe1bc197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.208+0000 7fe1c1c67700 1 -- 172.21.15.45:0/3586295063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc101d50 msgr2=0x7fe1bc19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.208+0000 7fe1c1c67700 1 --2- 172.21.15.45:0/3586295063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc101d50 0x7fe1bc19dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.208+0000 7fe1c1c67700 1 -- 172.21.15.45:0/3586295063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc197ff0 con 0x7fe1bc101390 2024-06-06T04:27:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.208+0000 7fe1c1c67700 1 --2- 172.21.15.45:0/3586295063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc101390 0x7fe1bc19d6d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8000c00 tx=0x7fe1b80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.208+0000 7fe1aeffd700 1 -- 172.21.15.45:0/3586295063 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8027070 con 0x7fe1bc101390 2024-06-06T04:27:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.208+0000 7fe1c3ecb700 1 -- 172.21.15.45:0/3586295063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1bc198280 con 0x7fe1bc101390 2024-06-06T04:27:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.208+0000 7fe1aeffd700 1 -- 172.21.15.45:0/3586295063 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1b8022470 con 0x7fe1bc101390 2024-06-06T04:27:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.208+0000 7fe1aeffd700 1 -- 172.21.15.45:0/3586295063 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8017650 con 0x7fe1bc101390 2024-06-06T04:27:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.208+0000 7fe1c3ecb700 1 -- 172.21.15.45:0/3586295063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1bc1a45a0 con 0x7fe1bc101390 2024-06-06T04:27:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.209+0000 7fe1c3ecb700 1 -- 172.21.15.45:0/3586295063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1bc069180 con 0x7fe1bc101390 2024-06-06T04:27:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.210+0000 7fe1aeffd700 1 -- 172.21.15.45:0/3586295063 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe1b802a070 con 0x7fe1bc101390 2024-06-06T04:27:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.211+0000 7fe1aeffd700 1 --2- 172.21.15.45:0/3586295063 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe1a80a2c20 0x7fe1a80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.211+0000 7fe1c1466700 1 --2- 172.21.15.45:0/3586295063 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe1a80a2c20 0x7fe1a80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:06.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.211+0000 7fe1aeffd700 1 -- 172.21.15.45:0/3586295063 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe1b80cf8c0 con 0x7fe1bc101390 2024-06-06T04:27:06.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.211+0000 7fe1c1466700 1 --2- 172.21.15.45:0/3586295063 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe1a80a2c20 0x7fe1a80a50e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc078a20 tx=0x7fe1b000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:06.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.216+0000 7fe1aeffd700 1 -- 172.21.15.45:0/3586295063 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe1b8013070 con 0x7fe1bc101390 2024-06-06T04:27:06.352 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jun 06 04:27:06 smithi045 podman[74619]: 2024-06-06 04:27:06.352403188 +0000 UTC m=+0.833968878 container init 4c72eb5e276405d355a2069cf0b8998adb9441642b3ff87020fb3fb8a16dc573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=) 2024-06-06T04:27:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.478+0000 7fe1c3ecb700 1 -- 172.21.15.45:0/3586295063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1bc198dc0 con 0x7fe1bc101390 2024-06-06T04:27:06.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.479+0000 7fe1aeffd700 1 -- 172.21.15.45:0/3586295063 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9644 (secure 0 0 0) 0x7fe1b809cd50 con 0x7fe1bc101390 2024-06-06T04:27:06.480 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:27:06.480 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-06T04:27:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.482+0000 7fe1c3ecb700 1 -- 172.21.15.45:0/3586295063 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe1a80a2c20 msgr2=0x7fe1a80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.482+0000 7fe1c3ecb700 1 --2- 172.21.15.45:0/3586295063 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe1a80a2c20 0x7fe1a80a50e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc078a20 tx=0x7fe1b000b410 comp rx=0 tx=0).stop 2024-06-06T04:27:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.482+0000 7fe1c3ecb700 1 -- 172.21.15.45:0/3586295063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc101390 msgr2=0x7fe1bc19d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.482+0000 7fe1c3ecb700 1 --2- 172.21.15.45:0/3586295063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc101390 0x7fe1bc19d6d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8000c00 tx=0x7fe1b80047b0 comp rx=0 tx=0).stop 2024-06-06T04:27:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.482+0000 7fe1c3ecb700 1 -- 172.21.15.45:0/3586295063 shutdown_connections 2024-06-06T04:27:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.482+0000 7fe1c3ecb700 1 --2- 172.21.15.45:0/3586295063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1bc107400 0x7fe1bc197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.482+0000 7fe1c3ecb700 1 --2- 172.21.15.45:0/3586295063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1bc101390 0x7fe1bc19d6d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.482+0000 7fe1c3ecb700 1 --2- 172.21.15.45:0/3586295063 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe1a80a2c20 0x7fe1a80a50e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.482+0000 7fe1c3ecb700 1 --2- 172.21.15.45:0/3586295063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1bc101d50 0x7fe1bc19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.482+0000 7fe1c3ecb700 1 -- 172.21.15.45:0/3586295063 >> 172.21.15.45:0/3586295063 conn(0x7fe1bc078340 msgr2=0x7fe1bc1001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.482+0000 7fe1c3ecb700 1 -- 172.21.15.45:0/3586295063 shutdown_connections 2024-06-06T04:27:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:06.482+0000 7fe1c3ecb700 1 -- 172.21.15.45:0/3586295063 wait complete. 2024-06-06T04:27:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 12 2024-06-06T04:27:06.726 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jun 06 04:27:06 smithi045 podman[74619]: 2024-06-06 04:27:06.356751502 +0000 UTC m=+0.838317200 container start 4c72eb5e276405d355a2069cf0b8998adb9441642b3ff87020fb3fb8a16dc573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-06T04:27:06.727 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jun 06 04:27:06 smithi045 ceph-mds[74669]: -- 172.21.15.45:0/575297767 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5577f4fb9a40 con 0x5577f4f8cc00 2024-06-06T04:27:06.727 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jun 06 04:27:06 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-a[74643]: starting mds.a at 2024-06-06T04:27:06.727 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jun 06 04:27:06 smithi045 bash[74619]: 4c72eb5e276405d355a2069cf0b8998adb9441642b3ff87020fb3fb8a16dc573 2024-06-06T04:27:06.727 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jun 06 04:27:06 smithi045 systemd[1]: Started Ceph mds.a for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:27:07.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:07 smithi086 ceph-mon[36053]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:07.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:07 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.45:6840/611193504,v1:172.21.15.45:6841/611193504] up:boot 2024-06-06T04:27:07.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:07 smithi086 ceph-mon[36053]: fsmap 11 up:standby 2024-06-06T04:27:07.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:07 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T04:27:07.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:07 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3586295063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:27:07.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:07 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:07.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:07 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:07.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:07 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:07.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:07 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:27:07.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:07 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:27:07.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:07 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:27:07.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:07 smithi086 ceph-mon[36053]: Deploying daemon mds.b on smithi086 2024-06-06T04:27:07.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:07 smithi154 ceph-mon[36428]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:07.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:07 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.45:6840/611193504,v1:172.21.15.45:6841/611193504] up:boot 2024-06-06T04:27:07.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:07 smithi154 ceph-mon[36428]: fsmap 11 up:standby 2024-06-06T04:27:07.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:07 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T04:27:07.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:07 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3586295063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:27:07.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:07 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:07.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:07 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:07.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:07 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:07.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:07 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:27:07.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:07 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:27:07.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:07 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:27:07.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:07 smithi154 ceph-mon[36428]: Deploying daemon mds.b on smithi086 2024-06-06T04:27:07.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:07 smithi045 ceph-mon[32299]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:07.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:07 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.45:6840/611193504,v1:172.21.15.45:6841/611193504] up:boot 2024-06-06T04:27:07.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:07 smithi045 ceph-mon[32299]: fsmap 11 up:standby 2024-06-06T04:27:07.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T04:27:07.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:07 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3586295063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:27:07.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:07.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:07.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:07.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T04:27:07.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T04:27:07.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:07 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:27:07.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:07 smithi045 ceph-mon[32299]: Deploying daemon mds.b on smithi086 2024-06-06T04:27:09.029 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:27:08 smithi086 systemd[1]: Starting Ceph mds.b for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:27:09.114 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:27:09.297 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:27:09 smithi086 podman[65920]: 2024-06-06 04:27:08.93220757 +0000 UTC m=+0.017684591 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:27:09.297 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:27:09 smithi086 podman[65920]: 2024-06-06 04:27:09.0450255 +0000 UTC m=+0.130502553 container create fca99af9487b5b8fe96ba03cdde64797259c739ff737dcd15b1ca140d5c30175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-06T04:27:09.380 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:09.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:09 smithi086 ceph-mon[36053]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:09.609 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:27:09 smithi086 podman[65920]: 2024-06-06 04:27:09.297489261 +0000 UTC m=+0.382966272 container init fca99af9487b5b8fe96ba03cdde64797259c739ff737dcd15b1ca140d5c30175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-06-06T04:27:09.609 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:27:09 smithi086 podman[65920]: 2024-06-06 04:27:09.300593871 +0000 UTC m=+0.386070888 container start fca99af9487b5b8fe96ba03cdde64797259c739ff737dcd15b1ca140d5c30175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-06T04:27:09.609 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:27:09 smithi086 ceph-mds[65939]: -- 172.21.15.86:0/1245783279 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562025dfba40 con 0x562025dcf000 2024-06-06T04:27:09.609 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:27:09 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-b[65935]: starting mds.b at 2024-06-06T04:27:09.609 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:27:09 smithi086 bash[65920]: fca99af9487b5b8fe96ba03cdde64797259c739ff737dcd15b1ca140d5c30175 2024-06-06T04:27:09.609 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:27:09 smithi086 systemd[1]: Started Ceph mds.b for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:27:09.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:09 smithi045 ceph-mon[32299]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:09.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:09 smithi154 ceph-mon[36428]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:10.549 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:10 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] up:boot 2024-06-06T04:27:10.549 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:10 smithi045 ceph-mon[32299]: fsmap 12 up:standby 2024-06-06T04:27:10.549 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T04:27:10.549 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:10.550 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:10.550 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:10.550 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:10.550 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:10.550 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:10 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:27:10.704 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:10 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] up:boot 2024-06-06T04:27:10.704 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:10 smithi154 ceph-mon[36428]: fsmap 12 up:standby 2024-06-06T04:27:10.704 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T04:27:10.704 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:10.704 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:10.704 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:10.704 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:10.705 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:10.705 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:10 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:27:10.771 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:10 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] up:boot 2024-06-06T04:27:10.772 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:10 smithi086 ceph-mon[36053]: fsmap 12 up:standby 2024-06-06T04:27:10.772 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T04:27:10.772 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:10.772 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:10.772 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:10.772 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:10.772 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:10.772 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:10 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:27:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.092+0000 7f8fd5f81700 1 -- 172.21.15.45:0/984660522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0107840 msgr2=0x7f8fd010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.092+0000 7f8fd5f81700 1 --2- 172.21.15.45:0/984660522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0107840 0x7f8fd010f3c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800b0d0 tx=0x7f8fc800b3e0 comp rx=0 tx=0).stop 2024-06-06T04:27:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.092+0000 7f8fd5f81700 1 -- 172.21.15.45:0/984660522 shutdown_connections 2024-06-06T04:27:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.092+0000 7f8fd5f81700 1 --2- 172.21.15.45:0/984660522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd0107840 0x7f8fd010f3c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.092+0000 7f8fd5f81700 1 --2- 172.21.15.45:0/984660522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fd0102740 0x7f8fd0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.092+0000 7f8fd5f81700 1 --2- 172.21.15.45:0/984660522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd01030f0 0x7f8fd0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.092+0000 7f8fd5f81700 1 -- 172.21.15.45:0/984660522 >> 172.21.15.45:0/984660522 conn(0x7f8fd00795f0 msgr2=0x7f8fd0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.092+0000 7f8fd5f81700 1 -- 172.21.15.45:0/984660522 shutdown_connections 2024-06-06T04:27:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.092+0000 7f8fd5f81700 1 -- 172.21.15.45:0/984660522 wait complete. 2024-06-06T04:27:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.093+0000 7f8fd5f81700 1 Processor -- start 2024-06-06T04:27:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.093+0000 7f8fd5f81700 1 -- start start 2024-06-06T04:27:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.093+0000 7f8fd5f81700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd0102740 0x7f8fd019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.093+0000 7f8fd5f81700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd01030f0 0x7f8fd019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.093+0000 7f8fd5f81700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fd0107840 0x7f8fd01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.093+0000 7f8fd5f81700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0112630 con 0x7f8fd01030f0 2024-06-06T04:27:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.093+0000 7f8fd5f81700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01124b0 con 0x7f8fd0107840 2024-06-06T04:27:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.093+0000 7f8fd5f81700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01127b0 con 0x7f8fd0102740 2024-06-06T04:27:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.093+0000 7f8fceffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd01030f0 0x7f8fd019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.093+0000 7f8fceffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd01030f0 0x7f8fd019ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34454/0 (socket says 172.21.15.45:34454) 2024-06-06T04:27:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.093+0000 7f8fceffd700 1 -- 172.21.15.45:0/430598382 learned_addr learned my addr 172.21.15.45:0/430598382 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.093+0000 7f8fcf7fe700 1 --2- 172.21.15.45:0/430598382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd0102740 0x7f8fd019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.093+0000 7f8fcffff700 1 --2- 172.21.15.45:0/430598382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fd0107840 0x7f8fd01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.094+0000 7f8fceffd700 1 -- 172.21.15.45:0/430598382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd0102740 msgr2=0x7f8fd019e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.094+0000 7f8fceffd700 1 --2- 172.21.15.45:0/430598382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd0102740 0x7f8fd019e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.094+0000 7f8fceffd700 1 -- 172.21.15.45:0/430598382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fd0107840 msgr2=0x7f8fd01988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.094+0000 7f8fceffd700 1 --2- 172.21.15.45:0/430598382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fd0107840 0x7f8fd01988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.094+0000 7f8fceffd700 1 -- 172.21.15.45:0/430598382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd0199050 con 0x7f8fd01030f0 2024-06-06T04:27:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.094+0000 7f8fceffd700 1 --2- 172.21.15.45:0/430598382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd01030f0 0x7f8fd019ec70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400d980 tx=0x7f8fc400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.094+0000 7f8fccff9700 1 -- 172.21.15.45:0/430598382 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4017070 con 0x7f8fd01030f0 2024-06-06T04:27:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.094+0000 7f8fd5f81700 1 -- 172.21.15.45:0/430598382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd0199340 con 0x7f8fd01030f0 2024-06-06T04:27:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.094+0000 7f8fccff9700 1 -- 172.21.15.45:0/430598382 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fc4005020 con 0x7f8fd01030f0 2024-06-06T04:27:11.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.094+0000 7f8fccff9700 1 -- 172.21.15.45:0/430598382 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc400fa90 con 0x7f8fd01030f0 2024-06-06T04:27:11.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.094+0000 7f8fd5f81700 1 -- 172.21.15.45:0/430598382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd01a5600 con 0x7f8fd01030f0 2024-06-06T04:27:11.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.095+0000 7f8fd5f81700 1 -- 172.21.15.45:0/430598382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd0069180 con 0x7f8fd01030f0 2024-06-06T04:27:11.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.096+0000 7f8fccff9700 1 -- 172.21.15.45:0/430598382 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8fc40051d0 con 0x7f8fd01030f0 2024-06-06T04:27:11.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.097+0000 7f8fccff9700 1 --2- 172.21.15.45:0/430598382 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8fb80a2c20 0x7f8fb80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:11.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.098+0000 7f8fcf7fe700 1 --2- 172.21.15.45:0/430598382 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8fb80a2c20 0x7f8fb80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:11.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.098+0000 7f8fccff9700 1 -- 172.21.15.45:0/430598382 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8fc40ce980 con 0x7f8fd01030f0 2024-06-06T04:27:11.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.098+0000 7f8fcf7fe700 1 --2- 172.21.15.45:0/430598382 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8fb80a2c20 0x7f8fb80a50e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0199a30 tx=0x7f8fc000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:11.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.103+0000 7f8fccff9700 1 -- 172.21.15.45:0/430598382 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8fc4020020 con 0x7f8fd01030f0 2024-06-06T04:27:11.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.366+0000 7f8fd5f81700 1 -- 172.21.15.45:0/430598382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fd0199e20 con 0x7f8fd01030f0 2024-06-06T04:27:11.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.366+0000 7f8fccff9700 1 -- 172.21.15.45:0/430598382 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10467 (secure 0 0 0) 0x7f8fc409bd80 con 0x7f8fd01030f0 2024-06-06T04:27:11.367 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:27:11.367 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-06-06T04:27:11.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.368+0000 7f8fd5f81700 1 -- 172.21.15.45:0/430598382 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8fb80a2c20 msgr2=0x7f8fb80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:11.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.368+0000 7f8fd5f81700 1 --2- 172.21.15.45:0/430598382 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8fb80a2c20 0x7f8fb80a50e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0199a30 tx=0x7f8fc000b410 comp rx=0 tx=0).stop 2024-06-06T04:27:11.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.368+0000 7f8fd5f81700 1 -- 172.21.15.45:0/430598382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd01030f0 msgr2=0x7f8fd019ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:11.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.368+0000 7f8fd5f81700 1 --2- 172.21.15.45:0/430598382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd01030f0 0x7f8fd019ec70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400d980 tx=0x7f8fc400dc90 comp rx=0 tx=0).stop 2024-06-06T04:27:11.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.368+0000 7f8fd5f81700 1 -- 172.21.15.45:0/430598382 shutdown_connections 2024-06-06T04:27:11.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.368+0000 7f8fd5f81700 1 --2- 172.21.15.45:0/430598382 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8fb80a2c20 0x7f8fb80a50e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:11.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.368+0000 7f8fd5f81700 1 --2- 172.21.15.45:0/430598382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fd0107840 0x7f8fd01988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:11.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.368+0000 7f8fd5f81700 1 --2- 172.21.15.45:0/430598382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd0102740 0x7f8fd019e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:11.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.368+0000 7f8fd5f81700 1 --2- 172.21.15.45:0/430598382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd01030f0 0x7f8fd019ec70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:11.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.368+0000 7f8fd5f81700 1 -- 172.21.15.45:0/430598382 >> 172.21.15.45:0/430598382 conn(0x7f8fd00795f0 msgr2=0x7f8fd0100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:11.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.369+0000 7f8fd5f81700 1 -- 172.21.15.45:0/430598382 shutdown_connections 2024-06-06T04:27:11.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:11.369+0000 7f8fd5f81700 1 -- 172.21.15.45:0/430598382 wait complete. 2024-06-06T04:27:11.371 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 13 2024-06-06T04:27:11.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:11 smithi045 ceph-mon[32299]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:11.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:11.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:11.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:11 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/430598382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:27:11.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:11.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:11 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:11 smithi086 ceph-mon[36053]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:11 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/430598382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:27:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:11 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:11.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:11 smithi154 ceph-mon[36428]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:11.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:11.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:11.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:11 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/430598382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:27:11.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:11.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:11 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:12.234 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-06-06T04:27:12.605 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:13.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:13 smithi045 ceph-mon[32299]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:13.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:13 smithi086 ceph-mon[36053]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:13.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:13 smithi154 ceph-mon[36428]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.188+0000 7f7bb5c24700 1 -- 172.21.15.45:0/37676928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bb0102740 msgr2=0x7f7bb0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.188+0000 7f7bb5c24700 1 --2- 172.21.15.45:0/37676928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bb0102740 0x7f7bb0102b20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00099c0 tx=0x7f7ba0009cd0 comp rx=0 tx=0).stop 2024-06-06T04:27:14.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.188+0000 7f7bb5c24700 1 -- 172.21.15.45:0/37676928 shutdown_connections 2024-06-06T04:27:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.188+0000 7f7bb5c24700 1 --2- 172.21.15.45:0/37676928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb0107840 0x7f7bb010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.188+0000 7f7bb5c24700 1 --2- 172.21.15.45:0/37676928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bb0102740 0x7f7bb0102b20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.188+0000 7f7bb5c24700 1 --2- 172.21.15.45:0/37676928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bb01030f0 0x7f7bb0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.188+0000 7f7bb5c24700 1 -- 172.21.15.45:0/37676928 >> 172.21.15.45:0/37676928 conn(0x7f7bb00795f0 msgr2=0x7f7bb0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.188+0000 7f7bb5c24700 1 -- 172.21.15.45:0/37676928 shutdown_connections 2024-06-06T04:27:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.188+0000 7f7bb5c24700 1 -- 172.21.15.45:0/37676928 wait complete. 2024-06-06T04:27:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.188+0000 7f7bb5c24700 1 Processor -- start 2024-06-06T04:27:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7bb5c24700 1 -- start start 2024-06-06T04:27:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7bb5c24700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bb0102740 0x7f7bb0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7bb5c24700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb01030f0 0x7f7bb0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7bb5c24700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bb0107840 0x7f7bb0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7bb5c24700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb0112610 con 0x7f7bb0107840 2024-06-06T04:27:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7bb5c24700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb0112490 con 0x7f7bb01030f0 2024-06-06T04:27:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7bb5c24700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb0112790 con 0x7f7bb0102740 2024-06-06T04:27:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7baffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bb0107840 0x7f7bb0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7baffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bb0107840 0x7f7bb0075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34460/0 (socket says 172.21.15.45:34460) 2024-06-06T04:27:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7baffff700 1 -- 172.21.15.45:0/1117359737 learned_addr learned my addr 172.21.15.45:0/1117359737 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7baf7fe700 1 --2- 172.21.15.45:0/1117359737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bb0102740 0x7f7bb0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7baeffd700 1 --2- 172.21.15.45:0/1117359737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb01030f0 0x7f7bb0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7baffff700 1 -- 172.21.15.45:0/1117359737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bb0102740 msgr2=0x7f7bb0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7baffff700 1 --2- 172.21.15.45:0/1117359737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bb0102740 0x7f7bb0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7baffff700 1 -- 172.21.15.45:0/1117359737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb01030f0 msgr2=0x7f7bb0074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7baffff700 1 --2- 172.21.15.45:0/1117359737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb01030f0 0x7f7bb0074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7baffff700 1 -- 172.21.15.45:0/1117359737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb01a7040 con 0x7f7bb0107840 2024-06-06T04:27:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7baf7fe700 1 --2- 172.21.15.45:0/1117359737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bb0102740 0x7f7bb0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:27:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.189+0000 7f7baeffd700 1 --2- 172.21.15.45:0/1117359737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb01030f0 0x7f7bb0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:27:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.190+0000 7f7baffff700 1 --2- 172.21.15.45:0/1117359737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bb0107840 0x7f7bb0075210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8007d70 tx=0x7f7ba800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.190+0000 7f7bacff9700 1 -- 172.21.15.45:0/1117359737 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8016020 con 0x7f7bb0107840 2024-06-06T04:27:14.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.190+0000 7f7bacff9700 1 -- 172.21.15.45:0/1117359737 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ba800ad40 con 0x7f7bb0107840 2024-06-06T04:27:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.190+0000 7f7bacff9700 1 -- 172.21.15.45:0/1117359737 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8011620 con 0x7f7bb0107840 2024-06-06T04:27:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.190+0000 7f7bb5c24700 1 -- 172.21.15.45:0/1117359737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb01a7330 con 0x7f7bb0107840 2024-06-06T04:27:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.190+0000 7f7bb5c24700 1 -- 172.21.15.45:0/1117359737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb01a7820 con 0x7f7bb0107840 2024-06-06T04:27:14.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.191+0000 7f7bb5c24700 1 -- 172.21.15.45:0/1117359737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb010aa80 con 0x7f7bb0107840 2024-06-06T04:27:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.192+0000 7f7bacff9700 1 -- 172.21.15.45:0/1117359737 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7ba8019020 con 0x7f7bb0107840 2024-06-06T04:27:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.193+0000 7f7bacff9700 1 --2- 172.21.15.45:0/1117359737 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7b980a2ca0 0x7f7b980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.193+0000 7f7baf7fe700 1 --2- 172.21.15.45:0/1117359737 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7b980a2ca0 0x7f7b980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.193+0000 7f7bacff9700 1 -- 172.21.15.45:0/1117359737 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7ba80ceb00 con 0x7f7bb0107840 2024-06-06T04:27:14.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.193+0000 7f7baf7fe700 1 --2- 172.21.15.45:0/1117359737 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7b980a2ca0 0x7f7b980a5160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0005b40 tx=0x7f7ba0005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:14.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.197+0000 7f7bacff9700 1 -- 172.21.15.45:0/1117359737 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7ba8020020 con 0x7f7bb0107840 2024-06-06T04:27:14.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.465+0000 7f7bb5c24700 1 -- 172.21.15.45:0/1117359737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7bb010a460 con 0x7f7bb0107840 2024-06-06T04:27:14.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.466+0000 7f7bacff9700 1 -- 172.21.15.45:0/1117359737 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f7ba809bf00 con 0x7f7bb0107840 2024-06-06T04:27:14.466 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:27:14.467 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-06-06T04:27:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:27:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-06-06T04:27:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-06-06T04:27:14.467 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:27:14.467 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-06-06T04:27:14.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.468+0000 7f7bb5c24700 1 -- 172.21.15.45:0/1117359737 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7b980a2ca0 msgr2=0x7f7b980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:14.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.468+0000 7f7bb5c24700 1 --2- 172.21.15.45:0/1117359737 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7b980a2ca0 0x7f7b980a5160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0005b40 tx=0x7f7ba0005ab0 comp rx=0 tx=0).stop 2024-06-06T04:27:14.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.468+0000 7f7bb5c24700 1 -- 172.21.15.45:0/1117359737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bb0107840 msgr2=0x7f7bb0075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:14.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.468+0000 7f7bb5c24700 1 --2- 172.21.15.45:0/1117359737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bb0107840 0x7f7bb0075210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8007d70 tx=0x7f7ba800a340 comp rx=0 tx=0).stop 2024-06-06T04:27:14.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.469+0000 7f7bb5c24700 1 -- 172.21.15.45:0/1117359737 shutdown_connections 2024-06-06T04:27:14.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.469+0000 7f7bb5c24700 1 --2- 172.21.15.45:0/1117359737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bb0107840 0x7f7bb0075210 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:14.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.469+0000 7f7bb5c24700 1 --2- 172.21.15.45:0/1117359737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bb0102740 0x7f7bb0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:14.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.469+0000 7f7bb5c24700 1 --2- 172.21.15.45:0/1117359737 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7b980a2ca0 0x7f7b980a5160 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:14.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.469+0000 7f7bb5c24700 1 --2- 172.21.15.45:0/1117359737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb01030f0 0x7f7bb0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:14.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.469+0000 7f7bb5c24700 1 -- 172.21.15.45:0/1117359737 >> 172.21.15.45:0/1117359737 conn(0x7f7bb00795f0 msgr2=0x7f7bb0108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:14.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.469+0000 7f7bb5c24700 1 -- 172.21.15.45:0/1117359737 shutdown_connections 2024-06-06T04:27:14.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:14.469+0000 7f7bb5c24700 1 -- 172.21.15.45:0/1117359737 wait complete. 2024-06-06T04:27:15.149 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-06-06T04:27:15.149 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-06-06T04:27:15.432 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:15.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:15 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:15.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:15 smithi045 ceph-mon[32299]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:15.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:15 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1117359737' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-06T04:27:15.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:15 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:15.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:15 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:15.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:15 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:15.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:15 smithi086 ceph-mon[36053]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:15.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:15 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1117359737' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-06T04:27:15.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:15 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:15.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:15 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:15.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:15 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:15.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:15 smithi154 ceph-mon[36428]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:15.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:15 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1117359737' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-06T04:27:15.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:15 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:15.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:15 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:16.496 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:16 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:27:16.496 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:16 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:27:16.496 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:16 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:16.496 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:16 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:27:16.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:16 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:27:16.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:16 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:27:16.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:16 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:16.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:16 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:27:16.702 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:16 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:27:16.703 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:16 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:27:16.703 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:16 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:16.703 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:16 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:27:17.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.093+0000 7ffb36de1700 1 -- 172.21.15.45:0/2369796263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb30101610 msgr2=0x7ffb301019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:17.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.093+0000 7ffb36de1700 1 --2- 172.21.15.45:0/2369796263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb30101610 0x7ffb301019f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffb200099c0 tx=0x7ffb20009cd0 comp rx=0 tx=0).stop 2024-06-06T04:27:17.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.093+0000 7ffb36de1700 1 -- 172.21.15.45:0/2369796263 shutdown_connections 2024-06-06T04:27:17.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.093+0000 7ffb36de1700 1 --2- 172.21.15.45:0/2369796263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb301066f0 0x7ffb3010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:17.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.093+0000 7ffb36de1700 1 --2- 172.21.15.45:0/2369796263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb30101610 0x7ffb301019f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:17.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.093+0000 7ffb36de1700 1 --2- 172.21.15.45:0/2369796263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb30101fc0 0x7ffb30106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:17.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.093+0000 7ffb36de1700 1 -- 172.21.15.45:0/2369796263 >> 172.21.15.45:0/2369796263 conn(0x7ffb30078430 msgr2=0x7ffb30078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:17.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.093+0000 7ffb36de1700 1 -- 172.21.15.45:0/2369796263 shutdown_connections 2024-06-06T04:27:17.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.093+0000 7ffb36de1700 1 -- 172.21.15.45:0/2369796263 wait complete. 2024-06-06T04:27:17.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.094+0000 7ffb36de1700 1 Processor -- start 2024-06-06T04:27:17.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.094+0000 7ffb36de1700 1 -- start start 2024-06-06T04:27:17.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.094+0000 7ffb36de1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb30101610 0x7ffb30199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:17.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.094+0000 7ffb36de1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb30101fc0 0x7ffb3019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:17.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.094+0000 7ffb36de1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb301066f0 0x7ffb301a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:17.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.094+0000 7ffb36de1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffb301115d0 con 0x7ffb30101610 2024-06-06T04:27:17.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.094+0000 7ffb36de1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb30111450 con 0x7ffb301066f0 2024-06-06T04:27:17.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.094+0000 7ffb36de1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffb30111750 con 0x7ffb30101fc0 2024-06-06T04:27:17.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.094+0000 7ffb34b7d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb30101610 0x7ffb30199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:17.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.094+0000 7ffb34b7d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb30101610 0x7ffb30199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34482/0 (socket says 172.21.15.45:34482) 2024-06-06T04:27:17.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.094+0000 7ffb34b7d700 1 -- 172.21.15.45:0/2639069000 learned_addr learned my addr 172.21.15.45:0/2639069000 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:17.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.094+0000 7ffb2ffff700 1 --2- 172.21.15.45:0/2639069000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb30101fc0 0x7ffb3019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:17.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.094+0000 7ffb3537e700 1 --2- 172.21.15.45:0/2639069000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb301066f0 0x7ffb301a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:17.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.095+0000 7ffb34b7d700 1 -- 172.21.15.45:0/2639069000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb30101fc0 msgr2=0x7ffb3019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:17.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.095+0000 7ffb34b7d700 1 --2- 172.21.15.45:0/2639069000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb30101fc0 0x7ffb3019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:17.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.095+0000 7ffb34b7d700 1 -- 172.21.15.45:0/2639069000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb301066f0 msgr2=0x7ffb301a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:17.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.095+0000 7ffb34b7d700 1 --2- 172.21.15.45:0/2639069000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb301066f0 0x7ffb301a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:17.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.095+0000 7ffb34b7d700 1 -- 172.21.15.45:0/2639069000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb301a5090 con 0x7ffb30101610 2024-06-06T04:27:17.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.095+0000 7ffb34b7d700 1 --2- 172.21.15.45:0/2639069000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb30101610 0x7ffb30199ad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffb20006010 tx=0x7ffb200045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:17.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.095+0000 7ffb2dffb700 1 -- 172.21.15.45:0/2639069000 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20026070 con 0x7ffb30101610 2024-06-06T04:27:17.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.095+0000 7ffb36de1700 1 -- 172.21.15.45:0/2639069000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb301a5260 con 0x7ffb30101610 2024-06-06T04:27:17.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.095+0000 7ffb2dffb700 1 -- 172.21.15.45:0/2639069000 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb2000f460 con 0x7ffb30101610 2024-06-06T04:27:17.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.095+0000 7ffb2dffb700 1 -- 172.21.15.45:0/2639069000 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20009e60 con 0x7ffb30101610 2024-06-06T04:27:17.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.095+0000 7ffb36de1700 1 -- 172.21.15.45:0/2639069000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb301a56b0 con 0x7ffb30101610 2024-06-06T04:27:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.096+0000 7ffb36de1700 1 -- 172.21.15.45:0/2639069000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb30069180 con 0x7ffb30101610 2024-06-06T04:27:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.097+0000 7ffb2dffb700 1 -- 172.21.15.45:0/2639069000 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffb20029070 con 0x7ffb30101610 2024-06-06T04:27:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.098+0000 7ffb2dffb700 1 --2- 172.21.15.45:0/2639069000 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ffb180a2c20 0x7ffb180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.098+0000 7ffb2ffff700 1 --2- 172.21.15.45:0/2639069000 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ffb180a2c20 0x7ffb180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.098+0000 7ffb2dffb700 1 -- 172.21.15.45:0/2639069000 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ffb200cf790 con 0x7ffb30101610 2024-06-06T04:27:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.099+0000 7ffb2ffff700 1 --2- 172.21.15.45:0/2639069000 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ffb180a2c20 0x7ffb180a50e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffb30101040 tx=0x7ffb2400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:17.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.102+0000 7ffb2dffb700 1 -- 172.21.15.45:0/2639069000 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffb20030020 con 0x7ffb30101610 2024-06-06T04:27:17.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:17.313+0000 7ffb36de1700 1 -- 172.21.15.45:0/2639069000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ffb3002dca0 con 0x7ffb30101610 2024-06-06T04:27:17.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:17 smithi045 ceph-mon[32299]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:17 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:18.010 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:17 smithi086 ceph-mon[36053]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:18.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:17 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:18.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:17 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:18.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:17 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:18.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:17 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:18.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:17 smithi154 ceph-mon[36428]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:18.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:17 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:18.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:17 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:18.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:17 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:18.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:17 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:18.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:18.289+0000 7ffb2dffb700 1 -- 172.21.15.45:0/2639069000 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7ffb20013070 con 0x7ffb30101610 2024-06-06T04:27:18.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:18.293+0000 7ffb36de1700 1 -- 172.21.15.45:0/2639069000 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ffb180a2c20 msgr2=0x7ffb180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:18.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:18.293+0000 7ffb36de1700 1 --2- 172.21.15.45:0/2639069000 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ffb180a2c20 0x7ffb180a50e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffb30101040 tx=0x7ffb2400b3c0 comp rx=0 tx=0).stop 2024-06-06T04:27:18.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:18.293+0000 7ffb36de1700 1 -- 172.21.15.45:0/2639069000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb30101610 msgr2=0x7ffb30199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:18.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:18.293+0000 7ffb36de1700 1 --2- 172.21.15.45:0/2639069000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb30101610 0x7ffb30199ad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffb20006010 tx=0x7ffb200045e0 comp rx=0 tx=0).stop 2024-06-06T04:27:18.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:18.293+0000 7ffb36de1700 1 -- 172.21.15.45:0/2639069000 shutdown_connections 2024-06-06T04:27:18.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:18.293+0000 7ffb36de1700 1 --2- 172.21.15.45:0/2639069000 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ffb180a2c20 0x7ffb180a50e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:18.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:18.293+0000 7ffb36de1700 1 --2- 172.21.15.45:0/2639069000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb301066f0 0x7ffb301a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:18.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:18.293+0000 7ffb36de1700 1 --2- 172.21.15.45:0/2639069000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb30101610 0x7ffb30199ad0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:18.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:18.293+0000 7ffb36de1700 1 --2- 172.21.15.45:0/2639069000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb30101fc0 0x7ffb3019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:18.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:18.293+0000 7ffb36de1700 1 -- 172.21.15.45:0/2639069000 >> 172.21.15.45:0/2639069000 conn(0x7ffb30078430 msgr2=0x7ffb301053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:18.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:18.293+0000 7ffb36de1700 1 -- 172.21.15.45:0/2639069000 shutdown_connections 2024-06-06T04:27:18.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:18.293+0000 7ffb36de1700 1 -- 172.21.15.45:0/2639069000 wait complete. 2024-06-06T04:27:18.297 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-06-06T04:27:18.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:18 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2639069000' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-06T04:27:18.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:18 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2639069000' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-06T04:27:18.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:18 smithi045 ceph-mon[32299]: osdmap e83: 12 total, 12 up, 12 in 2024-06-06T04:27:18.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:18.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:18.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:27:18.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:27:18.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:18.982 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-06-06T04:27:19.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:18 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2639069000' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-06T04:27:19.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:18 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2639069000' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-06T04:27:19.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:18 smithi086 ceph-mon[36053]: osdmap e83: 12 total, 12 up, 12 in 2024-06-06T04:27:19.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:19.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:19.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:27:19.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:27:19.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:19.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:18 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2639069000' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-06T04:27:19.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:18 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2639069000' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-06T04:27:19.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:18 smithi154 ceph-mon[36428]: osdmap e83: 12 total, 12 up, 12 in 2024-06-06T04:27:19.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:19.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:19.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:27:19.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:27:19.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:27:19.230 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:19.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:19 smithi045 ceph-mon[32299]: pgmap v290: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:19.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:19 smithi045 ceph-mon[32299]: osdmap e84: 12 total, 12 up, 12 in 2024-06-06T04:27:19.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:19 smithi045 ceph-mon[32299]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T04:27:20.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:19 smithi086 ceph-mon[36053]: pgmap v290: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:20.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:19 smithi086 ceph-mon[36053]: osdmap e84: 12 total, 12 up, 12 in 2024-06-06T04:27:20.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:19 smithi086 ceph-mon[36053]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T04:27:20.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:19 smithi154 ceph-mon[36428]: pgmap v290: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:20.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:19 smithi154 ceph-mon[36428]: osdmap e84: 12 total, 12 up, 12 in 2024-06-06T04:27:20.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:19 smithi154 ceph-mon[36428]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T04:27:20.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.835+0000 7f990105e700 1 -- 172.21.15.45:0/1642725775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98fc06a170 msgr2=0x7f98fc06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:20.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.835+0000 7f990105e700 1 --2- 172.21.15.45:0/1642725775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98fc06a170 0x7f98fc06a5f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009990 tx=0x7f98f0009ca0 comp rx=0 tx=0).stop 2024-06-06T04:27:20.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.835+0000 7f990105e700 1 -- 172.21.15.45:0/1642725775 shutdown_connections 2024-06-06T04:27:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.835+0000 7f990105e700 1 --2- 172.21.15.45:0/1642725775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98fc06ab30 0x7f98fc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.835+0000 7f990105e700 1 --2- 172.21.15.45:0/1642725775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98fc108500 0x7f98fc1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.835+0000 7f990105e700 1 --2- 172.21.15.45:0/1642725775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98fc06a170 0x7f98fc06a5f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.835+0000 7f990105e700 1 -- 172.21.15.45:0/1642725775 >> 172.21.15.45:0/1642725775 conn(0x7f98fc078340 msgr2=0x7f98fc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.835+0000 7f990105e700 1 -- 172.21.15.45:0/1642725775 shutdown_connections 2024-06-06T04:27:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.836+0000 7f990105e700 1 -- 172.21.15.45:0/1642725775 wait complete. 2024-06-06T04:27:20.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.836+0000 7f990105e700 1 Processor -- start 2024-06-06T04:27:20.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.836+0000 7f990105e700 1 -- start start 2024-06-06T04:27:20.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f990105e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98fc06a170 0x7f98fc19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:20.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f990105e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98fc06ab30 0x7f98fc19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:20.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f990105e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98fc108500 0x7f98fc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:20.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f990105e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc112710 con 0x7f98fc06ab30 2024-06-06T04:27:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f990105e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc112590 con 0x7f98fc108500 2024-06-06T04:27:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f990105e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc112890 con 0x7f98fc06a170 2024-06-06T04:27:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f98fa59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98fc06ab30 0x7f98fc19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f98fa59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98fc06ab30 0x7f98fc19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43852/0 (socket says 172.21.15.45:43852) 2024-06-06T04:27:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f98fa59c700 1 -- 172.21.15.45:0/2116882043 learned_addr learned my addr 172.21.15.45:0/2116882043 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f98fad9d700 1 --2- 172.21.15.45:0/2116882043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98fc06a170 0x7f98fc19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f98fb59e700 1 --2- 172.21.15.45:0/2116882043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98fc108500 0x7f98fc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f98fa59c700 1 -- 172.21.15.45:0/2116882043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98fc06a170 msgr2=0x7f98fc19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f98fa59c700 1 --2- 172.21.15.45:0/2116882043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98fc06a170 0x7f98fc19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f98fa59c700 1 -- 172.21.15.45:0/2116882043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98fc108500 msgr2=0x7f98fc198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f98fa59c700 1 --2- 172.21.15.45:0/2116882043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98fc108500 0x7f98fc198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f98fa59c700 1 -- 172.21.15.45:0/2116882043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98fc199170 con 0x7f98fc06ab30 2024-06-06T04:27:20.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f98fb59e700 1 --2- 172.21.15.45:0/2116882043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98fc108500 0x7f98fc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:27:20.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f98fad9d700 1 --2- 172.21.15.45:0/2116882043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98fc06a170 0x7f98fc19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:27:20.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.837+0000 7f98fa59c700 1 --2- 172.21.15.45:0/2116882043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98fc06ab30 0x7f98fc19ed90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f98f0000c00 tx=0x7f98f0003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:20.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.838+0000 7f98e3fff700 1 -- 172.21.15.45:0/2116882043 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f0027070 con 0x7f98fc06ab30 2024-06-06T04:27:20.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.838+0000 7f990105e700 1 -- 172.21.15.45:0/2116882043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98fc199400 con 0x7f98fc06ab30 2024-06-06T04:27:20.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.838+0000 7f98e3fff700 1 -- 172.21.15.45:0/2116882043 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98f000bdc0 con 0x7f98fc06ab30 2024-06-06T04:27:20.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.838+0000 7f98e3fff700 1 -- 172.21.15.45:0/2116882043 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f000f6c0 con 0x7f98fc06ab30 2024-06-06T04:27:20.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.838+0000 7f990105e700 1 -- 172.21.15.45:0/2116882043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98fc1a5720 con 0x7f98fc06ab30 2024-06-06T04:27:20.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.839+0000 7f990105e700 1 -- 172.21.15.45:0/2116882043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98fc10a9e0 con 0x7f98fc06ab30 2024-06-06T04:27:20.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.840+0000 7f98e3fff700 1 -- 172.21.15.45:0/2116882043 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f98f002a070 con 0x7f98fc06ab30 2024-06-06T04:27:20.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.841+0000 7f98e3fff700 1 --2- 172.21.15.45:0/2116882043 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f98e40a2c20 0x7f98e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:20.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.841+0000 7f98fad9d700 1 --2- 172.21.15.45:0/2116882043 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f98e40a2c20 0x7f98e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:20.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.841+0000 7f98e3fff700 1 -- 172.21.15.45:0/2116882043 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7f98f00d04a0 con 0x7f98fc06ab30 2024-06-06T04:27:20.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.841+0000 7f98fad9d700 1 --2- 172.21.15.45:0/2116882043 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f98e40a2c20 0x7f98e40a50e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f98fc199b50 tx=0x7f98ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:20.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:20.844+0000 7f98e3fff700 1 -- 172.21.15.45:0/2116882043 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f98f0031070 con 0x7f98fc06ab30 2024-06-06T04:27:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:21.054+0000 7f990105e700 1 -- 172.21.15.45:0/2116882043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f98fc069180 con 0x7f98fc06ab30 2024-06-06T04:27:21.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:21.538+0000 7f98e3fff700 1 -- 172.21.15.45:0/2116882043 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v86) ==== 171+0+0 (secure 0 0 0) 0x7f98f0013070 con 0x7f98fc06ab30 2024-06-06T04:27:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:21.540+0000 7f990105e700 1 -- 172.21.15.45:0/2116882043 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f98e40a2c20 msgr2=0x7f98e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:21.540+0000 7f990105e700 1 --2- 172.21.15.45:0/2116882043 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f98e40a2c20 0x7f98e40a50e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f98fc199b50 tx=0x7f98ec008040 comp rx=0 tx=0).stop 2024-06-06T04:27:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:21.540+0000 7f990105e700 1 -- 172.21.15.45:0/2116882043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98fc06ab30 msgr2=0x7f98fc19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:21.540+0000 7f990105e700 1 --2- 172.21.15.45:0/2116882043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98fc06ab30 0x7f98fc19ed90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f98f0000c00 tx=0x7f98f0003b30 comp rx=0 tx=0).stop 2024-06-06T04:27:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:21.541+0000 7f990105e700 1 -- 172.21.15.45:0/2116882043 shutdown_connections 2024-06-06T04:27:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:21.541+0000 7f990105e700 1 --2- 172.21.15.45:0/2116882043 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f98e40a2c20 0x7f98e40a50e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:21.541+0000 7f990105e700 1 --2- 172.21.15.45:0/2116882043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98fc108500 0x7f98fc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:21.541+0000 7f990105e700 1 --2- 172.21.15.45:0/2116882043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98fc06a170 0x7f98fc19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:21.541+0000 7f990105e700 1 --2- 172.21.15.45:0/2116882043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98fc06ab30 0x7f98fc19ed90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:21.541+0000 7f990105e700 1 -- 172.21.15.45:0/2116882043 >> 172.21.15.45:0/2116882043 conn(0x7f98fc078340 msgr2=0x7f98fc100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:21.541+0000 7f990105e700 1 -- 172.21.15.45:0/2116882043 shutdown_connections 2024-06-06T04:27:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:21.541+0000 7f990105e700 1 -- 172.21.15.45:0/2116882043 wait complete. 2024-06-06T04:27:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-06-06T04:27:21.847 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:21 smithi045 ceph-mon[32299]: osdmap e85: 12 total, 12 up, 12 in 2024-06-06T04:27:21.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:21 smithi086 ceph-mon[36053]: osdmap e85: 12 total, 12 up, 12 in 2024-06-06T04:27:21.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:21 smithi154 ceph-mon[36428]: osdmap e85: 12 total, 12 up, 12 in 2024-06-06T04:27:22.258 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-06-06T04:27:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:22.772 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:22 smithi045 ceph-mon[32299]: pgmap v293: 33 pgs: 7 creating+peering, 25 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:22.772 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:22 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2116882043' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-06T04:27:22.773 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:22 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2116882043' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-06T04:27:22.773 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:22 smithi045 ceph-mon[32299]: osdmap e86: 12 total, 12 up, 12 in 2024-06-06T04:27:22.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:22 smithi086 ceph-mon[36053]: pgmap v293: 33 pgs: 7 creating+peering, 25 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:22.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:22 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2116882043' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-06T04:27:22.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:22 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2116882043' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-06T04:27:22.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:22 smithi086 ceph-mon[36053]: osdmap e86: 12 total, 12 up, 12 in 2024-06-06T04:27:22.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:22 smithi154 ceph-mon[36428]: pgmap v293: 33 pgs: 7 creating+peering, 25 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:22.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:22 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2116882043' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-06T04:27:22.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:22 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2116882043' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-06T04:27:22.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:22 smithi154 ceph-mon[36428]: osdmap e86: 12 total, 12 up, 12 in 2024-06-06T04:27:23.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:23 smithi086 ceph-mon[36053]: pgmap v295: 97 pgs: 20 creating+peering, 63 unknown, 14 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:23.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:23 smithi086 ceph-mon[36053]: osdmap e87: 12 total, 12 up, 12 in 2024-06-06T04:27:23.891 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:23 smithi045 ceph-mon[32299]: pgmap v295: 97 pgs: 20 creating+peering, 63 unknown, 14 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:23.891 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:23 smithi045 ceph-mon[32299]: osdmap e87: 12 total, 12 up, 12 in 2024-06-06T04:27:23.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:23 smithi154 ceph-mon[36428]: pgmap v295: 97 pgs: 20 creating+peering, 63 unknown, 14 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:23.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:23 smithi154 ceph-mon[36428]: osdmap e87: 12 total, 12 up, 12 in 2024-06-06T04:27:24.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.202+0000 7ff4828ea700 1 -- 172.21.15.45:0/3234171696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff47c06ab60 msgr2=0x7ff47c06afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:24.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.202+0000 7ff4828ea700 1 --2- 172.21.15.45:0/3234171696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff47c06ab60 0x7ff47c06afe0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff470009990 tx=0x7ff470009ca0 comp rx=0 tx=0).stop 2024-06-06T04:27:24.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.202+0000 7ff4828ea700 1 -- 172.21.15.45:0/3234171696 shutdown_connections 2024-06-06T04:27:24.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.202+0000 7ff4828ea700 1 --2- 172.21.15.45:0/3234171696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff47c06b520 0x7ff47c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:24.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.202+0000 7ff4828ea700 1 --2- 172.21.15.45:0/3234171696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff47c108530 0x7ff47c108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:24.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.202+0000 7ff4828ea700 1 --2- 172.21.15.45:0/3234171696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff47c06ab60 0x7ff47c06afe0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:24.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.202+0000 7ff4828ea700 1 -- 172.21.15.45:0/3234171696 >> 172.21.15.45:0/3234171696 conn(0x7ff47c0783d0 msgr2=0x7ff47c0787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:24.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.202+0000 7ff4828ea700 1 -- 172.21.15.45:0/3234171696 shutdown_connections 2024-06-06T04:27:24.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.202+0000 7ff4828ea700 1 -- 172.21.15.45:0/3234171696 wait complete. 2024-06-06T04:27:24.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.203+0000 7ff4828ea700 1 Processor -- start 2024-06-06T04:27:24.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.203+0000 7ff4828ea700 1 -- start start 2024-06-06T04:27:24.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.203+0000 7ff4828ea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff47c06ab60 0x7ff47c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:24.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.203+0000 7ff4828ea700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff47c06b520 0x7ff47c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:24.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.203+0000 7ff4828ea700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff47c108530 0x7ff47c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:24.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.203+0000 7ff4828ea700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c112640 con 0x7ff47c06ab60 2024-06-06T04:27:24.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.203+0000 7ff4828ea700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c1124c0 con 0x7ff47c108530 2024-06-06T04:27:24.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.203+0000 7ff4828ea700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c1127c0 con 0x7ff47c06b520 2024-06-06T04:27:24.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.203+0000 7ff47bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff47c06ab60 0x7ff47c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:24.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.203+0000 7ff47bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff47c06ab60 0x7ff47c19e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43870/0 (socket says 172.21.15.45:43870) 2024-06-06T04:27:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.203+0000 7ff47bfff700 1 -- 172.21.15.45:0/2287455171 learned_addr learned my addr 172.21.15.45:0/2287455171 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.204+0000 7ff47b7fe700 1 --2- 172.21.15.45:0/2287455171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff47c06b520 0x7ff47c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.204+0000 7ff480e87700 1 --2- 172.21.15.45:0/2287455171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff47c108530 0x7ff47c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.204+0000 7ff47bfff700 1 -- 172.21.15.45:0/2287455171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff47c06b520 msgr2=0x7ff47c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.204+0000 7ff47bfff700 1 --2- 172.21.15.45:0/2287455171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff47c06b520 0x7ff47c19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.204+0000 7ff47bfff700 1 -- 172.21.15.45:0/2287455171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff47c108530 msgr2=0x7ff47c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.204+0000 7ff47bfff700 1 --2- 172.21.15.45:0/2287455171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff47c108530 0x7ff47c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.204+0000 7ff47bfff700 1 -- 172.21.15.45:0/2287455171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47c199100 con 0x7ff47c06ab60 2024-06-06T04:27:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.204+0000 7ff480e87700 1 --2- 172.21.15.45:0/2287455171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff47c108530 0x7ff47c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:27:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.204+0000 7ff47bfff700 1 --2- 172.21.15.45:0/2287455171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff47c06ab60 0x7ff47c19e7e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00d950 tx=0x7ff46c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.204+0000 7ff4797fa700 1 -- 172.21.15.45:0/2287455171 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c016070 con 0x7ff47c06ab60 2024-06-06T04:27:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.204+0000 7ff4828ea700 1 -- 172.21.15.45:0/2287455171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff47c1993f0 con 0x7ff47c06ab60 2024-06-06T04:27:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.204+0000 7ff4797fa700 1 -- 172.21.15.45:0/2287455171 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff46c009df0 con 0x7ff47c06ab60 2024-06-06T04:27:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.204+0000 7ff4828ea700 1 -- 172.21.15.45:0/2287455171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff47c1013c0 con 0x7ff47c06ab60 2024-06-06T04:27:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.204+0000 7ff4797fa700 1 -- 172.21.15.45:0/2287455171 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c020060 con 0x7ff47c06ab60 2024-06-06T04:27:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.205+0000 7ff4828ea700 1 -- 172.21.15.45:0/2287455171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff47c103bf0 con 0x7ff47c06ab60 2024-06-06T04:27:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.207+0000 7ff4797fa700 1 -- 172.21.15.45:0/2287455171 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff46c019070 con 0x7ff47c06ab60 2024-06-06T04:27:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.207+0000 7ff4797fa700 1 --2- 172.21.15.45:0/2287455171 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff4640a2c20 0x7ff4640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.207+0000 7ff47b7fe700 1 --2- 172.21.15.45:0/2287455171 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff4640a2c20 0x7ff4640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:24.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.207+0000 7ff4797fa700 1 -- 172.21.15.45:0/2287455171 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7879+0+0 (secure 0 0 0) 0x7ff46c0cedb0 con 0x7ff47c06ab60 2024-06-06T04:27:24.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.208+0000 7ff47b7fe700 1 --2- 172.21.15.45:0/2287455171 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff4640a2c20 0x7ff4640a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff4700098e0 tx=0x7ff470005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:24.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.211+0000 7ff4797fa700 1 -- 172.21.15.45:0/2287455171 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff46c09bec0 con 0x7ff47c06ab60 2024-06-06T04:27:24.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.461+0000 7ff4828ea700 1 -- 172.21.15.45:0/2287455171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff47c02dca0 con 0x7ff47c06ab60 2024-06-06T04:27:24.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.561+0000 7ff4797fa700 1 -- 172.21.15.45:0/2287455171 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7ff46c09b7c0 con 0x7ff47c06ab60 2024-06-06T04:27:24.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.564+0000 7ff4828ea700 1 -- 172.21.15.45:0/2287455171 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff4640a2c20 msgr2=0x7ff4640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:24.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.564+0000 7ff4828ea700 1 --2- 172.21.15.45:0/2287455171 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff4640a2c20 0x7ff4640a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff4700098e0 tx=0x7ff470005960 comp rx=0 tx=0).stop 2024-06-06T04:27:24.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.564+0000 7ff4828ea700 1 -- 172.21.15.45:0/2287455171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff47c06ab60 msgr2=0x7ff47c19e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:24.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.564+0000 7ff4828ea700 1 --2- 172.21.15.45:0/2287455171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff47c06ab60 0x7ff47c19e7e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00d950 tx=0x7ff46c00dc60 comp rx=0 tx=0).stop 2024-06-06T04:27:24.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.564+0000 7ff4828ea700 1 -- 172.21.15.45:0/2287455171 shutdown_connections 2024-06-06T04:27:24.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.564+0000 7ff4828ea700 1 --2- 172.21.15.45:0/2287455171 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff4640a2c20 0x7ff4640a50e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:24.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.564+0000 7ff4828ea700 1 --2- 172.21.15.45:0/2287455171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff47c108530 0x7ff47c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:24.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.564+0000 7ff4828ea700 1 --2- 172.21.15.45:0/2287455171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff47c06ab60 0x7ff47c19e7e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:24.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.564+0000 7ff4828ea700 1 --2- 172.21.15.45:0/2287455171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff47c06b520 0x7ff47c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:24.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.564+0000 7ff4828ea700 1 -- 172.21.15.45:0/2287455171 >> 172.21.15.45:0/2287455171 conn(0x7ff47c0783d0 msgr2=0x7ff47c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:24.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.564+0000 7ff4828ea700 1 -- 172.21.15.45:0/2287455171 shutdown_connections 2024-06-06T04:27:24.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:24.564+0000 7ff4828ea700 1 -- 172.21.15.45:0/2287455171 wait complete. 2024-06-06T04:27:24.566 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 2 and data pool 3 2024-06-06T04:27:24.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:24 smithi045 ceph-mon[32299]: osdmap e88: 12 total, 12 up, 12 in 2024-06-06T04:27:24.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:24 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2287455171' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-06T04:27:24.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:24 smithi045 ceph-mon[32299]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-06T04:27:24.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:24 smithi086 ceph-mon[36053]: osdmap e88: 12 total, 12 up, 12 in 2024-06-06T04:27:24.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:24 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2287455171' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-06T04:27:24.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:24 smithi086 ceph-mon[36053]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-06T04:27:24.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:24 smithi154 ceph-mon[36428]: osdmap e88: 12 total, 12 up, 12 in 2024-06-06T04:27:24.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:24 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2287455171' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-06T04:27:24.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:24 smithi154 ceph-mon[36428]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-06T04:27:25.235 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-06-06T04:27:25.235 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-06-06T04:27:25.487 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:25.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:25 smithi086 ceph-mon[36053]: pgmap v298: 97 pgs: 13 creating+peering, 14 unknown, 70 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:25.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:25 smithi086 ceph-mon[36053]: osdmap e89: 12 total, 12 up, 12 in 2024-06-06T04:27:25.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:25 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2287455171' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-06T04:27:25.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:25 smithi086 ceph-mon[36053]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-06T04:27:25.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:25 smithi086 ceph-mon[36053]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-06T04:27:25.859 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:25 smithi045 ceph-mon[32299]: pgmap v298: 97 pgs: 13 creating+peering, 14 unknown, 70 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:25.860 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:25 smithi045 ceph-mon[32299]: osdmap e89: 12 total, 12 up, 12 in 2024-06-06T04:27:25.860 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:25 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2287455171' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-06T04:27:25.860 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:25 smithi045 ceph-mon[32299]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-06T04:27:25.860 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:25 smithi045 ceph-mon[32299]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-06T04:27:25.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:25 smithi154 ceph-mon[36428]: pgmap v298: 97 pgs: 13 creating+peering, 14 unknown, 70 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:25.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:25 smithi154 ceph-mon[36428]: osdmap e89: 12 total, 12 up, 12 in 2024-06-06T04:27:25.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:25 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2287455171' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-06T04:27:25.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:25 smithi154 ceph-mon[36428]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-06T04:27:25.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:25 smithi154 ceph-mon[36428]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-06T04:27:26.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:26 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] up:active 2024-06-06T04:27:26.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:26 smithi045 ceph-mon[32299]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T04:27:26.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:26 smithi045 ceph-mon[32299]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T04:27:26.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:26 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] up:active 2024-06-06T04:27:26.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:26 smithi086 ceph-mon[36053]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T04:27:26.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:26 smithi086 ceph-mon[36053]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T04:27:26.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:26 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] up:active 2024-06-06T04:27:26.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:26 smithi154 ceph-mon[36428]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T04:27:26.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:26 smithi154 ceph-mon[36428]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T04:27:27.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.274+0000 7fb53d345700 1 -- 172.21.15.45:0/4011937633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb538101390 msgr2=0x7fb538101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.274+0000 7fb53d345700 1 --2- 172.21.15.45:0/4011937633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb538101390 0x7fb538101810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009990 tx=0x7fb52c009ca0 comp rx=0 tx=0).stop 2024-06-06T04:27:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.274+0000 7fb53d345700 1 -- 172.21.15.45:0/4011937633 shutdown_connections 2024-06-06T04:27:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.274+0000 7fb53d345700 1 --2- 172.21.15.45:0/4011937633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb538101d50 0x7fb53810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.274+0000 7fb53d345700 1 --2- 172.21.15.45:0/4011937633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb538107400 0x7fb5381077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.274+0000 7fb53d345700 1 --2- 172.21.15.45:0/4011937633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb538101390 0x7fb538101810 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.274+0000 7fb53d345700 1 -- 172.21.15.45:0/4011937633 >> 172.21.15.45:0/4011937633 conn(0x7fb538078340 msgr2=0x7fb538078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.274+0000 7fb53d345700 1 -- 172.21.15.45:0/4011937633 shutdown_connections 2024-06-06T04:27:27.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.274+0000 7fb53d345700 1 -- 172.21.15.45:0/4011937633 wait complete. 2024-06-06T04:27:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.275+0000 7fb53d345700 1 Processor -- start 2024-06-06T04:27:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.275+0000 7fb53d345700 1 -- start start 2024-06-06T04:27:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.275+0000 7fb53d345700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb538101390 0x7fb53819d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.275+0000 7fb53d345700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb538101d50 0x7fb53819dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.275+0000 7fb53d345700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb538107400 0x7fb538197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.275+0000 7fb53d345700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb5381115a0 con 0x7fb538101390 2024-06-06T04:27:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.275+0000 7fb53d345700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb538111420 con 0x7fb538107400 2024-06-06T04:27:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.275+0000 7fb53d345700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb538111720 con 0x7fb538101d50 2024-06-06T04:27:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.276+0000 7fb536ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb538101390 0x7fb53819d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.276+0000 7fb536ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb538101390 0x7fb53819d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43890/0 (socket says 172.21.15.45:43890) 2024-06-06T04:27:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.276+0000 7fb536ffd700 1 -- 172.21.15.45:0/3216254967 learned_addr learned my addr 172.21.15.45:0/3216254967 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:27.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.276+0000 7fb5377fe700 1 --2- 172.21.15.45:0/3216254967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb538107400 0x7fb538197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.276+0000 7fb5367fc700 1 --2- 172.21.15.45:0/3216254967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb538101d50 0x7fb53819dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.276+0000 7fb536ffd700 1 -- 172.21.15.45:0/3216254967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb538101d50 msgr2=0x7fb53819dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.276+0000 7fb536ffd700 1 --2- 172.21.15.45:0/3216254967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb538101d50 0x7fb53819dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.276+0000 7fb536ffd700 1 -- 172.21.15.45:0/3216254967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb538107400 msgr2=0x7fb538197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.276+0000 7fb536ffd700 1 --2- 172.21.15.45:0/3216254967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb538107400 0x7fb538197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.276+0000 7fb536ffd700 1 -- 172.21.15.45:0/3216254967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5381980c0 con 0x7fb538101390 2024-06-06T04:27:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.276+0000 7fb536ffd700 1 --2- 172.21.15.45:0/3216254967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb538101390 0x7fb53819d750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb52800d9e0 tx=0x7fb52800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.276+0000 7fb51ffff700 1 -- 172.21.15.45:0/3216254967 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52801b070 con 0x7fb538101390 2024-06-06T04:27:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.276+0000 7fb51ffff700 1 -- 172.21.15.45:0/3216254967 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb52800eb70 con 0x7fb538101390 2024-06-06T04:27:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.276+0000 7fb53d345700 1 -- 172.21.15.45:0/3216254967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5381983b0 con 0x7fb538101390 2024-06-06T04:27:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.277+0000 7fb51ffff700 1 -- 172.21.15.45:0/3216254967 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5280145a0 con 0x7fb538101390 2024-06-06T04:27:27.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.277+0000 7fb53d345700 1 -- 172.21.15.45:0/3216254967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb53806a170 con 0x7fb538101390 2024-06-06T04:27:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.278+0000 7fb53d345700 1 -- 172.21.15.45:0/3216254967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb538069180 con 0x7fb538101390 2024-06-06T04:27:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.279+0000 7fb51ffff700 1 -- 172.21.15.45:0/3216254967 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb52801e070 con 0x7fb538101390 2024-06-06T04:27:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.279+0000 7fb51ffff700 1 --2- 172.21.15.45:0/3216254967 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb5200a2ca0 0x7fb5200a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.280+0000 7fb5367fc700 1 --2- 172.21.15.45:0/3216254967 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb5200a2ca0 0x7fb5200a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:27.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.280+0000 7fb51ffff700 1 -- 172.21.15.45:0/3216254967 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fb5280d0c40 con 0x7fb538101390 2024-06-06T04:27:27.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.280+0000 7fb5367fc700 1 --2- 172.21.15.45:0/3216254967 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb5200a2ca0 0x7fb5200a5160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb52c00b5c0 tx=0x7fb52c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:27.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.283+0000 7fb51ffff700 1 -- 172.21.15.45:0/3216254967 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb528025070 con 0x7fb538101390 2024-06-06T04:27:27.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.500+0000 7fb53d345700 1 -- 172.21.15.45:0/3216254967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fb538198e40 con 0x7fb538101390 2024-06-06T04:27:27.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.576+0000 7fb51ffff700 1 -- 172.21.15.45:0/3216254967 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v90) ==== 157+0+0 (secure 0 0 0) 0x7fb52809dcf0 con 0x7fb538101390 2024-06-06T04:27:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.579+0000 7fb53d345700 1 -- 172.21.15.45:0/3216254967 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb5200a2ca0 msgr2=0x7fb5200a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.579+0000 7fb53d345700 1 --2- 172.21.15.45:0/3216254967 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb5200a2ca0 0x7fb5200a5160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb52c00b5c0 tx=0x7fb52c005960 comp rx=0 tx=0).stop 2024-06-06T04:27:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.579+0000 7fb53d345700 1 -- 172.21.15.45:0/3216254967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb538101390 msgr2=0x7fb53819d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.579+0000 7fb53d345700 1 --2- 172.21.15.45:0/3216254967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb538101390 0x7fb53819d750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb52800d9e0 tx=0x7fb52800dcf0 comp rx=0 tx=0).stop 2024-06-06T04:27:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.579+0000 7fb53d345700 1 -- 172.21.15.45:0/3216254967 shutdown_connections 2024-06-06T04:27:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.579+0000 7fb53d345700 1 --2- 172.21.15.45:0/3216254967 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb5200a2ca0 0x7fb5200a5160 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.579+0000 7fb53d345700 1 --2- 172.21.15.45:0/3216254967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb538107400 0x7fb538197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.579+0000 7fb53d345700 1 --2- 172.21.15.45:0/3216254967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb538101390 0x7fb53819d750 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.579+0000 7fb53d345700 1 --2- 172.21.15.45:0/3216254967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb538101d50 0x7fb53819dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.579+0000 7fb53d345700 1 -- 172.21.15.45:0/3216254967 >> 172.21.15.45:0/3216254967 conn(0x7fb538078340 msgr2=0x7fb5380fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.579+0000 7fb53d345700 1 -- 172.21.15.45:0/3216254967 shutdown_connections 2024-06-06T04:27:27.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:27.579+0000 7fb53d345700 1 -- 172.21.15.45:0/3216254967 wait complete. 2024-06-06T04:27:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:27 smithi086 ceph-mon[36053]: pgmap v300: 97 pgs: 13 creating+peering, 4 unknown, 80 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 212 B/s wr, 1 op/s 2024-06-06T04:27:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:27 smithi086 ceph-mon[36053]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-06T04:27:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:27 smithi086 ceph-mon[36053]: Cluster is now healthy 2024-06-06T04:27:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:27 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3216254967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-06T04:27:27.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:27 smithi045 ceph-mon[32299]: pgmap v300: 97 pgs: 13 creating+peering, 4 unknown, 80 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 212 B/s wr, 1 op/s 2024-06-06T04:27:27.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:27 smithi045 ceph-mon[32299]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-06T04:27:27.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:27 smithi045 ceph-mon[32299]: Cluster is now healthy 2024-06-06T04:27:27.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:27 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3216254967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-06T04:27:27.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:27 smithi154 ceph-mon[36428]: pgmap v300: 97 pgs: 13 creating+peering, 4 unknown, 80 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 212 B/s wr, 1 op/s 2024-06-06T04:27:27.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:27 smithi154 ceph-mon[36428]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-06T04:27:27.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:27 smithi154 ceph-mon[36428]: Cluster is now healthy 2024-06-06T04:27:27.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:27 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3216254967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-06T04:27:28.269 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-06-06T04:27:28.521 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:28.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:28 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3216254967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-06T04:27:28.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:28 smithi086 ceph-mon[36053]: osdmap e90: 12 total, 12 up, 12 in 2024-06-06T04:27:28.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:28 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3216254967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-06T04:27:28.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:28 smithi154 ceph-mon[36428]: osdmap e90: 12 total, 12 up, 12 in 2024-06-06T04:27:28.932 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:28 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3216254967' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-06T04:27:28.932 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:28 smithi045 ceph-mon[32299]: osdmap e90: 12 total, 12 up, 12 in 2024-06-06T04:27:29.757 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:29 smithi045 ceph-mon[32299]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-06-06T04:27:29.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:29 smithi086 ceph-mon[36053]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-06-06T04:27:29.859 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:27:29 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-b[65935]: 2024-06-06T04:27:29.441+0000 7f432ec43700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-06-06T04:27:29.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:29 smithi154 ceph-mon[36428]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-06-06T04:27:30.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.343+0000 7fe51cda1700 1 -- 172.21.15.45:0/1005021736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe518100e50 msgr2=0x7fe5181012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:30.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.343+0000 7fe51cda1700 1 --2- 172.21.15.45:0/1005021736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe518100e50 0x7fe5181012d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe500009990 tx=0x7fe500009ca0 comp rx=0 tx=0).stop 2024-06-06T04:27:30.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.343+0000 7fe51cda1700 1 -- 172.21.15.45:0/1005021736 shutdown_connections 2024-06-06T04:27:30.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.343+0000 7fe51cda1700 1 --2- 172.21.15.45:0/1005021736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe518101810 0x7fe51810f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:30.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.343+0000 7fe51cda1700 1 --2- 172.21.15.45:0/1005021736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe518107570 0x7fe518107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:30.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.343+0000 7fe51cda1700 1 --2- 172.21.15.45:0/1005021736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe518100e50 0x7fe5181012d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:30.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.343+0000 7fe51cda1700 1 -- 172.21.15.45:0/1005021736 >> 172.21.15.45:0/1005021736 conn(0x7fe5180766b0 msgr2=0x7fe518076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:30.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.343+0000 7fe51cda1700 1 -- 172.21.15.45:0/1005021736 shutdown_connections 2024-06-06T04:27:30.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.343+0000 7fe51cda1700 1 -- 172.21.15.45:0/1005021736 wait complete. 2024-06-06T04:27:30.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.344+0000 7fe51cda1700 1 Processor -- start 2024-06-06T04:27:30.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.344+0000 7fe51cda1700 1 -- start start 2024-06-06T04:27:30.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.344+0000 7fe51cda1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe518100e50 0x7fe5181a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:30.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.344+0000 7fe51cda1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe518101810 0x7fe5181a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:30.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.344+0000 7fe51cda1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe518107570 0x7fe51810e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:30.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.344+0000 7fe51cda1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe518112660 con 0x7fe518107570 2024-06-06T04:27:30.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.344+0000 7fe51cda1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181124e0 con 0x7fe518100e50 2024-06-06T04:27:30.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.344+0000 7fe51cda1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181127e0 con 0x7fe518101810 2024-06-06T04:27:30.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.345+0000 7fe516d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe518107570 0x7fe51810e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:30.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.345+0000 7fe516d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe518107570 0x7fe51810e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46748/0 (socket says 172.21.15.45:46748) 2024-06-06T04:27:30.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.345+0000 7fe516d9d700 1 -- 172.21.15.45:0/4005626118 learned_addr learned my addr 172.21.15.45:0/4005626118 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:30.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.345+0000 7fe51659c700 1 --2- 172.21.15.45:0/4005626118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe518100e50 0x7fe5181a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:30.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.345+0000 7fe50dd9b700 1 --2- 172.21.15.45:0/4005626118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe518101810 0x7fe5181a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.345+0000 7fe516d9d700 1 -- 172.21.15.45:0/4005626118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe518101810 msgr2=0x7fe5181a5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.345+0000 7fe516d9d700 1 --2- 172.21.15.45:0/4005626118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe518101810 0x7fe5181a5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.345+0000 7fe516d9d700 1 -- 172.21.15.45:0/4005626118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe518100e50 msgr2=0x7fe5181a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.345+0000 7fe516d9d700 1 --2- 172.21.15.45:0/4005626118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe518100e50 0x7fe5181a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.345+0000 7fe516d9d700 1 -- 172.21.15.45:0/4005626118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51810eae0 con 0x7fe518107570 2024-06-06T04:27:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.345+0000 7fe516d9d700 1 --2- 172.21.15.45:0/4005626118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe518107570 0x7fe51810e310 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe510007ee0 tx=0x7fe51000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.345+0000 7fe50ffff700 1 -- 172.21.15.45:0/4005626118 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510004bb0 con 0x7fe518107570 2024-06-06T04:27:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.345+0000 7fe50ffff700 1 -- 172.21.15.45:0/4005626118 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe510004d50 con 0x7fe518107570 2024-06-06T04:27:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.345+0000 7fe51cda1700 1 -- 172.21.15.45:0/4005626118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51810edd0 con 0x7fe518107570 2024-06-06T04:27:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.345+0000 7fe50ffff700 1 -- 172.21.15.45:0/4005626118 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5100058e0 con 0x7fe518107570 2024-06-06T04:27:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.345+0000 7fe51cda1700 1 -- 172.21.15.45:0/4005626118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5181abb20 con 0x7fe518107570 2024-06-06T04:27:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.346+0000 7fe51cda1700 1 -- 172.21.15.45:0/4005626118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe518050a00 con 0x7fe518107570 2024-06-06T04:27:30.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.348+0000 7fe50ffff700 1 -- 172.21.15.45:0/4005626118 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe51001a020 con 0x7fe518107570 2024-06-06T04:27:30.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.348+0000 7fe50ffff700 1 --2- 172.21.15.45:0/4005626118 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5040a2d40 0x7fe5040a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:30.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.348+0000 7fe51659c700 1 --2- 172.21.15.45:0/4005626118 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5040a2d40 0x7fe5040a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:30.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.348+0000 7fe50ffff700 1 -- 172.21.15.45:0/4005626118 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe5100ced90 con 0x7fe518107570 2024-06-06T04:27:30.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.349+0000 7fe51659c700 1 --2- 172.21.15.45:0/4005626118 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5040a2d40 0x7fe5040a5200 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe518077c30 tx=0x7fe50800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:30.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.352+0000 7fe50ffff700 1 -- 172.21.15.45:0/4005626118 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe510010070 con 0x7fe518107570 2024-06-06T04:27:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:30.559+0000 7fe51cda1700 1 -- 172.21.15.45:0/4005626118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe5181abd50 con 0x7fe518107570 2024-06-06T04:27:30.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:30 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/4005626118' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T04:27:30.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:30 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/4005626118' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T04:27:30.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:30 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/4005626118' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T04:27:31.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:31.596+0000 7fe50ffff700 1 -- 172.21.15.45:0/4005626118 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v92) ==== 228+0+0 (secure 0 0 0) 0x7fe51009bcc0 con 0x7fe518107570 2024-06-06T04:27:31.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:31.599+0000 7fe51cda1700 1 -- 172.21.15.45:0/4005626118 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5040a2d40 msgr2=0x7fe5040a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:31.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:31.599+0000 7fe51cda1700 1 --2- 172.21.15.45:0/4005626118 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5040a2d40 0x7fe5040a5200 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe518077c30 tx=0x7fe50800a380 comp rx=0 tx=0).stop 2024-06-06T04:27:31.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:31.599+0000 7fe51cda1700 1 -- 172.21.15.45:0/4005626118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe518107570 msgr2=0x7fe51810e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:31.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:31.599+0000 7fe51cda1700 1 --2- 172.21.15.45:0/4005626118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe518107570 0x7fe51810e310 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe510007ee0 tx=0x7fe51000d3b0 comp rx=0 tx=0).stop 2024-06-06T04:27:31.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:31.599+0000 7fe51cda1700 1 -- 172.21.15.45:0/4005626118 shutdown_connections 2024-06-06T04:27:31.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:31.599+0000 7fe51cda1700 1 --2- 172.21.15.45:0/4005626118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe518107570 0x7fe51810e310 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:31.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:31.599+0000 7fe51cda1700 1 --2- 172.21.15.45:0/4005626118 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5040a2d40 0x7fe5040a5200 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:31.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:31.599+0000 7fe51cda1700 1 --2- 172.21.15.45:0/4005626118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe518100e50 0x7fe5181a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:31.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:31.599+0000 7fe51cda1700 1 --2- 172.21.15.45:0/4005626118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe518101810 0x7fe5181a5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:31.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:31.599+0000 7fe51cda1700 1 -- 172.21.15.45:0/4005626118 >> 172.21.15.45:0/4005626118 conn(0x7fe5180766b0 msgr2=0x7fe5181062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:31.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:31.599+0000 7fe51cda1700 1 -- 172.21.15.45:0/4005626118 shutdown_connections 2024-06-06T04:27:31.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:31.599+0000 7fe51cda1700 1 -- 172.21.15.45:0/4005626118 wait complete. 2024-06-06T04:27:31.601 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data_ec' created 2024-06-06T04:27:31.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:31 smithi086 ceph-mon[36053]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-06-06T04:27:31.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:31 smithi086 ceph-mon[36053]: osdmap e91: 12 total, 12 up, 12 in 2024-06-06T04:27:31.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:31 smithi154 ceph-mon[36428]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-06-06T04:27:31.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:31 smithi154 ceph-mon[36428]: osdmap e91: 12 total, 12 up, 12 in 2024-06-06T04:27:31.974 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:31 smithi045 ceph-mon[32299]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-06-06T04:27:31.975 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:31 smithi045 ceph-mon[32299]: osdmap e91: 12 total, 12 up, 12 in 2024-06-06T04:27:32.402 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-06-06T04:27:32.678 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:32.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:32 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/4005626118' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-06T04:27:32.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:32 smithi045 ceph-mon[32299]: osdmap e92: 12 total, 12 up, 12 in 2024-06-06T04:27:32.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:32 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/4005626118' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-06T04:27:32.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:32 smithi086 ceph-mon[36053]: osdmap e92: 12 total, 12 up, 12 in 2024-06-06T04:27:32.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:32 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/4005626118' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-06T04:27:32.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:32 smithi154 ceph-mon[36428]: osdmap e92: 12 total, 12 up, 12 in 2024-06-06T04:27:33.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:33 smithi086 ceph-mon[36053]: pgmap v306: 129 pgs: 6 creating+peering, 26 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-06-06T04:27:33.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:33 smithi086 ceph-mon[36053]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T04:27:33.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:33 smithi086 ceph-mon[36053]: osdmap e93: 12 total, 12 up, 12 in 2024-06-06T04:27:33.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:33 smithi154 ceph-mon[36428]: pgmap v306: 129 pgs: 6 creating+peering, 26 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-06-06T04:27:33.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:33 smithi154 ceph-mon[36428]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T04:27:33.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:33 smithi154 ceph-mon[36428]: osdmap e93: 12 total, 12 up, 12 in 2024-06-06T04:27:33.937 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:33 smithi045 ceph-mon[32299]: pgmap v306: 129 pgs: 6 creating+peering, 26 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-06-06T04:27:33.937 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:33 smithi045 ceph-mon[32299]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T04:27:33.937 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:33 smithi045 ceph-mon[32299]: osdmap e93: 12 total, 12 up, 12 in 2024-06-06T04:27:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.372+0000 7fbc177c2700 1 -- 172.21.15.45:0/3592146565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc10107840 msgr2=0x7fbc1010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.372+0000 7fbc177c2700 1 --2- 172.21.15.45:0/3592146565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc10107840 0x7fbc1010f3c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800b0d0 tx=0x7fbc0800b3e0 comp rx=0 tx=0).stop 2024-06-06T04:27:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.372+0000 7fbc177c2700 1 -- 172.21.15.45:0/3592146565 shutdown_connections 2024-06-06T04:27:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.372+0000 7fbc177c2700 1 --2- 172.21.15.45:0/3592146565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc10107840 0x7fbc1010f3c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.373+0000 7fbc177c2700 1 --2- 172.21.15.45:0/3592146565 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc10102740 0x7fbc10102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.373+0000 7fbc177c2700 1 --2- 172.21.15.45:0/3592146565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc101030f0 0x7fbc10107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.373+0000 7fbc177c2700 1 -- 172.21.15.45:0/3592146565 >> 172.21.15.45:0/3592146565 conn(0x7fbc100795f0 msgr2=0x7fbc10079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.373+0000 7fbc177c2700 1 -- 172.21.15.45:0/3592146565 shutdown_connections 2024-06-06T04:27:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.373+0000 7fbc177c2700 1 -- 172.21.15.45:0/3592146565 wait complete. 2024-06-06T04:27:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.373+0000 7fbc177c2700 1 Processor -- start 2024-06-06T04:27:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.373+0000 7fbc177c2700 1 -- start start 2024-06-06T04:27:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.373+0000 7fbc177c2700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc10102740 0x7fbc1019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.373+0000 7fbc177c2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc101030f0 0x7fbc1019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc177c2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc10107840 0x7fbc10198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc177c2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc10112690 con 0x7fbc10107840 2024-06-06T04:27:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc177c2700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc10112510 con 0x7fbc10102740 2024-06-06T04:27:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc177c2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbc10112810 con 0x7fbc101030f0 2024-06-06T04:27:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc15d5f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc10107840 0x7fbc10198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc15d5f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc10107840 0x7fbc10198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46760/0 (socket says 172.21.15.45:46760) 2024-06-06T04:27:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc15d5f700 1 -- 172.21.15.45:0/2329965930 learned_addr learned my addr 172.21.15.45:0/2329965930 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc1555e700 1 --2- 172.21.15.45:0/2329965930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc10102740 0x7fbc1019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc14d5d700 1 --2- 172.21.15.45:0/2329965930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc101030f0 0x7fbc1019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc15d5f700 1 -- 172.21.15.45:0/2329965930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc101030f0 msgr2=0x7fbc1019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc15d5f700 1 --2- 172.21.15.45:0/2329965930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc101030f0 0x7fbc1019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc15d5f700 1 -- 172.21.15.45:0/2329965930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc10102740 msgr2=0x7fbc1019e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc15d5f700 1 --2- 172.21.15.45:0/2329965930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc10102740 0x7fbc1019e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc15d5f700 1 -- 172.21.15.45:0/2329965930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc101990b0 con 0x7fbc10107840 2024-06-06T04:27:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc15d5f700 1 --2- 172.21.15.45:0/2329965930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc10107840 0x7fbc10198950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009fd0 tx=0x7fbc08007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc067fc700 1 -- 172.21.15.45:0/2329965930 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0801f070 con 0x7fbc10107840 2024-06-06T04:27:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc067fc700 1 -- 172.21.15.45:0/2329965930 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc08003c00 con 0x7fbc10107840 2024-06-06T04:27:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc067fc700 1 -- 172.21.15.45:0/2329965930 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0801b770 con 0x7fbc10107840 2024-06-06T04:27:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc177c2700 1 -- 172.21.15.45:0/2329965930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc10199340 con 0x7fbc10107840 2024-06-06T04:27:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.374+0000 7fbc177c2700 1 -- 172.21.15.45:0/2329965930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc101a5610 con 0x7fbc10107840 2024-06-06T04:27:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.375+0000 7fbc177c2700 1 -- 172.21.15.45:0/2329965930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc10069180 con 0x7fbc10107840 2024-06-06T04:27:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.377+0000 7fbc067fc700 1 -- 172.21.15.45:0/2329965930 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbc08022070 con 0x7fbc10107840 2024-06-06T04:27:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.377+0000 7fbc067fc700 1 --2- 172.21.15.45:0/2329965930 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbbfc0a2c20 0x7fbbfc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.377+0000 7fbc1555e700 1 --2- 172.21.15.45:0/2329965930 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbbfc0a2c20 0x7fbbfc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.378+0000 7fbc067fc700 1 -- 172.21.15.45:0/2329965930 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8770+0+0 (secure 0 0 0) 0x7fbc0800f020 con 0x7fbc10107840 2024-06-06T04:27:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.378+0000 7fbc1555e700 1 --2- 172.21.15.45:0/2329965930 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbbfc0a2c20 0x7fbbfc0a50e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbc10107340 tx=0x7fbc0c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.381+0000 7fbc067fc700 1 -- 172.21.15.45:0/2329965930 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbc08029020 con 0x7fbc10107840 2024-06-06T04:27:34.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.589+0000 7fbc177c2700 1 -- 172.21.15.45:0/2329965930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fbc1010aa80 con 0x7fbc10107840 2024-06-06T04:27:34.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.612+0000 7fbc067fc700 1 -- 172.21.15.45:0/2329965930 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v95) ==== 169+0+0 (secure 0 0 0) 0x7fbc0809d700 con 0x7fbc10107840 2024-06-06T04:27:34.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.615+0000 7fbc177c2700 1 -- 172.21.15.45:0/2329965930 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbbfc0a2c20 msgr2=0x7fbbfc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:34.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.615+0000 7fbc177c2700 1 --2- 172.21.15.45:0/2329965930 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbbfc0a2c20 0x7fbbfc0a50e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbc10107340 tx=0x7fbc0c009190 comp rx=0 tx=0).stop 2024-06-06T04:27:34.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.615+0000 7fbc177c2700 1 -- 172.21.15.45:0/2329965930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc10107840 msgr2=0x7fbc10198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:34.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.615+0000 7fbc177c2700 1 --2- 172.21.15.45:0/2329965930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc10107840 0x7fbc10198950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009fd0 tx=0x7fbc08007a20 comp rx=0 tx=0).stop 2024-06-06T04:27:34.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.615+0000 7fbc177c2700 1 -- 172.21.15.45:0/2329965930 shutdown_connections 2024-06-06T04:27:34.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.616+0000 7fbc177c2700 1 --2- 172.21.15.45:0/2329965930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc10107840 0x7fbc10198950 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:34.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.616+0000 7fbc177c2700 1 --2- 172.21.15.45:0/2329965930 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbbfc0a2c20 0x7fbbfc0a50e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:34.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.616+0000 7fbc177c2700 1 --2- 172.21.15.45:0/2329965930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc10102740 0x7fbc1019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:34.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.616+0000 7fbc177c2700 1 --2- 172.21.15.45:0/2329965930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc101030f0 0x7fbc1019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:34.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.616+0000 7fbc177c2700 1 -- 172.21.15.45:0/2329965930 >> 172.21.15.45:0/2329965930 conn(0x7fbc100795f0 msgr2=0x7fbc10100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:34.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.616+0000 7fbc177c2700 1 -- 172.21.15.45:0/2329965930 shutdown_connections 2024-06-06T04:27:34.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:34.616+0000 7fbc177c2700 1 -- 172.21.15.45:0/2329965930 wait complete. 2024-06-06T04:27:34.618 INFO:teuthology.orchestra.run.smithi045.stderr:set pool 4 allow_ec_overwrites to true 2024-06-06T04:27:34.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:34 smithi154 ceph-mon[36428]: osdmap e94: 12 total, 12 up, 12 in 2024-06-06T04:27:34.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:34 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2329965930' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-06T04:27:34.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:34 smithi045 ceph-mon[32299]: osdmap e94: 12 total, 12 up, 12 in 2024-06-06T04:27:34.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:34 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2329965930' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-06T04:27:35.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:34 smithi086 ceph-mon[36053]: osdmap e94: 12 total, 12 up, 12 in 2024-06-06T04:27:35.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:34 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2329965930' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-06T04:27:35.307 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-06-06T04:27:35.559 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:35.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:35 smithi154 ceph-mon[36428]: pgmap v309: 129 pgs: 10 creating+peering, 6 unknown, 113 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:35.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:35 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2329965930' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-06T04:27:35.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:35 smithi154 ceph-mon[36428]: osdmap e95: 12 total, 12 up, 12 in 2024-06-06T04:27:35.939 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:35 smithi045 ceph-mon[32299]: pgmap v309: 129 pgs: 10 creating+peering, 6 unknown, 113 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:35.939 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:35 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2329965930' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-06T04:27:35.939 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:35 smithi045 ceph-mon[32299]: osdmap e95: 12 total, 12 up, 12 in 2024-06-06T04:27:36.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:35 smithi086 ceph-mon[36053]: pgmap v309: 129 pgs: 10 creating+peering, 6 unknown, 113 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:36.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:35 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2329965930' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-06T04:27:36.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:35 smithi086 ceph-mon[36053]: osdmap e95: 12 total, 12 up, 12 in 2024-06-06T04:27:37.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.439+0000 7f6e57389700 1 -- 172.21.15.45:0/3515552726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e50102770 msgr2=0x7f6e50102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:37.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.439+0000 7f6e57389700 1 --2- 172.21.15.45:0/3515552726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e50102770 0x7f6e50102b50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0099c0 tx=0x7f6e4c009cd0 comp rx=0 tx=0).stop 2024-06-06T04:27:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.439+0000 7f6e57389700 1 -- 172.21.15.45:0/3515552726 shutdown_connections 2024-06-06T04:27:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.439+0000 7f6e57389700 1 --2- 172.21.15.45:0/3515552726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e50107860 0x7f6e5010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.439+0000 7f6e57389700 1 --2- 172.21.15.45:0/3515552726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e50102770 0x7f6e50102b50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.439+0000 7f6e57389700 1 --2- 172.21.15.45:0/3515552726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e50103120 0x7f6e50107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.439+0000 7f6e57389700 1 -- 172.21.15.45:0/3515552726 >> 172.21.15.45:0/3515552726 conn(0x7f6e50079600 msgr2=0x7f6e50079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.439+0000 7f6e57389700 1 -- 172.21.15.45:0/3515552726 shutdown_connections 2024-06-06T04:27:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.439+0000 7f6e57389700 1 -- 172.21.15.45:0/3515552726 wait complete. 2024-06-06T04:27:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.439+0000 7f6e57389700 1 Processor -- start 2024-06-06T04:27:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.439+0000 7f6e57389700 1 -- start start 2024-06-06T04:27:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e57389700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e50102770 0x7f6e5019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e57389700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e50103120 0x7f6e5019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e57389700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e50107860 0x7f6e50198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:37.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e57389700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e50112630 con 0x7f6e50103120 2024-06-06T04:27:37.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e57389700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e501124b0 con 0x7f6e50107860 2024-06-06T04:27:37.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e57389700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6e501127b0 con 0x7f6e50102770 2024-06-06T04:27:37.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e54924700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e50103120 0x7f6e5019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:37.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e54924700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e50103120 0x7f6e5019ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46792/0 (socket says 172.21.15.45:46792) 2024-06-06T04:27:37.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e54924700 1 -- 172.21.15.45:0/4200566924 learned_addr learned my addr 172.21.15.45:0/4200566924 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:37.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e55926700 1 --2- 172.21.15.45:0/4200566924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e50107860 0x7f6e50198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:37.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e55125700 1 --2- 172.21.15.45:0/4200566924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e50102770 0x7f6e5019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:37.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e54924700 1 -- 172.21.15.45:0/4200566924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e50102770 msgr2=0x7f6e5019e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:37.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e54924700 1 --2- 172.21.15.45:0/4200566924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e50102770 0x7f6e5019e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:37.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e54924700 1 -- 172.21.15.45:0/4200566924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e50107860 msgr2=0x7f6e50198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:37.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e54924700 1 --2- 172.21.15.45:0/4200566924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e50107860 0x7f6e50198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:37.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e54924700 1 -- 172.21.15.45:0/4200566924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e50198fd0 con 0x7f6e50103120 2024-06-06T04:27:37.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e55125700 1 --2- 172.21.15.45:0/4200566924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e50102770 0x7f6e5019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:27:37.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.440+0000 7f6e54924700 1 --2- 172.21.15.45:0/4200566924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e50103120 0x7f6e5019ebf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000c9b0 tx=0x7f6e4000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:37.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.441+0000 7f6e467fc700 1 -- 172.21.15.45:0/4200566924 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40007b00 con 0x7f6e50103120 2024-06-06T04:27:37.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.441+0000 7f6e57389700 1 -- 172.21.15.45:0/4200566924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e501992c0 con 0x7f6e50103120 2024-06-06T04:27:37.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.441+0000 7f6e467fc700 1 -- 172.21.15.45:0/4200566924 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e40007ca0 con 0x7f6e50103120 2024-06-06T04:27:37.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.441+0000 7f6e467fc700 1 -- 172.21.15.45:0/4200566924 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40018660 con 0x7f6e50103120 2024-06-06T04:27:37.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.441+0000 7f6e57389700 1 -- 172.21.15.45:0/4200566924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e501a5580 con 0x7f6e50103120 2024-06-06T04:27:37.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.442+0000 7f6e57389700 1 -- 172.21.15.45:0/4200566924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e5010aaa0 con 0x7f6e50103120 2024-06-06T04:27:37.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.443+0000 7f6e467fc700 1 -- 172.21.15.45:0/4200566924 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6e4000f450 con 0x7f6e50103120 2024-06-06T04:27:37.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.444+0000 7f6e467fc700 1 --2- 172.21.15.45:0/4200566924 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6e3c0a2c20 0x7f6e3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:37.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.444+0000 7f6e55125700 1 --2- 172.21.15.45:0/4200566924 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6e3c0a2c20 0x7f6e3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:37.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.444+0000 7f6e467fc700 1 -- 172.21.15.45:0/4200566924 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8770+0+0 (secure 0 0 0) 0x7f6e400cf500 con 0x7f6e50103120 2024-06-06T04:27:37.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.444+0000 7f6e55125700 1 --2- 172.21.15.45:0/4200566924 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6e3c0a2c20 0x7f6e3c0a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c000c00 tx=0x7f6e4c005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:37.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.448+0000 7f6e467fc700 1 -- 172.21.15.45:0/4200566924 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6e40014030 con 0x7f6e50103120 2024-06-06T04:27:37.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:37.703+0000 7f6e57389700 1 -- 172.21.15.45:0/4200566924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f6e50069180 con 0x7f6e50103120 2024-06-06T04:27:37.704 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:37 smithi045 ceph-mon[32299]: pgmap v311: 129 pgs: 10 creating+peering, 119 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:37.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:37 smithi154 ceph-mon[36428]: pgmap v311: 129 pgs: 10 creating+peering, 119 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:38.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:37 smithi086 ceph-mon[36053]: pgmap v311: 129 pgs: 10 creating+peering, 119 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:38.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:38.623+0000 7f6e467fc700 1 -- 172.21.15.45:0/4200566924 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f6e4009c270 con 0x7f6e50103120 2024-06-06T04:27:38.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:38.626+0000 7f6e57389700 1 -- 172.21.15.45:0/4200566924 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6e3c0a2c20 msgr2=0x7f6e3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:38.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:38.626+0000 7f6e57389700 1 --2- 172.21.15.45:0/4200566924 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6e3c0a2c20 0x7f6e3c0a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c000c00 tx=0x7f6e4c005940 comp rx=0 tx=0).stop 2024-06-06T04:27:38.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:38.626+0000 7f6e57389700 1 -- 172.21.15.45:0/4200566924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e50103120 msgr2=0x7f6e5019ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:38.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:38.626+0000 7f6e57389700 1 --2- 172.21.15.45:0/4200566924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e50103120 0x7f6e5019ebf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000c9b0 tx=0x7f6e4000ccc0 comp rx=0 tx=0).stop 2024-06-06T04:27:38.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:38.626+0000 7f6e57389700 1 -- 172.21.15.45:0/4200566924 shutdown_connections 2024-06-06T04:27:38.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:38.626+0000 7f6e57389700 1 --2- 172.21.15.45:0/4200566924 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6e3c0a2c20 0x7f6e3c0a50e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:38.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:38.626+0000 7f6e57389700 1 --2- 172.21.15.45:0/4200566924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e50107860 0x7f6e50198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:38.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:38.626+0000 7f6e57389700 1 --2- 172.21.15.45:0/4200566924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e50102770 0x7f6e5019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:38.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:38.626+0000 7f6e57389700 1 --2- 172.21.15.45:0/4200566924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e50103120 0x7f6e5019ebf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:38.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:38.626+0000 7f6e57389700 1 -- 172.21.15.45:0/4200566924 >> 172.21.15.45:0/4200566924 conn(0x7f6e50079600 msgr2=0x7f6e50101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:38.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:38.626+0000 7f6e57389700 1 -- 172.21.15.45:0/4200566924 shutdown_connections 2024-06-06T04:27:38.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:38.626+0000 7f6e57389700 1 -- 172.21.15.45:0/4200566924 wait complete. 2024-06-06T04:27:38.628 INFO:teuthology.orchestra.run.smithi045.stderr:added data pool 4 to fsmap 2024-06-06T04:27:38.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:38 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/4200566924' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-06T04:27:38.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:38 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/4200566924' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-06T04:27:39.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:38 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/4200566924' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-06T04:27:39.420 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:27:39.676 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:39.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:39 smithi045 ceph-mon[32299]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:39.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:39 smithi045 ceph-mon[32299]: osdmap e96: 12 total, 12 up, 12 in 2024-06-06T04:27:39.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:39 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/4200566924' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-06T04:27:39.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:39 smithi045 ceph-mon[32299]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T04:27:39.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:39 smithi154 ceph-mon[36428]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:39.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:39 smithi154 ceph-mon[36428]: osdmap e96: 12 total, 12 up, 12 in 2024-06-06T04:27:39.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:39 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/4200566924' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-06T04:27:39.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:39 smithi154 ceph-mon[36428]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T04:27:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:39 smithi086 ceph-mon[36053]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:39 smithi086 ceph-mon[36053]: osdmap e96: 12 total, 12 up, 12 in 2024-06-06T04:27:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:39 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/4200566924' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-06T04:27:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:39 smithi086 ceph-mon[36053]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T04:27:40.753 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:40 smithi045 ceph-mon[32299]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-06T04:27:40.782 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:40 smithi045 ceph-mon[32299]: Cluster is now healthy 2024-06-06T04:27:40.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:40 smithi154 ceph-mon[36428]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-06T04:27:40.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:40 smithi154 ceph-mon[36428]: Cluster is now healthy 2024-06-06T04:27:41.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:40 smithi086 ceph-mon[36053]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-06T04:27:41.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:40 smithi086 ceph-mon[36053]: Cluster is now healthy 2024-06-06T04:27:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.330+0000 7f80cfea4700 1 -- 172.21.15.45:0/2566476948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c8102030 msgr2=0x7f80c810e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.330+0000 7f80cfea4700 1 --2- 172.21.15.45:0/2566476948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c8102030 0x7f80c810e220 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009990 tx=0x7f80c0009ca0 comp rx=0 tx=0).stop 2024-06-06T04:27:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.330+0000 7f80cfea4700 1 -- 172.21.15.45:0/2566476948 shutdown_connections 2024-06-06T04:27:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.330+0000 7f80cfea4700 1 --2- 172.21.15.45:0/2566476948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c8102030 0x7f80c810e220 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.330+0000 7f80cfea4700 1 --2- 172.21.15.45:0/2566476948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80c81074e0 0x7f80c81078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.330+0000 7f80cfea4700 1 --2- 172.21.15.45:0/2566476948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c8101670 0x7f80c8101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.330+0000 7f80cfea4700 1 -- 172.21.15.45:0/2566476948 >> 172.21.15.45:0/2566476948 conn(0x7f80c8078280 msgr2=0x7f80c8078690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:41.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.330+0000 7f80cfea4700 1 -- 172.21.15.45:0/2566476948 shutdown_connections 2024-06-06T04:27:41.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.330+0000 7f80cfea4700 1 -- 172.21.15.45:0/2566476948 wait complete. 2024-06-06T04:27:41.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.331+0000 7f80cfea4700 1 Processor -- start 2024-06-06T04:27:41.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.331+0000 7f80cfea4700 1 -- start start 2024-06-06T04:27:41.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.331+0000 7f80cfea4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80c8101670 0x7f80c81030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:41.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.331+0000 7f80cfea4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c8102030 0x7f80c8103600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:41.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.331+0000 7f80cfea4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c81074e0 0x7f80c8103b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:41.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.331+0000 7f80cfea4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f80c8111630 con 0x7f80c81074e0 2024-06-06T04:27:41.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.331+0000 7f80cfea4700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81114b0 con 0x7f80c8102030 2024-06-06T04:27:41.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.331+0000 7f80cfea4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81117b0 con 0x7f80c8101670 2024-06-06T04:27:41.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.331+0000 7f80ce441700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c81074e0 0x7f80c8103b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.331+0000 7f80ce441700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c81074e0 0x7f80c8103b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42806/0 (socket says 172.21.15.45:42806) 2024-06-06T04:27:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.331+0000 7f80ce441700 1 -- 172.21.15.45:0/1955129855 learned_addr learned my addr 172.21.15.45:0/1955129855 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.331+0000 7f80cdc40700 1 --2- 172.21.15.45:0/1955129855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80c8101670 0x7f80c81030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.332+0000 7f80cd43f700 1 --2- 172.21.15.45:0/1955129855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c8102030 0x7f80c8103600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.332+0000 7f80ce441700 1 -- 172.21.15.45:0/1955129855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80c8101670 msgr2=0x7f80c81030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.332+0000 7f80ce441700 1 --2- 172.21.15.45:0/1955129855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80c8101670 0x7f80c81030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.332+0000 7f80ce441700 1 -- 172.21.15.45:0/1955129855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c8102030 msgr2=0x7f80c8103600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.332+0000 7f80ce441700 1 --2- 172.21.15.45:0/1955129855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c8102030 0x7f80c8103600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.332+0000 7f80ce441700 1 -- 172.21.15.45:0/1955129855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c81a70a0 con 0x7f80c81074e0 2024-06-06T04:27:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.332+0000 7f80cd43f700 1 --2- 172.21.15.45:0/1955129855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c8102030 0x7f80c8103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:27:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.332+0000 7f80ce441700 1 --2- 172.21.15.45:0/1955129855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c81074e0 0x7f80c8103b40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f80c0005910 tx=0x7f80c00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.332+0000 7f80beffd700 1 -- 172.21.15.45:0/1955129855 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0027070 con 0x7f80c81074e0 2024-06-06T04:27:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.332+0000 7f80beffd700 1 -- 172.21.15.45:0/1955129855 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80c0003f10 con 0x7f80c81074e0 2024-06-06T04:27:41.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.332+0000 7f80cfea4700 1 -- 172.21.15.45:0/1955129855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c81a7330 con 0x7f80c81074e0 2024-06-06T04:27:41.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.332+0000 7f80beffd700 1 -- 172.21.15.45:0/1955129855 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0022bb0 con 0x7f80c81074e0 2024-06-06T04:27:41.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.332+0000 7f80cfea4700 1 -- 172.21.15.45:0/1955129855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c81a77e0 con 0x7f80c81074e0 2024-06-06T04:27:41.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.333+0000 7f80cfea4700 1 -- 172.21.15.45:0/1955129855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c8102b00 con 0x7f80c81074e0 2024-06-06T04:27:41.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.334+0000 7f80beffd700 1 -- 172.21.15.45:0/1955129855 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f80c002a070 con 0x7f80c81074e0 2024-06-06T04:27:41.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.335+0000 7f80beffd700 1 --2- 172.21.15.45:0/1955129855 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f80b40a2ca0 0x7f80b40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:41.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.335+0000 7f80cdc40700 1 --2- 172.21.15.45:0/1955129855 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f80b40a2ca0 0x7f80b40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:41.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.335+0000 7f80beffd700 1 -- 172.21.15.45:0/1955129855 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f80c00d07b0 con 0x7f80c81074e0 2024-06-06T04:27:41.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.336+0000 7f80cdc40700 1 --2- 172.21.15.45:0/1955129855 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f80b40a2ca0 0x7f80b40a5160 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f80c400a400 tx=0x7f80c400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:41.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.339+0000 7f80beffd700 1 -- 172.21.15.45:0/1955129855 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f80c0031070 con 0x7f80c81074e0 2024-06-06T04:27:41.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.593+0000 7f80cfea4700 1 -- 172.21.15.45:0/1955129855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80c8069180 con 0x7f80c81074e0 2024-06-06T04:27:41.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.594+0000 7f80beffd700 1 -- 172.21.15.45:0/1955129855 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11785 (secure 0 0 0) 0x7f80c8069180 con 0x7f80c81074e0 2024-06-06T04:27:41.595 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:27:41.595 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-06T04:27:24.462897+0000","modified":"2024-06-06T04:27:37.705153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24473,"qdb_cluster":[24473]},"id":1}]} 2024-06-06T04:27:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.596+0000 7f80cfea4700 1 -- 172.21.15.45:0/1955129855 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f80b40a2ca0 msgr2=0x7f80b40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:41.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.596+0000 7f80cfea4700 1 --2- 172.21.15.45:0/1955129855 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f80b40a2ca0 0x7f80b40a5160 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f80c400a400 tx=0x7f80c400d040 comp rx=0 tx=0).stop 2024-06-06T04:27:41.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.596+0000 7f80cfea4700 1 -- 172.21.15.45:0/1955129855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c81074e0 msgr2=0x7f80c8103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:41.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.596+0000 7f80cfea4700 1 --2- 172.21.15.45:0/1955129855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c81074e0 0x7f80c8103b40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f80c0005910 tx=0x7f80c00047b0 comp rx=0 tx=0).stop 2024-06-06T04:27:41.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.596+0000 7f80cfea4700 1 -- 172.21.15.45:0/1955129855 shutdown_connections 2024-06-06T04:27:41.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.596+0000 7f80cfea4700 1 --2- 172.21.15.45:0/1955129855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80c81074e0 0x7f80c8103b40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:41.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.596+0000 7f80cfea4700 1 --2- 172.21.15.45:0/1955129855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80c8101670 0x7f80c81030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:41.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.596+0000 7f80cfea4700 1 --2- 172.21.15.45:0/1955129855 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f80b40a2ca0 0x7f80b40a5160 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:41.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.596+0000 7f80cfea4700 1 --2- 172.21.15.45:0/1955129855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80c8102030 0x7f80c8103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:41.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.596+0000 7f80cfea4700 1 -- 172.21.15.45:0/1955129855 >> 172.21.15.45:0/1955129855 conn(0x7f80c8078280 msgr2=0x7f80c8100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:41.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.596+0000 7f80cfea4700 1 -- 172.21.15.45:0/1955129855 shutdown_connections 2024-06-06T04:27:41.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:41.596+0000 7f80cfea4700 1 -- 172.21.15.45:0/1955129855 wait complete. 2024-06-06T04:27:41.598 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-06-06T04:27:41.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:41 smithi154 ceph-mon[36428]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:41.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:41 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1955129855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:27:41.960 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:41 smithi045 ceph-mon[32299]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:41.960 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:41 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1955129855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:27:42.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:41 smithi086 ceph-mon[36053]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:42.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:41 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1955129855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:27:42.339 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T04:27:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:43.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:43 smithi045 ceph-mon[32299]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:43.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:43 smithi154 ceph-mon[36428]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:44.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:43 smithi086 ceph-mon[36053]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:44.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.152+0000 7f7e00f2e700 1 -- 172.21.15.45:0/3866142401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dfc103120 msgr2=0x7f7dfc107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:44.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.152+0000 7f7e00f2e700 1 --2- 172.21.15.45:0/3866142401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dfc103120 0x7f7dfc107130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7df00099c0 tx=0x7f7df0009cd0 comp rx=0 tx=0).stop 2024-06-06T04:27:44.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.152+0000 7f7e00f2e700 1 -- 172.21.15.45:0/3866142401 shutdown_connections 2024-06-06T04:27:44.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.152+0000 7f7e00f2e700 1 --2- 172.21.15.45:0/3866142401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dfc107860 0x7f7dfc10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:44.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.152+0000 7f7e00f2e700 1 --2- 172.21.15.45:0/3866142401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7dfc102770 0x7f7dfc102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:44.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.152+0000 7f7e00f2e700 1 --2- 172.21.15.45:0/3866142401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dfc103120 0x7f7dfc107130 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:44.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.152+0000 7f7e00f2e700 1 -- 172.21.15.45:0/3866142401 >> 172.21.15.45:0/3866142401 conn(0x7f7dfc079600 msgr2=0x7f7dfc079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:44.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.152+0000 7f7e00f2e700 1 -- 172.21.15.45:0/3866142401 shutdown_connections 2024-06-06T04:27:44.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.152+0000 7f7e00f2e700 1 -- 172.21.15.45:0/3866142401 wait complete. 2024-06-06T04:27:44.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.153+0000 7f7e00f2e700 1 Processor -- start 2024-06-06T04:27:44.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.153+0000 7f7e00f2e700 1 -- start start 2024-06-06T04:27:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.153+0000 7f7e00f2e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dfc102770 0x7f7dfc19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.153+0000 7f7e00f2e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dfc103120 0x7f7dfc19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.153+0000 7f7e00f2e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7dfc107860 0x7f7dfc198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.153+0000 7f7e00f2e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc112640 con 0x7f7dfc102770 2024-06-06T04:27:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.153+0000 7f7e00f2e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1124c0 con 0x7f7dfc103120 2024-06-06T04:27:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.153+0000 7f7e00f2e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1127c0 con 0x7f7dfc107860 2024-06-06T04:27:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.153+0000 7f7dfa59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dfc102770 0x7f7dfc19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.153+0000 7f7dfa59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dfc102770 0x7f7dfc19e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42828/0 (socket says 172.21.15.45:42828) 2024-06-06T04:27:44.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.154+0000 7f7dfa59c700 1 -- 172.21.15.45:0/628500583 learned_addr learned my addr 172.21.15.45:0/628500583 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:44.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.154+0000 7f7df9d9b700 1 --2- 172.21.15.45:0/628500583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dfc103120 0x7f7dfc19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:44.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.154+0000 7f7dfad9d700 1 --2- 172.21.15.45:0/628500583 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7dfc107860 0x7f7dfc198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:44.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.154+0000 7f7dfa59c700 1 -- 172.21.15.45:0/628500583 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7dfc107860 msgr2=0x7f7dfc198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:44.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.154+0000 7f7dfa59c700 1 --2- 172.21.15.45:0/628500583 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7dfc107860 0x7f7dfc198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:44.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.154+0000 7f7dfa59c700 1 -- 172.21.15.45:0/628500583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dfc103120 msgr2=0x7f7dfc19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:44.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.154+0000 7f7dfa59c700 1 --2- 172.21.15.45:0/628500583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dfc103120 0x7f7dfc19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:44.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.154+0000 7f7dfa59c700 1 -- 172.21.15.45:0/628500583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dfc199080 con 0x7f7dfc102770 2024-06-06T04:27:44.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.154+0000 7f7dfad9d700 1 --2- 172.21.15.45:0/628500583 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7dfc107860 0x7f7dfc198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:27:44.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.154+0000 7f7df9d9b700 1 --2- 172.21.15.45:0/628500583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dfc103120 0x7f7dfc19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:27:44.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.154+0000 7f7dfa59c700 1 --2- 172.21.15.45:0/628500583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dfc102770 0x7f7dfc19e760 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00d980 tx=0x7f7dec00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:44.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.154+0000 7f7deb7fe700 1 -- 172.21.15.45:0/628500583 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec017070 con 0x7f7dfc102770 2024-06-06T04:27:44.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.154+0000 7f7deb7fe700 1 -- 172.21.15.45:0/628500583 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dec004830 con 0x7f7dfc102770 2024-06-06T04:27:44.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.154+0000 7f7e00f2e700 1 -- 172.21.15.45:0/628500583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dfc199370 con 0x7f7dfc102770 2024-06-06T04:27:44.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.154+0000 7f7deb7fe700 1 -- 172.21.15.45:0/628500583 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec005440 con 0x7f7dfc102770 2024-06-06T04:27:44.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.154+0000 7f7e00f2e700 1 -- 172.21.15.45:0/628500583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dfc1a5630 con 0x7f7dfc102770 2024-06-06T04:27:44.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.155+0000 7f7e00f2e700 1 -- 172.21.15.45:0/628500583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dfc069180 con 0x7f7dfc102770 2024-06-06T04:27:44.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.157+0000 7f7deb7fe700 1 -- 172.21.15.45:0/628500583 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7dec0055e0 con 0x7f7dfc102770 2024-06-06T04:27:44.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.157+0000 7f7deb7fe700 1 --2- 172.21.15.45:0/628500583 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7de40a2ca0 0x7f7de40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:44.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.157+0000 7f7df9d9b700 1 --2- 172.21.15.45:0/628500583 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7de40a2ca0 0x7f7de40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:44.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.157+0000 7f7deb7fe700 1 -- 172.21.15.45:0/628500583 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f7dec0cf1b0 con 0x7f7dfc102770 2024-06-06T04:27:44.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.158+0000 7f7df9d9b700 1 --2- 172.21.15.45:0/628500583 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7de40a2ca0 0x7f7de40a5160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc1073a0 tx=0x7f7df0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:44.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.162+0000 7f7deb7fe700 1 -- 172.21.15.45:0/628500583 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7dec020020 con 0x7f7dfc102770 2024-06-06T04:27:44.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.368+0000 7f7e00f2e700 1 -- 172.21.15.45:0/628500583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7dfc02dca0 con 0x7f7dfc102770 2024-06-06T04:27:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.370+0000 7f7deb7fe700 1 -- 172.21.15.45:0/628500583 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24103 (secure 0 0 0) 0x7f7dec09bed0 con 0x7f7dfc102770 2024-06-06T04:27:44.371 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:27:44.371 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":96,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","created":"2024-06-06T04:19:02.626916+0000","modified":"2024-06-06T04:27:37.705006+0000","last_up_change":"2024-06-06T04:26:17.390429+0000","last_in_change":"2024-06-06T04:25:56.938388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T04:22:28.797137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T04:27:17.750625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T04:27:21.486917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T04:27:31.029818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d3cee4a-5025-45b3-b217-082af37e79db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6803","nonce":2111759250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6805","nonce":2111759250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6809","nonce":2111759250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6807","nonce":2111759250}]},"public_addr":"172.21.15.45:6803/2111759250","cluster_addr":"172.21.15.45:6805/2111759250","heartbeat_back_addr":"172.21.15.45:6809/2111759250","heartbeat_front_addr":"172.21.15.45:6807/2111759250","state":["exists","up"]},{"osd":1,"uuid":"75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6801","nonce":3732225287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6803","nonce":3732225287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6807","nonce":3732225287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6805","nonce":3732225287}]},"public_addr":"172.21.15.86:6801/3732225287","cluster_addr":"172.21.15.86:6803/3732225287","heartbeat_back_addr":"172.21.15.86:6807/3732225287","heartbeat_front_addr":"172.21.15.86:6805/3732225287","state":["exists","up"]},{"osd":2,"uuid":"bb20565f-f9ae-47e3-9604-c69f737cff4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6801","nonce":2898783039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6803","nonce":2898783039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6807","nonce":2898783039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6805","nonce":2898783039}]},"public_addr":"172.21.15.154:6801/2898783039","cluster_addr":"172.21.15.154:6803/2898783039","heartbeat_back_addr":"172.21.15.154:6807/2898783039","heartbeat_front_addr":"172.21.15.154:6805/2898783039","state":["exists","up"]},{"osd":3,"uuid":"e1160799-7811-4326-9792-7e6c7d189c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6811","nonce":3436753135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6813","nonce":3436753135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6817","nonce":3436753135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6815","nonce":3436753135}]},"public_addr":"172.21.15.45:6811/3436753135","cluster_addr":"172.21.15.45:6813/3436753135","heartbeat_back_addr":"172.21.15.45:6817/3436753135","heartbeat_front_addr":"172.21.15.45:6815/3436753135","state":["exists","up"]},{"osd":4,"uuid":"6b389bc8-e19a-47f3-83ff-7b782d146d0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6809","nonce":3030235796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6811","nonce":3030235796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6815","nonce":3030235796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6813","nonce":3030235796}]},"public_addr":"172.21.15.86:6809/3030235796","cluster_addr":"172.21.15.86:6811/3030235796","heartbeat_back_addr":"172.21.15.86:6815/3030235796","heartbeat_front_addr":"172.21.15.86:6813/3030235796","state":["exists","up"]},{"osd":5,"uuid":"069de7b2-89bf-4150-bed7-7b8832573dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6809","nonce":2222268646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6811","nonce":2222268646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6815","nonce":2222268646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6813","nonce":2222268646}]},"public_addr":"172.21.15.154:6809/2222268646","cluster_addr":"172.21.15.154:6811/2222268646","heartbeat_back_addr":"172.21.15.154:6815/2222268646","heartbeat_front_addr":"172.21.15.154:6813/2222268646","state":["exists","up"]},{"osd":6,"uuid":"4c70b627-eafa-4e27-ba2a-4d8e1e335962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6819","nonce":384734939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6821","nonce":384734939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6825","nonce":384734939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6823","nonce":384734939}]},"public_addr":"172.21.15.45:6819/384734939","cluster_addr":"172.21.15.45:6821/384734939","heartbeat_back_addr":"172.21.15.45:6825/384734939","heartbeat_front_addr":"172.21.15.45:6823/384734939","state":["exists","up"]},{"osd":7,"uuid":"3b09b875-be52-4be0-b116-7fd9b0c26705","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6817","nonce":1119791005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6819","nonce":1119791005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6823","nonce":1119791005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6821","nonce":1119791005}]},"public_addr":"172.21.15.86:6817/1119791005","cluster_addr":"172.21.15.86:6819/1119791005","heartbeat_back_addr":"172.21.15.86:6823/1119791005","heartbeat_front_addr":"172.21.15.86:6821/1119791005","state":["exists","up"]},{"osd":8,"uuid":"cdf1dd1b-5392-4836-8e65-4c7c5b688d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6817","nonce":3054812496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6819","nonce":3054812496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6823","nonce":3054812496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6821","nonce":3054812496}]},"public_addr":"172.21.15.154:6817/3054812496","cluster_addr":"172.21.15.154:6819/3054812496","heartbeat_back_addr":"172.21.15.154:6823/3054812496","heartbeat_front_addr":"172.21.15.154:6821/3054812496","state":["exists","up"]},{"osd":9,"uuid":"acede92d-2cc1-4967-a357-7e5e201e99d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6827","nonce":1360655510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6829","nonce":1360655510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6833","nonce":1360655510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6831","nonce":1360655510}]},"public_addr":"172.21.15.45:6827/1360655510","cluster_addr":"172.21.15.45:6829/1360655510","heartbeat_back_addr":"172.21.15.45:6833/1360655510","heartbeat_front_addr":"172.21.15.45:6831/1360655510","state":["exists","up"]},{"osd":10,"uuid":"a2d0c488-1892-4da4-adbf-952c22df0ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6825","nonce":1701691877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6827","nonce":1701691877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6831","nonce":1701691877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6829","nonce":1701691877}]},"public_addr":"172.21.15.86:6825/1701691877","cluster_addr":"172.21.15.86:6827/1701691877","heartbeat_back_addr":"172.21.15.86:6831/1701691877","heartbeat_front_addr":"172.21.15.86:6829/1701691877","state":["exists","up"]},{"osd":11,"uuid":"6b349c4c-b672-480b-ac6a-8594dab0eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6825","nonce":2065648007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6827","nonce":2065648007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6831","nonce":2065648007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6829","nonce":2065648007}]},"public_addr":"172.21.15.154:6825/2065648007","cluster_addr":"172.21.15.154:6827/2065648007","heartbeat_back_addr":"172.21.15.154:6831/2065648007","heartbeat_front_addr":"172.21.15.154:6829/2065648007","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:21:37.147394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:00.776617+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:24.786849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:50.487455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:14.869272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:39.815158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:05.972619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:31.410063+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:56.795119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:23.114884+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:49.256504+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:26:15.251967+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/36732849":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/1500318775":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6801/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6800/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/2041380391":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/2504003341":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/3798552599":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/498952822":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6800/1562511056":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6801/1562511056":"2024-06-07T04:19:37.026199+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T04:27:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.372+0000 7f7e00f2e700 1 -- 172.21.15.45:0/628500583 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7de40a2ca0 msgr2=0x7f7de40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.372+0000 7f7e00f2e700 1 --2- 172.21.15.45:0/628500583 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7de40a2ca0 0x7f7de40a5160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc1073a0 tx=0x7f7df0005fd0 comp rx=0 tx=0).stop 2024-06-06T04:27:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.372+0000 7f7e00f2e700 1 -- 172.21.15.45:0/628500583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dfc102770 msgr2=0x7f7dfc19e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.372+0000 7f7e00f2e700 1 --2- 172.21.15.45:0/628500583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dfc102770 0x7f7dfc19e760 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00d980 tx=0x7f7dec00dd40 comp rx=0 tx=0).stop 2024-06-06T04:27:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.372+0000 7f7e00f2e700 1 -- 172.21.15.45:0/628500583 shutdown_connections 2024-06-06T04:27:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.372+0000 7f7e00f2e700 1 --2- 172.21.15.45:0/628500583 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7dfc107860 0x7f7dfc198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.372+0000 7f7e00f2e700 1 --2- 172.21.15.45:0/628500583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dfc102770 0x7f7dfc19e760 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.372+0000 7f7e00f2e700 1 --2- 172.21.15.45:0/628500583 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7de40a2ca0 0x7f7de40a5160 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.372+0000 7f7e00f2e700 1 --2- 172.21.15.45:0/628500583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dfc103120 0x7f7dfc19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.372+0000 7f7e00f2e700 1 -- 172.21.15.45:0/628500583 >> 172.21.15.45:0/628500583 conn(0x7f7dfc079600 msgr2=0x7f7dfc100a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.372+0000 7f7e00f2e700 1 -- 172.21.15.45:0/628500583 shutdown_connections 2024-06-06T04:27:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:44.372+0000 7f7e00f2e700 1 -- 172.21.15.45:0/628500583 wait complete. 2024-06-06T04:27:44.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:44 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/628500583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:27:45.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:44 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/628500583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:27:45.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:44 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/628500583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:27:45.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T04:27:45.449 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:45.893 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:45 smithi045 ceph-mon[32299]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:46.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:45 smithi086 ceph-mon[36053]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:46.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:45 smithi154 ceph-mon[36428]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.199+0000 7f6761d55700 1 -- 172.21.15.45:0/410826925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f675c102740 msgr2=0x7f675c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.199+0000 7f6761d55700 1 --2- 172.21.15.45:0/410826925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f675c102740 0x7f675c102b20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f674c0099c0 tx=0x7f674c009cd0 comp rx=0 tx=0).stop 2024-06-06T04:27:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.199+0000 7f6761d55700 1 -- 172.21.15.45:0/410826925 shutdown_connections 2024-06-06T04:27:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.199+0000 7f6761d55700 1 --2- 172.21.15.45:0/410826925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f675c107840 0x7f675c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.199+0000 7f6761d55700 1 --2- 172.21.15.45:0/410826925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f675c102740 0x7f675c102b20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.199+0000 7f6761d55700 1 --2- 172.21.15.45:0/410826925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f675c1030f0 0x7f675c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.199+0000 7f6761d55700 1 -- 172.21.15.45:0/410826925 >> 172.21.15.45:0/410826925 conn(0x7f675c0795f0 msgr2=0x7f675c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.199+0000 7f6761d55700 1 -- 172.21.15.45:0/410826925 shutdown_connections 2024-06-06T04:27:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.199+0000 7f6761d55700 1 -- 172.21.15.45:0/410826925 wait complete. 2024-06-06T04:27:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.200+0000 7f6761d55700 1 Processor -- start 2024-06-06T04:27:47.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.200+0000 7f6761d55700 1 -- start start 2024-06-06T04:27:47.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f6761d55700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f675c102740 0x7f675c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:47.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f6761d55700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f675c1030f0 0x7f675c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:47.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f6761d55700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f675c107840 0x7f675c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:47.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f6761d55700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f675c112660 con 0x7f675c1030f0 2024-06-06T04:27:47.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f6761d55700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1124e0 con 0x7f675c102740 2024-06-06T04:27:47.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f6761d55700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1127e0 con 0x7f675c107840 2024-06-06T04:27:47.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f675affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f675c1030f0 0x7f675c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:47.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f675affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f675c1030f0 0x7f675c19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42840/0 (socket says 172.21.15.45:42840) 2024-06-06T04:27:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f675affd700 1 -- 172.21.15.45:0/2594483579 learned_addr learned my addr 172.21.15.45:0/2594483579 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f675b7fe700 1 --2- 172.21.15.45:0/2594483579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f675c102740 0x7f675c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f675bfff700 1 --2- 172.21.15.45:0/2594483579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f675c107840 0x7f675c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f675affd700 1 -- 172.21.15.45:0/2594483579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f675c107840 msgr2=0x7f675c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f675affd700 1 --2- 172.21.15.45:0/2594483579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f675c107840 0x7f675c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f675affd700 1 -- 172.21.15.45:0/2594483579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f675c102740 msgr2=0x7f675c19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f675affd700 1 --2- 172.21.15.45:0/2594483579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f675c102740 0x7f675c19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f675affd700 1 -- 172.21.15.45:0/2594483579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675c1990a0 con 0x7f675c1030f0 2024-06-06T04:27:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.201+0000 7f675affd700 1 --2- 172.21.15.45:0/2594483579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f675c1030f0 0x7f675c19ecc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f675000c920 tx=0x7f675000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.202+0000 7f6758ff9700 1 -- 172.21.15.45:0/2594483579 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750007b00 con 0x7f675c1030f0 2024-06-06T04:27:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.202+0000 7f6758ff9700 1 -- 172.21.15.45:0/2594483579 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6750004ce0 con 0x7f675c1030f0 2024-06-06T04:27:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.202+0000 7f6761d55700 1 -- 172.21.15.45:0/2594483579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675c199390 con 0x7f675c1030f0 2024-06-06T04:27:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.202+0000 7f6758ff9700 1 -- 172.21.15.45:0/2594483579 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750005550 con 0x7f675c1030f0 2024-06-06T04:27:47.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.202+0000 7f6761d55700 1 -- 172.21.15.45:0/2594483579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f675c1a5650 con 0x7f675c1030f0 2024-06-06T04:27:47.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.203+0000 7f6761d55700 1 -- 172.21.15.45:0/2594483579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f675c10aa80 con 0x7f675c1030f0 2024-06-06T04:27:47.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.204+0000 7f6758ff9700 1 -- 172.21.15.45:0/2594483579 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6750004750 con 0x7f675c1030f0 2024-06-06T04:27:47.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.204+0000 7f6758ff9700 1 --2- 172.21.15.45:0/2594483579 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f67440a2ca0 0x7f67440a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:47.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.205+0000 7f675b7fe700 1 --2- 172.21.15.45:0/2594483579 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f67440a2ca0 0x7f67440a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:47.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.205+0000 7f6758ff9700 1 -- 172.21.15.45:0/2594483579 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f67500cf100 con 0x7f675c1030f0 2024-06-06T04:27:47.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.205+0000 7f675b7fe700 1 --2- 172.21.15.45:0/2594483579 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f67440a2ca0 0x7f67440a5160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f674c00b580 tx=0x7f674c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.209+0000 7f6758ff9700 1 -- 172.21.15.45:0/2594483579 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6750014030 con 0x7f675c1030f0 2024-06-06T04:27:47.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.416+0000 7f6761d55700 1 -- 172.21.15.45:0/2594483579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f675c10a460 con 0x7f675c1030f0 2024-06-06T04:27:47.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.418+0000 7f6758ff9700 1 -- 172.21.15.45:0/2594483579 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24103 (secure 0 0 0) 0x7f675009be20 con 0x7f675c1030f0 2024-06-06T04:27:47.419 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:27:47.419 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":96,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","created":"2024-06-06T04:19:02.626916+0000","modified":"2024-06-06T04:27:37.705006+0000","last_up_change":"2024-06-06T04:26:17.390429+0000","last_in_change":"2024-06-06T04:25:56.938388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T04:22:28.797137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T04:27:17.750625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T04:27:21.486917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T04:27:31.029818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d3cee4a-5025-45b3-b217-082af37e79db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6803","nonce":2111759250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6805","nonce":2111759250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6809","nonce":2111759250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6807","nonce":2111759250}]},"public_addr":"172.21.15.45:6803/2111759250","cluster_addr":"172.21.15.45:6805/2111759250","heartbeat_back_addr":"172.21.15.45:6809/2111759250","heartbeat_front_addr":"172.21.15.45:6807/2111759250","state":["exists","up"]},{"osd":1,"uuid":"75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6801","nonce":3732225287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6803","nonce":3732225287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6807","nonce":3732225287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6805","nonce":3732225287}]},"public_addr":"172.21.15.86:6801/3732225287","cluster_addr":"172.21.15.86:6803/3732225287","heartbeat_back_addr":"172.21.15.86:6807/3732225287","heartbeat_front_addr":"172.21.15.86:6805/3732225287","state":["exists","up"]},{"osd":2,"uuid":"bb20565f-f9ae-47e3-9604-c69f737cff4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6801","nonce":2898783039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6803","nonce":2898783039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6807","nonce":2898783039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6805","nonce":2898783039}]},"public_addr":"172.21.15.154:6801/2898783039","cluster_addr":"172.21.15.154:6803/2898783039","heartbeat_back_addr":"172.21.15.154:6807/2898783039","heartbeat_front_addr":"172.21.15.154:6805/2898783039","state":["exists","up"]},{"osd":3,"uuid":"e1160799-7811-4326-9792-7e6c7d189c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6811","nonce":3436753135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6813","nonce":3436753135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6817","nonce":3436753135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6815","nonce":3436753135}]},"public_addr":"172.21.15.45:6811/3436753135","cluster_addr":"172.21.15.45:6813/3436753135","heartbeat_back_addr":"172.21.15.45:6817/3436753135","heartbeat_front_addr":"172.21.15.45:6815/3436753135","state":["exists","up"]},{"osd":4,"uuid":"6b389bc8-e19a-47f3-83ff-7b782d146d0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6809","nonce":3030235796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6811","nonce":3030235796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6815","nonce":3030235796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6813","nonce":3030235796}]},"public_addr":"172.21.15.86:6809/3030235796","cluster_addr":"172.21.15.86:6811/3030235796","heartbeat_back_addr":"172.21.15.86:6815/3030235796","heartbeat_front_addr":"172.21.15.86:6813/3030235796","state":["exists","up"]},{"osd":5,"uuid":"069de7b2-89bf-4150-bed7-7b8832573dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6809","nonce":2222268646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6811","nonce":2222268646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6815","nonce":2222268646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6813","nonce":2222268646}]},"public_addr":"172.21.15.154:6809/2222268646","cluster_addr":"172.21.15.154:6811/2222268646","heartbeat_back_addr":"172.21.15.154:6815/2222268646","heartbeat_front_addr":"172.21.15.154:6813/2222268646","state":["exists","up"]},{"osd":6,"uuid":"4c70b627-eafa-4e27-ba2a-4d8e1e335962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6819","nonce":384734939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6821","nonce":384734939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6825","nonce":384734939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6823","nonce":384734939}]},"public_addr":"172.21.15.45:6819/384734939","cluster_addr":"172.21.15.45:6821/384734939","heartbeat_back_addr":"172.21.15.45:6825/384734939","heartbeat_front_addr":"172.21.15.45:6823/384734939","state":["exists","up"]},{"osd":7,"uuid":"3b09b875-be52-4be0-b116-7fd9b0c26705","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6817","nonce":1119791005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6819","nonce":1119791005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6823","nonce":1119791005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6821","nonce":1119791005}]},"public_addr":"172.21.15.86:6817/1119791005","cluster_addr":"172.21.15.86:6819/1119791005","heartbeat_back_addr":"172.21.15.86:6823/1119791005","heartbeat_front_addr":"172.21.15.86:6821/1119791005","state":["exists","up"]},{"osd":8,"uuid":"cdf1dd1b-5392-4836-8e65-4c7c5b688d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6817","nonce":3054812496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6819","nonce":3054812496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6823","nonce":3054812496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6821","nonce":3054812496}]},"public_addr":"172.21.15.154:6817/3054812496","cluster_addr":"172.21.15.154:6819/3054812496","heartbeat_back_addr":"172.21.15.154:6823/3054812496","heartbeat_front_addr":"172.21.15.154:6821/3054812496","state":["exists","up"]},{"osd":9,"uuid":"acede92d-2cc1-4967-a357-7e5e201e99d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6827","nonce":1360655510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6829","nonce":1360655510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6833","nonce":1360655510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6831","nonce":1360655510}]},"public_addr":"172.21.15.45:6827/1360655510","cluster_addr":"172.21.15.45:6829/1360655510","heartbeat_back_addr":"172.21.15.45:6833/1360655510","heartbeat_front_addr":"172.21.15.45:6831/1360655510","state":["exists","up"]},{"osd":10,"uuid":"a2d0c488-1892-4da4-adbf-952c22df0ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6825","nonce":1701691877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6827","nonce":1701691877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6831","nonce":1701691877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6829","nonce":1701691877}]},"public_addr":"172.21.15.86:6825/1701691877","cluster_addr":"172.21.15.86:6827/1701691877","heartbeat_back_addr":"172.21.15.86:6831/1701691877","heartbeat_front_addr":"172.21.15.86:6829/1701691877","state":["exists","up"]},{"osd":11,"uuid":"6b349c4c-b672-480b-ac6a-8594dab0eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6825","nonce":2065648007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6827","nonce":2065648007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6831","nonce":2065648007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6829","nonce":2065648007}]},"public_addr":"172.21.15.154:6825/2065648007","cluster_addr":"172.21.15.154:6827/2065648007","heartbeat_back_addr":"172.21.15.154:6831/2065648007","heartbeat_front_addr":"172.21.15.154:6829/2065648007","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:21:37.147394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:00.776617+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:24.786849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:50.487455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:14.869272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:39.815158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:05.972619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:31.410063+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:56.795119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:23.114884+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:49.256504+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:26:15.251967+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/36732849":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/1500318775":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6801/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6800/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/2041380391":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/2504003341":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/3798552599":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/498952822":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6800/1562511056":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6801/1562511056":"2024-06-07T04:19:37.026199+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T04:27:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.420+0000 7f6761d55700 1 -- 172.21.15.45:0/2594483579 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f67440a2ca0 msgr2=0x7f67440a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.420+0000 7f6761d55700 1 --2- 172.21.15.45:0/2594483579 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f67440a2ca0 0x7f67440a5160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f674c00b580 tx=0x7f674c005fb0 comp rx=0 tx=0).stop 2024-06-06T04:27:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.420+0000 7f6761d55700 1 -- 172.21.15.45:0/2594483579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f675c1030f0 msgr2=0x7f675c19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.420+0000 7f6761d55700 1 --2- 172.21.15.45:0/2594483579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f675c1030f0 0x7f675c19ecc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f675000c920 tx=0x7f675000cc30 comp rx=0 tx=0).stop 2024-06-06T04:27:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.420+0000 7f6761d55700 1 -- 172.21.15.45:0/2594483579 shutdown_connections 2024-06-06T04:27:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.420+0000 7f6761d55700 1 --2- 172.21.15.45:0/2594483579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f675c107840 0x7f675c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.420+0000 7f6761d55700 1 --2- 172.21.15.45:0/2594483579 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f67440a2ca0 0x7f67440a5160 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.420+0000 7f6761d55700 1 --2- 172.21.15.45:0/2594483579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f675c102740 0x7f675c19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.420+0000 7f6761d55700 1 --2- 172.21.15.45:0/2594483579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f675c1030f0 0x7f675c19ecc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.420+0000 7f6761d55700 1 -- 172.21.15.45:0/2594483579 >> 172.21.15.45:0/2594483579 conn(0x7f675c0795f0 msgr2=0x7f675c100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.421+0000 7f6761d55700 1 -- 172.21.15.45:0/2594483579 shutdown_connections 2024-06-06T04:27:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:47.421+0000 7f6761d55700 1 -- 172.21.15.45:0/2594483579 wait complete. 2024-06-06T04:27:47.798 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:47 smithi045 ceph-mon[32299]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:47.798 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:47 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2594483579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:27:48.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:47 smithi086 ceph-mon[36053]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:48.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:47 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2594483579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:27:48.144 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:27:48.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:47 smithi154 ceph-mon[36428]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:48.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:47 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2594483579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:27:48.396 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:49.886 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:49 smithi045 ceph-mon[32299]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:50.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:49 smithi086 ceph-mon[36053]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.134+0000 7f370b3e8700 1 -- 172.21.15.45:0/508442999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37041073f0 msgr2=0x7f37041077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.134+0000 7f370b3e8700 1 --2- 172.21.15.45:0/508442999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37041073f0 0x7f37041077d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f370000b0a0 tx=0x7f370000b3b0 comp rx=0 tx=0).stop 2024-06-06T04:27:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.134+0000 7f370b3e8700 1 -- 172.21.15.45:0/508442999 shutdown_connections 2024-06-06T04:27:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.134+0000 7f370b3e8700 1 --2- 172.21.15.45:0/508442999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3704101d40 0x7f370410e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.134+0000 7f370b3e8700 1 --2- 172.21.15.45:0/508442999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37041073f0 0x7f37041077d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.134+0000 7f370b3e8700 1 --2- 172.21.15.45:0/508442999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3704101380 0x7f3704101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.134+0000 7f370b3e8700 1 -- 172.21.15.45:0/508442999 >> 172.21.15.45:0/508442999 conn(0x7f3704078300 msgr2=0x7f3704078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.134+0000 7f370b3e8700 1 -- 172.21.15.45:0/508442999 shutdown_connections 2024-06-06T04:27:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.134+0000 7f370b3e8700 1 -- 172.21.15.45:0/508442999 wait complete. 2024-06-06T04:27:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.135+0000 7f370b3e8700 1 Processor -- start 2024-06-06T04:27:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.135+0000 7f370b3e8700 1 -- start start 2024-06-06T04:27:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.135+0000 7f370b3e8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3704101380 0x7f3704103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.135+0000 7f370b3e8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3704101d40 0x7f37041035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.135+0000 7f370b3e8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37041073f0 0x7f3704103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.135+0000 7f370b3e8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3704111590 con 0x7f37041073f0 2024-06-06T04:27:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.135+0000 7f370b3e8700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3704111410 con 0x7f3704101380 2024-06-06T04:27:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.135+0000 7f370b3e8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3704111710 con 0x7f3704101d40 2024-06-06T04:27:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.135+0000 7f3709985700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37041073f0 0x7f3704103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.135+0000 7f3709985700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37041073f0 0x7f3704103ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35858/0 (socket says 172.21.15.45:35858) 2024-06-06T04:27:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.135+0000 7f3709985700 1 -- 172.21.15.45:0/2789962109 learned_addr learned my addr 172.21.15.45:0/2789962109 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.135+0000 7f3709184700 1 --2- 172.21.15.45:0/2789962109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3704101380 0x7f3704103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:50.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.135+0000 7f3708983700 1 --2- 172.21.15.45:0/2789962109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3704101d40 0x7f37041035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:50.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.136+0000 7f3709184700 1 -- 172.21.15.45:0/2789962109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3704101d40 msgr2=0x7f37041035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:50.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.136+0000 7f3709184700 1 --2- 172.21.15.45:0/2789962109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3704101d40 0x7f37041035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:50.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.136+0000 7f3709184700 1 -- 172.21.15.45:0/2789962109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37041073f0 msgr2=0x7f3704103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:50.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.136+0000 7f3709184700 1 --2- 172.21.15.45:0/2789962109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37041073f0 0x7f3704103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:50.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.136+0000 7f3709184700 1 -- 172.21.15.45:0/2789962109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37041a70b0 con 0x7f3704101380 2024-06-06T04:27:50.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.136+0000 7f3709985700 1 --2- 172.21.15.45:0/2789962109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37041073f0 0x7f3704103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:27:50.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.136+0000 7f3708983700 1 --2- 172.21.15.45:0/2789962109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3704101d40 0x7f37041035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:27:50.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.136+0000 7f3709184700 1 --2- 172.21.15.45:0/2789962109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3704101380 0x7f3704103060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3700000f80 tx=0x7f3700004ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:50.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.137+0000 7f36fa7fc700 1 -- 172.21.15.45:0/2789962109 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700021070 con 0x7f3704101380 2024-06-06T04:27:50.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.137+0000 7f36fa7fc700 1 -- 172.21.15.45:0/2789962109 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3700007b80 con 0x7f3704101380 2024-06-06T04:27:50.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.137+0000 7f370b3e8700 1 -- 172.21.15.45:0/2789962109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37041a7340 con 0x7f3704101380 2024-06-06T04:27:50.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.137+0000 7f36fa7fc700 1 -- 172.21.15.45:0/2789962109 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37000052a0 con 0x7f3704101380 2024-06-06T04:27:50.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.137+0000 7f370b3e8700 1 -- 172.21.15.45:0/2789962109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37041a7840 con 0x7f3704101380 2024-06-06T04:27:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.138+0000 7f370b3e8700 1 -- 172.21.15.45:0/2789962109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3704069180 con 0x7f3704101380 2024-06-06T04:27:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.139+0000 7f36fa7fc700 1 -- 172.21.15.45:0/2789962109 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3700024070 con 0x7f3704101380 2024-06-06T04:27:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.140+0000 7f36fa7fc700 1 --2- 172.21.15.45:0/2789962109 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f36f00a2ca0 0x7f36f00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.140+0000 7f3708983700 1 --2- 172.21.15.45:0/2789962109 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f36f00a2ca0 0x7f36f00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.140+0000 7f36fa7fc700 1 -- 172.21.15.45:0/2789962109 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f370000f020 con 0x7f3704101380 2024-06-06T04:27:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.140+0000 7f3708983700 1 --2- 172.21.15.45:0/2789962109 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f36f00a2ca0 0x7f36f00a5160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f37040ffae0 tx=0x7f36f400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:50.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.145+0000 7f36fa7fc700 1 -- 172.21.15.45:0/2789962109 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f370002b020 con 0x7f3704101380 2024-06-06T04:27:50.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:49 smithi154 ceph-mon[36428]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:50.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.401+0000 7f370b3e8700 1 -- 172.21.15.45:0/2789962109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37041a7b60 con 0x7f3704101380 2024-06-06T04:27:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.403+0000 7f36fa7fc700 1 -- 172.21.15.45:0/2789962109 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11785 (secure 0 0 0) 0x7f370009d550 con 0x7f3704101380 2024-06-06T04:27:50.403 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:27:50.403 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-06T04:27:24.462897+0000","modified":"2024-06-06T04:27:37.705153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24473,"qdb_cluster":[24473]},"id":1}]} 2024-06-06T04:27:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.405+0000 7f370b3e8700 1 -- 172.21.15.45:0/2789962109 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f36f00a2ca0 msgr2=0x7f36f00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.405+0000 7f370b3e8700 1 --2- 172.21.15.45:0/2789962109 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f36f00a2ca0 0x7f36f00a5160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f37040ffae0 tx=0x7f36f400b410 comp rx=0 tx=0).stop 2024-06-06T04:27:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.405+0000 7f370b3e8700 1 -- 172.21.15.45:0/2789962109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3704101380 msgr2=0x7f3704103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.405+0000 7f370b3e8700 1 --2- 172.21.15.45:0/2789962109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3704101380 0x7f3704103060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3700000f80 tx=0x7f3700004ce0 comp rx=0 tx=0).stop 2024-06-06T04:27:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.405+0000 7f370b3e8700 1 -- 172.21.15.45:0/2789962109 shutdown_connections 2024-06-06T04:27:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.405+0000 7f370b3e8700 1 --2- 172.21.15.45:0/2789962109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37041073f0 0x7f3704103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.405+0000 7f370b3e8700 1 --2- 172.21.15.45:0/2789962109 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f36f00a2ca0 0x7f36f00a5160 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.405+0000 7f370b3e8700 1 --2- 172.21.15.45:0/2789962109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3704101380 0x7f3704103060 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.405+0000 7f370b3e8700 1 --2- 172.21.15.45:0/2789962109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3704101d40 0x7f37041035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.405+0000 7f370b3e8700 1 -- 172.21.15.45:0/2789962109 >> 172.21.15.45:0/2789962109 conn(0x7f3704078300 msgr2=0x7f3704100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.406+0000 7f370b3e8700 1 -- 172.21.15.45:0/2789962109 shutdown_connections 2024-06-06T04:27:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:50.406+0000 7f370b3e8700 1 -- 172.21.15.45:0/2789962109 wait complete. 2024-06-06T04:27:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-06-06T04:27:50.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:50 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2789962109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:27:51.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:50 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2789962109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:27:51.178 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T04:27:51.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:50 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2789962109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:27:51.447 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:51.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:51 smithi045 ceph-mon[32299]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:52.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:51 smithi086 ceph-mon[36053]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:52.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:51 smithi154 ceph-mon[36428]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:53.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.226+0000 7fbb1ebc1700 1 -- 172.21.15.45:0/829211830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb18101620 msgr2=0x7fbb18101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:53.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.226+0000 7fbb1ebc1700 1 --2- 172.21.15.45:0/829211830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb18101620 0x7fbb18101a00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0099c0 tx=0x7fbb0c009cd0 comp rx=0 tx=0).stop 2024-06-06T04:27:53.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.226+0000 7fbb1ebc1700 1 -- 172.21.15.45:0/829211830 shutdown_connections 2024-06-06T04:27:53.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.226+0000 7fbb1ebc1700 1 --2- 172.21.15.45:0/829211830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb181066b0 0x7fbb1810e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:53.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.226+0000 7fbb1ebc1700 1 --2- 172.21.15.45:0/829211830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb18101620 0x7fbb18101a00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:53.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.226+0000 7fbb1ebc1700 1 --2- 172.21.15.45:0/829211830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb18101fd0 0x7fbb18105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:53.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.226+0000 7fbb1ebc1700 1 -- 172.21.15.45:0/829211830 >> 172.21.15.45:0/829211830 conn(0x7fbb18078340 msgr2=0x7fbb18078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:53.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.226+0000 7fbb1ebc1700 1 -- 172.21.15.45:0/829211830 shutdown_connections 2024-06-06T04:27:53.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.226+0000 7fbb1ebc1700 1 -- 172.21.15.45:0/829211830 wait complete. 2024-06-06T04:27:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb1ebc1700 1 Processor -- start 2024-06-06T04:27:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb1ebc1700 1 -- start start 2024-06-06T04:27:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb1ebc1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb18101620 0x7fbb1819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb1ebc1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb18101fd0 0x7fbb1819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb1ebc1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb181066b0 0x7fbb18198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb1ebc1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18111590 con 0x7fbb18101fd0 2024-06-06T04:27:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb1ebc1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18111410 con 0x7fbb18101620 2024-06-06T04:27:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb1ebc1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18111710 con 0x7fbb181066b0 2024-06-06T04:27:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb17fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb18101fd0 0x7fbb1819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb17fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb18101fd0 0x7fbb1819eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35882/0 (socket says 172.21.15.45:35882) 2024-06-06T04:27:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb17fff700 1 -- 172.21.15.45:0/3047343503 learned_addr learned my addr 172.21.15.45:0/3047343503 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb1c95d700 1 --2- 172.21.15.45:0/3047343503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb18101620 0x7fbb1819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb17fff700 1 -- 172.21.15.45:0/3047343503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb181066b0 msgr2=0x7fbb18198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb17fff700 1 --2- 172.21.15.45:0/3047343503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb181066b0 0x7fbb18198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb17fff700 1 -- 172.21.15.45:0/3047343503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb18101620 msgr2=0x7fbb1819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:53.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb17fff700 1 --2- 172.21.15.45:0/3047343503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb18101620 0x7fbb1819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:53.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.227+0000 7fbb17fff700 1 -- 172.21.15.45:0/3047343503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb18199180 con 0x7fbb18101fd0 2024-06-06T04:27:53.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.228+0000 7fbb17fff700 1 --2- 172.21.15.45:0/3047343503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb18101fd0 0x7fbb1819eda0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400d980 tx=0x7fbb0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:53.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.228+0000 7fbb15ffb700 1 -- 172.21.15.45:0/3047343503 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04017070 con 0x7fbb18101fd0 2024-06-06T04:27:53.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.228+0000 7fbb1ebc1700 1 -- 172.21.15.45:0/3047343503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb18199470 con 0x7fbb18101fd0 2024-06-06T04:27:53.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.228+0000 7fbb15ffb700 1 -- 172.21.15.45:0/3047343503 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb04009de0 con 0x7fbb18101fd0 2024-06-06T04:27:53.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.228+0000 7fbb15ffb700 1 -- 172.21.15.45:0/3047343503 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0400f3c0 con 0x7fbb18101fd0 2024-06-06T04:27:53.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.228+0000 7fbb1ebc1700 1 -- 172.21.15.45:0/3047343503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb18102fa0 con 0x7fbb18101fd0 2024-06-06T04:27:53.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.229+0000 7fbb1ebc1700 1 -- 172.21.15.45:0/3047343503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb18069180 con 0x7fbb18101fd0 2024-06-06T04:27:53.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.230+0000 7fbb15ffb700 1 -- 172.21.15.45:0/3047343503 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbb04009930 con 0x7fbb18101fd0 2024-06-06T04:27:53.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.231+0000 7fbb15ffb700 1 --2- 172.21.15.45:0/3047343503 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbb080a2ca0 0x7fbb080a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:53.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.231+0000 7fbb1c95d700 1 --2- 172.21.15.45:0/3047343503 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbb080a2ca0 0x7fbb080a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:53.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.231+0000 7fbb15ffb700 1 -- 172.21.15.45:0/3047343503 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7fbb040cf110 con 0x7fbb18101fd0 2024-06-06T04:27:53.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.232+0000 7fbb1c95d700 1 --2- 172.21.15.45:0/3047343503 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbb080a2ca0 0x7fbb080a5160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009990 tx=0x7fbb0c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:53.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.236+0000 7fbb15ffb700 1 -- 172.21.15.45:0/3047343503 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbb0409bdd0 con 0x7fbb18101fd0 2024-06-06T04:27:53.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.442+0000 7fbb1ebc1700 1 -- 172.21.15.45:0/3047343503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb1802dca0 con 0x7fbb18101fd0 2024-06-06T04:27:53.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.443+0000 7fbb15ffb700 1 -- 172.21.15.45:0/3047343503 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24103 (secure 0 0 0) 0x7fbb040860e0 con 0x7fbb18101fd0 2024-06-06T04:27:53.444 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:27:53.444 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":96,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","created":"2024-06-06T04:19:02.626916+0000","modified":"2024-06-06T04:27:37.705006+0000","last_up_change":"2024-06-06T04:26:17.390429+0000","last_in_change":"2024-06-06T04:25:56.938388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T04:22:28.797137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T04:27:17.750625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T04:27:21.486917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T04:27:31.029818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d3cee4a-5025-45b3-b217-082af37e79db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6803","nonce":2111759250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6805","nonce":2111759250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6809","nonce":2111759250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6807","nonce":2111759250}]},"public_addr":"172.21.15.45:6803/2111759250","cluster_addr":"172.21.15.45:6805/2111759250","heartbeat_back_addr":"172.21.15.45:6809/2111759250","heartbeat_front_addr":"172.21.15.45:6807/2111759250","state":["exists","up"]},{"osd":1,"uuid":"75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6801","nonce":3732225287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6803","nonce":3732225287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6807","nonce":3732225287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6805","nonce":3732225287}]},"public_addr":"172.21.15.86:6801/3732225287","cluster_addr":"172.21.15.86:6803/3732225287","heartbeat_back_addr":"172.21.15.86:6807/3732225287","heartbeat_front_addr":"172.21.15.86:6805/3732225287","state":["exists","up"]},{"osd":2,"uuid":"bb20565f-f9ae-47e3-9604-c69f737cff4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6801","nonce":2898783039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6803","nonce":2898783039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6807","nonce":2898783039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6805","nonce":2898783039}]},"public_addr":"172.21.15.154:6801/2898783039","cluster_addr":"172.21.15.154:6803/2898783039","heartbeat_back_addr":"172.21.15.154:6807/2898783039","heartbeat_front_addr":"172.21.15.154:6805/2898783039","state":["exists","up"]},{"osd":3,"uuid":"e1160799-7811-4326-9792-7e6c7d189c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6811","nonce":3436753135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6813","nonce":3436753135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6817","nonce":3436753135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6815","nonce":3436753135}]},"public_addr":"172.21.15.45:6811/3436753135","cluster_addr":"172.21.15.45:6813/3436753135","heartbeat_back_addr":"172.21.15.45:6817/3436753135","heartbeat_front_addr":"172.21.15.45:6815/3436753135","state":["exists","up"]},{"osd":4,"uuid":"6b389bc8-e19a-47f3-83ff-7b782d146d0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6809","nonce":3030235796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6811","nonce":3030235796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6815","nonce":3030235796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6813","nonce":3030235796}]},"public_addr":"172.21.15.86:6809/3030235796","cluster_addr":"172.21.15.86:6811/3030235796","heartbeat_back_addr":"172.21.15.86:6815/3030235796","heartbeat_front_addr":"172.21.15.86:6813/3030235796","state":["exists","up"]},{"osd":5,"uuid":"069de7b2-89bf-4150-bed7-7b8832573dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6809","nonce":2222268646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6811","nonce":2222268646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6815","nonce":2222268646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6813","nonce":2222268646}]},"public_addr":"172.21.15.154:6809/2222268646","cluster_addr":"172.21.15.154:6811/2222268646","heartbeat_back_addr":"172.21.15.154:6815/2222268646","heartbeat_front_addr":"172.21.15.154:6813/2222268646","state":["exists","up"]},{"osd":6,"uuid":"4c70b627-eafa-4e27-ba2a-4d8e1e335962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6819","nonce":384734939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6821","nonce":384734939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6825","nonce":384734939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6823","nonce":384734939}]},"public_addr":"172.21.15.45:6819/384734939","cluster_addr":"172.21.15.45:6821/384734939","heartbeat_back_addr":"172.21.15.45:6825/384734939","heartbeat_front_addr":"172.21.15.45:6823/384734939","state":["exists","up"]},{"osd":7,"uuid":"3b09b875-be52-4be0-b116-7fd9b0c26705","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6817","nonce":1119791005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6819","nonce":1119791005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6823","nonce":1119791005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6821","nonce":1119791005}]},"public_addr":"172.21.15.86:6817/1119791005","cluster_addr":"172.21.15.86:6819/1119791005","heartbeat_back_addr":"172.21.15.86:6823/1119791005","heartbeat_front_addr":"172.21.15.86:6821/1119791005","state":["exists","up"]},{"osd":8,"uuid":"cdf1dd1b-5392-4836-8e65-4c7c5b688d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6817","nonce":3054812496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6819","nonce":3054812496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6823","nonce":3054812496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6821","nonce":3054812496}]},"public_addr":"172.21.15.154:6817/3054812496","cluster_addr":"172.21.15.154:6819/3054812496","heartbeat_back_addr":"172.21.15.154:6823/3054812496","heartbeat_front_addr":"172.21.15.154:6821/3054812496","state":["exists","up"]},{"osd":9,"uuid":"acede92d-2cc1-4967-a357-7e5e201e99d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6827","nonce":1360655510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6829","nonce":1360655510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6833","nonce":1360655510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6831","nonce":1360655510}]},"public_addr":"172.21.15.45:6827/1360655510","cluster_addr":"172.21.15.45:6829/1360655510","heartbeat_back_addr":"172.21.15.45:6833/1360655510","heartbeat_front_addr":"172.21.15.45:6831/1360655510","state":["exists","up"]},{"osd":10,"uuid":"a2d0c488-1892-4da4-adbf-952c22df0ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6825","nonce":1701691877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6827","nonce":1701691877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6831","nonce":1701691877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6829","nonce":1701691877}]},"public_addr":"172.21.15.86:6825/1701691877","cluster_addr":"172.21.15.86:6827/1701691877","heartbeat_back_addr":"172.21.15.86:6831/1701691877","heartbeat_front_addr":"172.21.15.86:6829/1701691877","state":["exists","up"]},{"osd":11,"uuid":"6b349c4c-b672-480b-ac6a-8594dab0eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6825","nonce":2065648007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6827","nonce":2065648007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6831","nonce":2065648007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6829","nonce":2065648007}]},"public_addr":"172.21.15.154:6825/2065648007","cluster_addr":"172.21.15.154:6827/2065648007","heartbeat_back_addr":"172.21.15.154:6831/2065648007","heartbeat_front_addr":"172.21.15.154:6829/2065648007","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:21:37.147394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:00.776617+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:24.786849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:50.487455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:14.869272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:39.815158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:05.972619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:31.410063+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:56.795119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:23.114884+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:49.256504+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:26:15.251967+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/36732849":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/1500318775":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6801/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6800/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/2041380391":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/2504003341":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/3798552599":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/498952822":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6800/1562511056":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6801/1562511056":"2024-06-07T04:19:37.026199+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T04:27:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.445+0000 7fbb1ebc1700 1 -- 172.21.15.45:0/3047343503 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbb080a2ca0 msgr2=0x7fbb080a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.445+0000 7fbb1ebc1700 1 --2- 172.21.15.45:0/3047343503 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbb080a2ca0 0x7fbb080a5160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009990 tx=0x7fbb0c005fb0 comp rx=0 tx=0).stop 2024-06-06T04:27:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.445+0000 7fbb1ebc1700 1 -- 172.21.15.45:0/3047343503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb18101fd0 msgr2=0x7fbb1819eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.445+0000 7fbb1ebc1700 1 --2- 172.21.15.45:0/3047343503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb18101fd0 0x7fbb1819eda0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400d980 tx=0x7fbb0400dc90 comp rx=0 tx=0).stop 2024-06-06T04:27:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.445+0000 7fbb1ebc1700 1 -- 172.21.15.45:0/3047343503 shutdown_connections 2024-06-06T04:27:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.445+0000 7fbb1ebc1700 1 --2- 172.21.15.45:0/3047343503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb181066b0 0x7fbb18198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.445+0000 7fbb1ebc1700 1 --2- 172.21.15.45:0/3047343503 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbb080a2ca0 0x7fbb080a5160 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.445+0000 7fbb1ebc1700 1 --2- 172.21.15.45:0/3047343503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb18101620 0x7fbb1819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.445+0000 7fbb1ebc1700 1 --2- 172.21.15.45:0/3047343503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb18101fd0 0x7fbb1819eda0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.445+0000 7fbb1ebc1700 1 -- 172.21.15.45:0/3047343503 >> 172.21.15.45:0/3047343503 conn(0x7fbb18078340 msgr2=0x7fbb1806af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.446+0000 7fbb1ebc1700 1 -- 172.21.15.45:0/3047343503 shutdown_connections 2024-06-06T04:27:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:53.446+0000 7fbb1ebc1700 1 -- 172.21.15.45:0/3047343503 wait complete. 2024-06-06T04:27:53.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:53 smithi045 ceph-mon[32299]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:53.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:53 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3047343503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:27:54.081 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-06-06T04:27:24.462897+0000', 'modified': '2024-06-06T04:27:37.705153+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24473}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24473': {'gid': 24473, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.86:6839/816791193', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 816791193}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 816791193}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24473, 'qdb_cluster': [24473]} 2024-06-06T04:27:54.081 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-06-06T04:27:54.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-06T04:27:54.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:53 smithi086 ceph-mon[36053]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:54.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:53 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3047343503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:27:54.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:53 smithi154 ceph-mon[36428]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:54.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:53 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3047343503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:27:54.333 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:55.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.860+0000 7f47f8309700 1 -- 172.21.15.45:0/796107621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f006a170 msgr2=0x7f47f006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.860+0000 7f47f8309700 1 --2- 172.21.15.45:0/796107621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f006a170 0x7f47f006a5f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009990 tx=0x7f47e0009ca0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.861+0000 7f47f8309700 1 -- 172.21.15.45:0/796107621 shutdown_connections 2024-06-06T04:27:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.861+0000 7f47f8309700 1 --2- 172.21.15.45:0/796107621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f006ab30 0x7f47f010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.861+0000 7f47f8309700 1 --2- 172.21.15.45:0/796107621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47f0108500 0x7f47f01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.861+0000 7f47f8309700 1 --2- 172.21.15.45:0/796107621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f006a170 0x7f47f006a5f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.861+0000 7f47f8309700 1 -- 172.21.15.45:0/796107621 >> 172.21.15.45:0/796107621 conn(0x7f47f0078340 msgr2=0x7f47f0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.861+0000 7f47f8309700 1 -- 172.21.15.45:0/796107621 shutdown_connections 2024-06-06T04:27:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.861+0000 7f47f8309700 1 -- 172.21.15.45:0/796107621 wait complete. 2024-06-06T04:27:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.861+0000 7f47f8309700 1 Processor -- start 2024-06-06T04:27:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.861+0000 7f47f8309700 1 -- start start 2024-06-06T04:27:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f8309700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f006a170 0x7f47f019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f8309700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47f006ab30 0x7f47f019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f8309700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f0108500 0x7f47f0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f8309700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0112710 con 0x7f47f0108500 2024-06-06T04:27:55.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f8309700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0112590 con 0x7f47f006a170 2024-06-06T04:27:55.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f8309700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0112890 con 0x7f47f006ab30 2024-06-06T04:27:55.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f68a6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f0108500 0x7f47f0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:55.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f68a6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f0108500 0x7f47f0198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35902/0 (socket says 172.21.15.45:35902) 2024-06-06T04:27:55.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f68a6700 1 -- 172.21.15.45:0/4056922793 learned_addr learned my addr 172.21.15.45:0/4056922793 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:55.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f58a4700 1 --2- 172.21.15.45:0/4056922793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47f006ab30 0x7f47f019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:55.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f60a5700 1 --2- 172.21.15.45:0/4056922793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f006a170 0x7f47f019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:55.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f68a6700 1 -- 172.21.15.45:0/4056922793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47f006ab30 msgr2=0x7f47f019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:55.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f68a6700 1 --2- 172.21.15.45:0/4056922793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47f006ab30 0x7f47f019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f68a6700 1 -- 172.21.15.45:0/4056922793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f006a170 msgr2=0x7f47f019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:55.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f68a6700 1 --2- 172.21.15.45:0/4056922793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f006a170 0x7f47f019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f68a6700 1 -- 172.21.15.45:0/4056922793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f0199170 con 0x7f47f0108500 2024-06-06T04:27:55.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47f68a6700 1 --2- 172.21.15.45:0/4056922793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f0108500 0x7f47f0198a10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f47e800b970 tx=0x7f47e800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:55.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47e77fe700 1 -- 172.21.15.45:0/4056922793 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8019070 con 0x7f47f0108500 2024-06-06T04:27:55.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47e77fe700 1 -- 172.21.15.45:0/4056922793 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47e8014440 con 0x7f47f0108500 2024-06-06T04:27:55.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.862+0000 7f47e77fe700 1 -- 172.21.15.45:0/4056922793 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e800ce60 con 0x7f47f0108500 2024-06-06T04:27:55.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.863+0000 7f47f8309700 1 -- 172.21.15.45:0/4056922793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f0199460 con 0x7f47f0108500 2024-06-06T04:27:55.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.863+0000 7f47f8309700 1 -- 172.21.15.45:0/4056922793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f01a5720 con 0x7f47f0108500 2024-06-06T04:27:55.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.865+0000 7f47e77fe700 1 -- 172.21.15.45:0/4056922793 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f47e801c070 con 0x7f47f0108500 2024-06-06T04:27:55.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.866+0000 7f47e77fe700 1 --2- 172.21.15.45:0/4056922793 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f47dc0a2ca0 0x7f47dc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.866+0000 7f47f60a5700 1 --2- 172.21.15.45:0/4056922793 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f47dc0a2ca0 0x7f47dc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:55.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.866+0000 7f47e77fe700 1 -- 172.21.15.45:0/4056922793 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f47e80ce690 con 0x7f47f0108500 2024-06-06T04:27:55.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.866+0000 7f4802b8cb80 1 Processor -- start 2024-06-06T04:27:55.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.866+0000 7f47f60a5700 1 --2- 172.21.15.45:0/4056922793 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f47dc0a2ca0 0x7f47dc0a5160 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f47ec006fd0 tx=0x7f47ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.866+0000 7f4802b8cb80 1 -- start start 2024-06-06T04:27:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.866+0000 7f4802b8cb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 0x5621e1414e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f4802b8cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e1415340 0x5621e140eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f4802b8cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e140f2e0 0x5621e140f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f4802b8cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5621e135c7c0 con 0x5621e1415340 2024-06-06T04:27:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f4802b8cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5621e13046d0 con 0x5621e140f2e0 2024-06-06T04:27:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f4802b8cb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5621e1358780 con 0x5621e1414a20 2024-06-06T04:27:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f47f58a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e1415340 0x5621e140eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f47f58a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e1415340 0x5621e140eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35916/0 (socket says 172.21.15.45:35916) 2024-06-06T04:27:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f47f58a4700 1 -- 172.21.15.45:0/1044187001 learned_addr learned my addr 172.21.15.45:0/1044187001 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f47f60a5700 1 --2- 172.21.15.45:0/1044187001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 0x5621e1414e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f47f68a6700 1 --2- 172.21.15.45:0/1044187001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e140f2e0 0x5621e140f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f47f58a4700 1 -- 172.21.15.45:0/1044187001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 msgr2=0x5621e1414e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:27:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f47f58a4700 1 --2- 172.21.15.45:0/1044187001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 0x5621e1414e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f47f58a4700 1 -- 172.21.15.45:0/1044187001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e140f2e0 msgr2=0x5621e140f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:27:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f47f58a4700 1 --2- 172.21.15.45:0/1044187001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e140f2e0 0x5621e140f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f47f58a4700 1 -- 172.21.15.45:0/1044187001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621e12faf00 con 0x5621e1415340 2024-06-06T04:27:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f47f60a5700 1 --2- 172.21.15.45:0/1044187001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 0x5621e1414e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:27:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f47f68a6700 1 --2- 172.21.15.45:0/1044187001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e140f2e0 0x5621e140f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:27:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f47f58a4700 1 --2- 172.21.15.45:0/1044187001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e1415340 0x5621e140eda0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f47e0015000 tx=0x7f47e0009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3525d5bdb988a42 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f47e4d70700 1 -- 172.21.15.45:0/1044187001 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0027070 con 0x5621e1415340 2024-06-06T04:27:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f47e4d70700 1 -- 172.21.15.45:0/1044187001 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47e0003d00 con 0x5621e1415340 2024-06-06T04:27:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.867+0000 7f47e4d70700 1 -- 172.21.15.45:0/1044187001 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0005510 con 0x5621e1415340 2024-06-06T04:27:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.868+0000 7f4802b8cb80 1 -- 172.21.15.45:0/1044187001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e1415340 msgr2=0x5621e140eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.868+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/1044187001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e1415340 0x5621e140eda0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f47e0015000 tx=0x7f47e0009ee0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.868+0000 7f4802b8cb80 1 -- 172.21.15.45:0/1044187001 shutdown_connections 2024-06-06T04:27:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.868+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/1044187001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e140f2e0 0x5621e140f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.868+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/1044187001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 0x5621e1414e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.868+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/1044187001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e1415340 0x5621e140eda0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.868+0000 7f4802b8cb80 1 -- 172.21.15.45:0/1044187001 >> 172.21.15.45:0/1044187001 conn(0x5621e1335c70 msgr2=0x5621e1405800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.868+0000 7f4802b8cb80 1 -- 172.21.15.45:0/1044187001 shutdown_connections 2024-06-06T04:27:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.868+0000 7f4802b8cb80 1 -- 172.21.15.45:0/1044187001 wait complete. 2024-06-06T04:27:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f4802b8cb80 1 -- start start 2024-06-06T04:27:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f4802b8cb80 1 Processor -- start 2024-06-06T04:27:55.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f4802b8cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e140f2e0 0x5621e141b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f4802b8cb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 0x5621e14183d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f4802b8cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e1415340 0x5621e1418910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f4802b8cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5621e141bd20 con 0x5621e140f2e0 2024-06-06T04:27:55.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f4802b8cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5621e12fcba0 con 0x5621e1415340 2024-06-06T04:27:55.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f4802b8cb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5621e141bea0 con 0x5621e1414a20 2024-06-06T04:27:55.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f4802b8cb80 10 client.0 _ll_register_callbacks cb 0x5621e12f68d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:27:55.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f47f60a5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e140f2e0 0x5621e141b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:55.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f47f60a5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e140f2e0 0x5621e141b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35918/0 (socket says 172.21.15.45:35918) 2024-06-06T04:27:55.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f47f60a5700 1 -- 172.21.15.45:0/3889650886 learned_addr learned my addr 172.21.15.45:0/3889650886 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:55.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f47f68a6700 1 --2- 172.21.15.45:0/3889650886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e1415340 0x5621e1418910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:55.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f47f58a4700 1 --2- 172.21.15.45:0/3889650886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 0x5621e14183d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:55.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f47f60a5700 1 -- 172.21.15.45:0/3889650886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 msgr2=0x5621e14183d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:27:55.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f47f60a5700 1 --2- 172.21.15.45:0/3889650886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 0x5621e14183d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f47f60a5700 1 -- 172.21.15.45:0/3889650886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e1415340 msgr2=0x5621e1418910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:27:55.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.869+0000 7f47f60a5700 1 --2- 172.21.15.45:0/3889650886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e1415340 0x5621e1418910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.870+0000 7f47f60a5700 1 -- 172.21.15.45:0/3889650886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621e12ab880 con 0x5621e140f2e0 2024-06-06T04:27:55.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.870+0000 7f47f58a4700 1 --2- 172.21.15.45:0/3889650886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 0x5621e14183d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:27:55.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.870+0000 7f47f60a5700 1 --2- 172.21.15.45:0/3889650886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e140f2e0 0x5621e141b0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0092d0 tx=0x7f47ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1f9a32f90d77d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:55.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.870+0000 7f47d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:27:55.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:55 smithi045 ceph-mon[32299]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:55.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.870+0000 7f47d9ffb700 1 -- 172.21.15.45:0/3889650886 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec014c20 con 0x5621e140f2e0 2024-06-06T04:27:55.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.870+0000 7f47d9ffb700 1 -- 172.21.15.45:0/3889650886 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f47ec00f040 con 0x5621e140f2e0 2024-06-06T04:27:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.870+0000 7f4802b8cb80 1 -- 172.21.15.45:0/3889650886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e0003d00 con 0x5621e140f2e0 2024-06-06T04:27:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.870+0000 7f47d9ffb700 1 -- 172.21.15.45:0/3889650886 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f47ec014dc0 con 0x5621e140f2e0 2024-06-06T04:27:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.870+0000 7f4802b8cb80 10 client.14703 fetch_fsmap learned FSMap version 17 2024-06-06T04:27:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.871+0000 7f4802b8cb80 1 -- 172.21.15.45:0/3889650886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5621e1426640 con 0x5621e140f2e0 2024-06-06T04:27:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.871+0000 7f47d9ffb700 1 -- 172.21.15.45:0/3889650886 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47e0003d00 con 0x5621e140f2e0 2024-06-06T04:27:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.871+0000 7f47d9ffb700 1 -- 172.21.15.45:0/3889650886 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f47ec013070 con 0x5621e140f2e0 2024-06-06T04:27:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.871+0000 7f4802b8cb80 10 client.14703 fetch_fsmap finished waiting for FSMap version 17 2024-06-06T04:27:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.871+0000 7f4802b8cb80 10 client.14703 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T04:27:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.871+0000 7f4802b8cb80 20 client.14703 populate_metadata read hostname 'smithi045' 2024-06-06T04:27:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.871+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/3889650886 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5621e1428d00 0x5621e142a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.871+0000 7f4802b8cb80 4 client.14703 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:27:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.871+0000 7f4802b8cb80 1 -- 172.21.15.45:0/3889650886 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621e1369cc0 con 0x5621e1428d00 2024-06-06T04:27:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.871+0000 7f47f58a4700 1 --2- 172.21.15.45:0/3889650886 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5621e1428d00 0x5621e142a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.872+0000 7f47f58a4700 1 --2- 172.21.15.45:0/3889650886 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5621e1428d00 0x5621e142a170 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3a94bf5d0d240b6 server_cookie=45c78ff5ca5b8ba3 in_seq=0 out_seq=0 2024-06-06T04:27:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.872+0000 7f47d9ffb700 10 client.14703 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:27:55.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.873+0000 7f47d9ffb700 1 -- 172.21.15.45:0/3889650886 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5621e1369cc0 con 0x5621e1428d00 2024-06-06T04:27:55.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.873+0000 7f47d9ffb700 10 client.14703 handle_command_reply: tid=0 2024-06-06T04:27:55.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.873+0000 7f4802b8cb80 1 client.14703 shutdown 2024-06-06T04:27:55.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.873+0000 7f4802b8cb80 1 -- 172.21.15.45:0/3889650886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e140f2e0 msgr2=0x5621e141b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:55.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.873+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/3889650886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e140f2e0 0x5621e141b0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0092d0 tx=0x7f47ec004750 comp rx=0 tx=0).stop 2024-06-06T04:27:55.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.873+0000 7f4802b8cb80 1 -- 172.21.15.45:0/3889650886 shutdown_connections 2024-06-06T04:27:55.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.873+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/3889650886 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5621e1428d00 0x5621e142a170 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.873+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/3889650886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e1415340 0x5621e1418910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.873+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/3889650886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e140f2e0 0x5621e141b0d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.873+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/3889650886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 0x5621e14183d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.873+0000 7f4802b8cb80 1 -- 172.21.15.45:0/3889650886 >> 172.21.15.45:0/3889650886 conn(0x5621e1335c70 msgr2=0x5621e1409980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:55.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.873+0000 7f47d9ffb700 0 client.14703 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:27:55.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.873+0000 7f4802b8cb80 1 -- 172.21.15.45:0/3889650886 shutdown_connections 2024-06-06T04:27:55.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.873+0000 7f4802b8cb80 1 -- 172.21.15.45:0/3889650886 wait complete. 2024-06-06T04:27:55.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.873+0000 7f4802b8cb80 20 client.14703 trim_cache size 0 max 16384 2024-06-06T04:27:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.895+0000 7f4802b8cb80 1 Processor -- start 2024-06-06T04:27:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.895+0000 7f4802b8cb80 1 -- start start 2024-06-06T04:27:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.895+0000 7f4802b8cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e140f2e0 0x5621e140a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.895+0000 7f4802b8cb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 0x5621e14335e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.895+0000 7f4802b8cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e1415340 0x5621e1433b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f4802b8cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5621e14284b0 con 0x5621e140f2e0 2024-06-06T04:27:55.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f4802b8cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5621e1428b80 con 0x5621e1415340 2024-06-06T04:27:55.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f4802b8cb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5621e1429250 con 0x5621e1414a20 2024-06-06T04:27:55.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f47f60a5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e140f2e0 0x5621e140a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:55.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f47f60a5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e140f2e0 0x5621e140a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35922/0 (socket says 172.21.15.45:35922) 2024-06-06T04:27:55.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f47f60a5700 1 -- 172.21.15.45:0/3597916644 learned_addr learned my addr 172.21.15.45:0/3597916644 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:55.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f47f58a4700 1 --2- 172.21.15.45:0/3597916644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 0x5621e14335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:55.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f47f68a6700 1 --2- 172.21.15.45:0/3597916644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e1415340 0x5621e1433b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:55.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f47f60a5700 1 -- 172.21.15.45:0/3597916644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 msgr2=0x5621e14335e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:27:55.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f47f60a5700 1 --2- 172.21.15.45:0/3597916644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 0x5621e14335e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f47f60a5700 1 -- 172.21.15.45:0/3597916644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e1415340 msgr2=0x5621e1433b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:27:55.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f47f60a5700 1 --2- 172.21.15.45:0/3597916644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e1415340 0x5621e1433b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f47f60a5700 1 -- 172.21.15.45:0/3597916644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621e142a4e0 con 0x5621e140f2e0 2024-06-06T04:27:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f47f58a4700 1 --2- 172.21.15.45:0/3597916644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 0x5621e14335e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:27:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f47f68a6700 1 --2- 172.21.15.45:0/3597916644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e1415340 0x5621e1433b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:27:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f47f60a5700 1 --2- 172.21.15.45:0/3597916644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e140f2e0 0x5621e140a090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0048a0 tx=0x7f47ec006d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0e324b5fd997f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f47dbfff700 1 -- 172.21.15.45:0/3597916644 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec014b20 con 0x5621e140f2e0 2024-06-06T04:27:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f47dbfff700 1 -- 172.21.15.45:0/3597916644 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47ec038070 con 0x5621e140f2e0 2024-06-06T04:27:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.896+0000 7f47dbfff700 1 -- 172.21.15.45:0/3597916644 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec012d80 con 0x5621e140f2e0 2024-06-06T04:27:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.897+0000 7f4802b8cb80 1 -- 172.21.15.45:0/3597916644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e140f2e0 msgr2=0x5621e140a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.897+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/3597916644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e140f2e0 0x5621e140a090 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0048a0 tx=0x7f47ec006d50 comp rx=0 tx=0).stop 2024-06-06T04:27:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.897+0000 7f4802b8cb80 1 -- 172.21.15.45:0/3597916644 shutdown_connections 2024-06-06T04:27:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.897+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/3597916644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e1415340 0x5621e1433b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.897+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/3597916644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e140f2e0 0x5621e140a090 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.897+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/3597916644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1414a20 0x5621e14335e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.897+0000 7f4802b8cb80 1 -- 172.21.15.45:0/3597916644 >> 172.21.15.45:0/3597916644 conn(0x5621e1335c70 msgr2=0x5621e1409980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.897+0000 7f4802b8cb80 1 -- 172.21.15.45:0/3597916644 shutdown_connections 2024-06-06T04:27:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.897+0000 7f4802b8cb80 1 -- 172.21.15.45:0/3597916644 wait complete. 2024-06-06T04:27:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.897+0000 7f4802b8cb80 1 -- start start 2024-06-06T04:27:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f4802b8cb80 1 Processor -- start 2024-06-06T04:27:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f4802b8cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e140f2e0 0x5621e142fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f4802b8cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e1414a20 0x5621e1430130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f4802b8cb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1415340 0x5621e148d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f4802b8cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5621e1429ff0 con 0x5621e1414a20 2024-06-06T04:27:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f4802b8cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5621e1429920 con 0x5621e140f2e0 2024-06-06T04:27:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f4802b8cb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5621e142a6c0 con 0x5621e1415340 2024-06-06T04:27:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f4802b8cb80 10 client.0 _ll_register_callbacks cb 0x5621e12f68d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:27:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f47f58a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e1414a20 0x5621e1430130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f47f58a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e1414a20 0x5621e1430130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35928/0 (socket says 172.21.15.45:35928) 2024-06-06T04:27:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f47f58a4700 1 -- 172.21.15.45:0/949226762 learned_addr learned my addr 172.21.15.45:0/949226762 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f47f58a4700 1 -- 172.21.15.45:0/949226762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1415340 msgr2=0x5621e148d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:27:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f47f58a4700 1 --2- 172.21.15.45:0/949226762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1415340 0x5621e148d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f47f58a4700 1 -- 172.21.15.45:0/949226762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e140f2e0 msgr2=0x5621e142fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:27:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f47f58a4700 1 --2- 172.21.15.45:0/949226762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e140f2e0 0x5621e142fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f47f58a4700 1 -- 172.21.15.45:0/949226762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621e148dba0 con 0x5621e1414a20 2024-06-06T04:27:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f47f58a4700 1 --2- 172.21.15.45:0/949226762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e1414a20 0x5621e1430130 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009990 tx=0x7f47e0005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f131786688ed0aab server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.898+0000 7f47d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:27:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.899+0000 7f47d9ffb700 1 -- 172.21.15.45:0/949226762 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0027040 con 0x5621e1414a20 2024-06-06T04:27:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.899+0000 7f47d9ffb700 1 -- 172.21.15.45:0/949226762 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f47e0003850 con 0x5621e1414a20 2024-06-06T04:27:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.899+0000 7f4802b8cb80 1 -- 172.21.15.45:0/949226762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47ec038070 con 0x5621e1414a20 2024-06-06T04:27:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.899+0000 7f47d9ffb700 1 -- 172.21.15.45:0/949226762 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f47e0003a50 con 0x5621e1414a20 2024-06-06T04:27:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.899+0000 7f4802b8cb80 10 client.14709 fetch_fsmap learned FSMap version 17 2024-06-06T04:27:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.899+0000 7f4802b8cb80 1 -- 172.21.15.45:0/949226762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5621e1494f10 con 0x5621e1414a20 2024-06-06T04:27:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.899+0000 7f47d9ffb700 1 -- 172.21.15.45:0/949226762 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47ec038070 con 0x5621e1414a20 2024-06-06T04:27:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.899+0000 7f47d9ffb700 1 -- 172.21.15.45:0/949226762 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f47e0022710 con 0x5621e1414a20 2024-06-06T04:27:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.899+0000 7f4802b8cb80 10 client.14709 fetch_fsmap finished waiting for FSMap version 17 2024-06-06T04:27:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.899+0000 7f4802b8cb80 10 client.14709 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T04:27:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.899+0000 7f4802b8cb80 20 client.14709 populate_metadata read hostname 'smithi045' 2024-06-06T04:27:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.899+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/949226762 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5621e14995c0 0x5621e14982d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.899+0000 7f4802b8cb80 4 client.14709 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T04:27:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.899+0000 7f4802b8cb80 1 -- 172.21.15.45:0/949226762 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "status"}) -- 0x5621e1364240 con 0x5621e14995c0 2024-06-06T04:27:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.900+0000 7f47f60a5700 1 --2- 172.21.15.45:0/949226762 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5621e14995c0 0x5621e14982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.900+0000 7f47f60a5700 1 --2- 172.21.15.45:0/949226762 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5621e14995c0 0x5621e14982d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=156f2d02162b6037 server_cookie=2989a04e6ed13019 in_seq=0 out_seq=0 2024-06-06T04:27:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.900+0000 7f47d9ffb700 10 client.14709 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:27:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.900+0000 7f47d9ffb700 1 -- 172.21.15.45:0/949226762 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5621e1364240 con 0x5621e14995c0 2024-06-06T04:27:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.900+0000 7f47d9ffb700 10 client.14709 handle_command_reply: tid=0 2024-06-06T04:27:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.900+0000 7f4802b8cb80 1 client.14709 shutdown 2024-06-06T04:27:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.900+0000 7f4802b8cb80 1 -- 172.21.15.45:0/949226762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e1414a20 msgr2=0x5621e1430130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.900+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/949226762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e1414a20 0x5621e1430130 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009990 tx=0x7f47e0005900 comp rx=0 tx=0).stop 2024-06-06T04:27:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.900+0000 7f4802b8cb80 1 -- 172.21.15.45:0/949226762 shutdown_connections 2024-06-06T04:27:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.900+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/949226762 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5621e14995c0 0x5621e14982d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.900+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/949226762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5621e1415340 0x5621e148d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.900+0000 7f47d9ffb700 0 client.14709 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:27:55.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.900+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/949226762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5621e140f2e0 0x5621e142fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.900+0000 7f4802b8cb80 1 --2- 172.21.15.45:0/949226762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621e1414a20 0x5621e1430130 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.900+0000 7f4802b8cb80 1 -- 172.21.15.45:0/949226762 >> 172.21.15.45:0/949226762 conn(0x5621e1335c70 msgr2=0x5621e13fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:55.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.901+0000 7f4802b8cb80 1 -- 172.21.15.45:0/949226762 shutdown_connections 2024-06-06T04:27:55.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.901+0000 7f4802b8cb80 1 -- 172.21.15.45:0/949226762 wait complete. 2024-06-06T04:27:55.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.901+0000 7f4802b8cb80 20 client.14709 trim_cache size 0 max 16384 2024-06-06T04:27:55.907 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:27:55.907 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "c4f6b938-23bb-11ef-bc9b-c7b262605968", 2024-06-06T04:27:55.908 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-06-06T04:27:55.908 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24473, 2024-06-06T04:27:55.908 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-06-06T04:27:55.908 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-06-06T04:27:55.908 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-06-06T04:27:55.908 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 31.337151714000001, 2024-06-06T04:27:55.908 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 17, 2024-06-06T04:27:55.908 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 96, 2024-06-06T04:27:55.908 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 96, 2024-06-06T04:27:55.908 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 46.574625140999999 2024-06-06T04:27:55.908 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:27:55.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.901+0000 7f47f8309700 1 -- 172.21.15.45:0/4056922793 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f47dc0a2ca0 msgr2=0x7f47dc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:55.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.901+0000 7f47f8309700 1 --2- 172.21.15.45:0/4056922793 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f47dc0a2ca0 0x7f47dc0a5160 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f47ec006fd0 tx=0x7f47ec008040 comp rx=0 tx=0).stop 2024-06-06T04:27:55.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.901+0000 7f47f8309700 1 -- 172.21.15.45:0/4056922793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f0108500 msgr2=0x7f47f0198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:55.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.901+0000 7f47f8309700 1 --2- 172.21.15.45:0/4056922793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f0108500 0x7f47f0198a10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f47e800b970 tx=0x7f47e800bd30 comp rx=0 tx=0).stop 2024-06-06T04:27:55.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.901+0000 7f47f8309700 1 -- 172.21.15.45:0/4056922793 shutdown_connections 2024-06-06T04:27:55.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.901+0000 7f47f8309700 1 --2- 172.21.15.45:0/4056922793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f0108500 0x7f47f0198a10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.901+0000 7f47f8309700 1 --2- 172.21.15.45:0/4056922793 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f47dc0a2ca0 0x7f47dc0a5160 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.901+0000 7f47f8309700 1 --2- 172.21.15.45:0/4056922793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47f006a170 0x7f47f019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.901+0000 7f47f8309700 1 --2- 172.21.15.45:0/4056922793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47f006ab30 0x7f47f019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:55.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.901+0000 7f47f8309700 1 -- 172.21.15.45:0/4056922793 >> 172.21.15.45:0/4056922793 conn(0x7f47f0078340 msgr2=0x7f47f00fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:55.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.902+0000 7f47f8309700 1 -- 172.21.15.45:0/4056922793 shutdown_connections 2024-06-06T04:27:55.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:55.902+0000 7f47f8309700 1 -- 172.21.15.45:0/4056922793 wait complete. 2024-06-06T04:27:56.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:55 smithi086 ceph-mon[36053]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:56.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:55 smithi154 ceph-mon[36428]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:56.633 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-06T04:27:56.633 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-06T04:27:56.634 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-06T04:27:56.888 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:27:57.956 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:57 smithi045 ceph-mon[32299]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:58.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:57 smithi086 ceph-mon[36053]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:58.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:57 smithi154 ceph-mon[36428]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:58.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.474+0000 7fd9fc122700 1 -- 172.21.15.45:0/162785737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9f4102e80 msgr2=0x7fd9f410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:58.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.474+0000 7fd9fc122700 1 --2- 172.21.15.45:0/162785737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9f4102e80 0x7fd9f410f330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00b0a0 tx=0x7fd9ec00b3b0 comp rx=0 tx=0).stop 2024-06-06T04:27:58.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.474+0000 7fd9fc122700 1 -- 172.21.15.45:0/162785737 shutdown_connections 2024-06-06T04:27:58.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.474+0000 7fd9fc122700 1 --2- 172.21.15.45:0/162785737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9f4102e80 0x7fd9f410f330 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:58.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.474+0000 7fd9fc122700 1 --2- 172.21.15.45:0/162785737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9f41084f0 0x7fd9f41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:58.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.474+0000 7fd9fc122700 1 --2- 172.21.15.45:0/162785737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9f41024e0 0x7fd9f4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:58.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.474+0000 7fd9fc122700 1 -- 172.21.15.45:0/162785737 >> 172.21.15.45:0/162785737 conn(0x7fd9f4079530 msgr2=0x7fd9f4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:58.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.475+0000 7fd9fc122700 1 -- 172.21.15.45:0/162785737 shutdown_connections 2024-06-06T04:27:58.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.475+0000 7fd9fc122700 1 -- 172.21.15.45:0/162785737 wait complete. 2024-06-06T04:27:58.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.475+0000 7fd9fc122700 1 Processor -- start 2024-06-06T04:27:58.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.475+0000 7fd9fc122700 1 -- start start 2024-06-06T04:27:58.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.475+0000 7fd9fc122700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9f41024e0 0x7fd9f4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:58.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.475+0000 7fd9fc122700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9f4102e80 0x7fd9f4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:58.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.475+0000 7fd9fc122700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9f41084f0 0x7fd9f4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:58.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.475+0000 7fd9fc122700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f41126a0 con 0x7fd9f41024e0 2024-06-06T04:27:58.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.475+0000 7fd9fc122700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f4112520 con 0x7fd9f41084f0 2024-06-06T04:27:58.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.475+0000 7fd9fc122700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f4112820 con 0x7fd9f4102e80 2024-06-06T04:27:58.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9f9ebe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9f41024e0 0x7fd9f4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:58.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9f9ebe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9f41024e0 0x7fd9f4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47220/0 (socket says 172.21.15.45:47220) 2024-06-06T04:27:58.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9f9ebe700 1 -- 172.21.15.45:0/1844316301 learned_addr learned my addr 172.21.15.45:0/1844316301 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:27:58.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9f96bd700 1 --2- 172.21.15.45:0/1844316301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9f4102e80 0x7fd9f4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:58.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9fa6bf700 1 --2- 172.21.15.45:0/1844316301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9f41084f0 0x7fd9f4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:58.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9f9ebe700 1 -- 172.21.15.45:0/1844316301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9f4102e80 msgr2=0x7fd9f4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:58.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9f9ebe700 1 --2- 172.21.15.45:0/1844316301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9f4102e80 0x7fd9f4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:58.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9f9ebe700 1 -- 172.21.15.45:0/1844316301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9f41084f0 msgr2=0x7fd9f4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:58.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9f9ebe700 1 --2- 172.21.15.45:0/1844316301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9f41084f0 0x7fd9f4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:58.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9f9ebe700 1 -- 172.21.15.45:0/1844316301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f41a7020 con 0x7fd9f41024e0 2024-06-06T04:27:58.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9fa6bf700 1 --2- 172.21.15.45:0/1844316301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9f41084f0 0x7fd9f4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:27:58.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9f96bd700 1 --2- 172.21.15.45:0/1844316301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9f4102e80 0x7fd9f4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:27:58.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9f9ebe700 1 --2- 172.21.15.45:0/1844316301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9f41024e0 0x7fd9f4074790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000c9b0 tx=0x7fd9f000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:58.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9eaffd700 1 -- 172.21.15.45:0/1844316301 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0007b00 con 0x7fd9f41024e0 2024-06-06T04:27:58.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9eaffd700 1 -- 172.21.15.45:0/1844316301 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9f0007ca0 con 0x7fd9f41024e0 2024-06-06T04:27:58.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9fc122700 1 -- 172.21.15.45:0/1844316301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f41a7310 con 0x7fd9f41024e0 2024-06-06T04:27:58.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9eaffd700 1 -- 172.21.15.45:0/1844316301 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0018730 con 0x7fd9f41024e0 2024-06-06T04:27:58.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.476+0000 7fd9fc122700 1 -- 172.21.15.45:0/1844316301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f41a78a0 con 0x7fd9f41024e0 2024-06-06T04:27:58.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.477+0000 7fd9fc122700 1 -- 172.21.15.45:0/1844316301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f4069180 con 0x7fd9f41024e0 2024-06-06T04:27:58.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.478+0000 7fd9eaffd700 1 -- 172.21.15.45:0/1844316301 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd9f000f450 con 0x7fd9f41024e0 2024-06-06T04:27:58.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.479+0000 7fd9eaffd700 1 --2- 172.21.15.45:0/1844316301 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd9e00a2ca0 0x7fd9e00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:27:58.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.479+0000 7fd9f96bd700 1 --2- 172.21.15.45:0/1844316301 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd9e00a2ca0 0x7fd9e00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:27:58.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.479+0000 7fd9eaffd700 1 -- 172.21.15.45:0/1844316301 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7fd9f00cf500 con 0x7fd9f41024e0 2024-06-06T04:27:58.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.480+0000 7fd9f96bd700 1 --2- 172.21.15.45:0/1844316301 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd9e00a2ca0 0x7fd9e00a5160 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4079aa0 tx=0x7fd9e4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:27:58.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.483+0000 7fd9eaffd700 1 -- 172.21.15.45:0/1844316301 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd9f0014030 con 0x7fd9f41024e0 2024-06-06T04:27:58.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.736+0000 7fd9fc122700 1 -- 172.21.15.45:0/1844316301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd9f402dca0 con 0x7fd9f41024e0 2024-06-06T04:27:58.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.736+0000 7fd9eaffd700 1 -- 172.21.15.45:0/1844316301 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fd9f000de20 con 0x7fd9f41024e0 2024-06-06T04:27:58.737 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-06T04:27:58.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.739+0000 7fd9fc122700 1 -- 172.21.15.45:0/1844316301 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd9e00a2ca0 msgr2=0x7fd9e00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:58.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.739+0000 7fd9fc122700 1 --2- 172.21.15.45:0/1844316301 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd9e00a2ca0 0x7fd9e00a5160 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4079aa0 tx=0x7fd9e4008040 comp rx=0 tx=0).stop 2024-06-06T04:27:58.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.739+0000 7fd9fc122700 1 -- 172.21.15.45:0/1844316301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9f41024e0 msgr2=0x7fd9f4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:27:58.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.739+0000 7fd9fc122700 1 --2- 172.21.15.45:0/1844316301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9f41024e0 0x7fd9f4074790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000c9b0 tx=0x7fd9f000cd70 comp rx=0 tx=0).stop 2024-06-06T04:27:58.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.739+0000 7fd9fc122700 1 -- 172.21.15.45:0/1844316301 shutdown_connections 2024-06-06T04:27:58.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.739+0000 7fd9fc122700 1 --2- 172.21.15.45:0/1844316301 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd9e00a2ca0 0x7fd9e00a5160 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:58.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.739+0000 7fd9fc122700 1 --2- 172.21.15.45:0/1844316301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9f41084f0 0x7fd9f4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:58.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.739+0000 7fd9fc122700 1 --2- 172.21.15.45:0/1844316301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9f41024e0 0x7fd9f4074790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:58.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.739+0000 7fd9fc122700 1 --2- 172.21.15.45:0/1844316301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9f4102e80 0x7fd9f4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:27:58.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.739+0000 7fd9fc122700 1 -- 172.21.15.45:0/1844316301 >> 172.21.15.45:0/1844316301 conn(0x7fd9f4079530 msgr2=0x7fd9f4100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:27:58.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.739+0000 7fd9fc122700 1 -- 172.21.15.45:0/1844316301 shutdown_connections 2024-06-06T04:27:58.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:58.739+0000 7fd9fc122700 1 -- 172.21.15.45:0/1844316301 wait complete. 2024-06-06T04:27:59.320 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-06-06T04:27:59.320 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-06T04:27:59.320 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-06-06T04:27:59.320 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-06-06T04:27:59.320 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-06T04:27:59.320 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-06-06T04:27:59.321 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-06T04:27:59.321 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-06T04:27:59.321 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-06-06T04:27:59.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:27:59.321 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-06-06T04:27:59.339 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-06T04:27:59.339 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-06T04:27:59.339 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-06-06T04:27:59.339 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-06-06T04:27:59.339 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-06-06T04:27:59.339 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-06-06T04:27:59.339 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T04:27:59.339 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-06-06T04:27:59.339 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-06-06T04:27:59.339 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T04:27:59.339 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-06-06T04:27:59.339 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-06-06T04:27:59.339 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-06T04:27:59.340 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-06-06T04:27:59.340 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-06-06T04:27:59.340 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-06-06T04:27:59.340 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 40453sec preferred_lft 40453sec 2024-06-06T04:27:59.340 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-06-06T04:27:59.340 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-06-06T04:27:59.340 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T04:27:59.340 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-06-06T04:27:59.340 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-06-06T04:27:59.341 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-06T04:27:59.341 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T04:27:59.341 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-06-06T04:27:59.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2024-06-06T04:27:59.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2024-06-06T04:27:59.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2024-06-06T04:27:59.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-06T04:27:59.341 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-06-06T04:27:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T04:27:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T04:27:59.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:27:59.563 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-06T04:27:59.589 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-06-06T04:27:59.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:27:59.591 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-06-06T04:27:59.666 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-06-06T04:27:59.667 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-06T04:27:59.667 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-06T04:27:59.667 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-06T04:27:59.667 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-06T04:27:59.667 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T04:27:59.667 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-06-06T04:27:59.667 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-06T04:27:59.667 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-06T04:27:59.668 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-06T04:27:59.668 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-06-06T04:27:59.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T04:27:59.865 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:59 smithi045 ceph-mon[32299]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:27:59.865 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:27:59 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1844316301' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T04:27:59.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:27:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T04:27:59.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:27:59.956 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-06-06T04:27:59.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:27:59.972 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-06-06T04:28:00.028 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T04:28:00.028 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-06-06T04:28:00.028 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T04:28:00.028 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-06-06T04:28:00.028 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-06-06T04:28:00.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:59 smithi086 ceph-mon[36053]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:28:00.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:27:59 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1844316301' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T04:28:00.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T04:28:00.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T04:28:00.149 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-06-06T04:28:00.149 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T04:28:00.149 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-06-06T04:28:00.149 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-06-06T04:28:00.149 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-06T04:28:00.149 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-06-06T04:28:00.149 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-06-06T04:28:00.149 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-06-06T04:28:00.150 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-06-06T04:28:00.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:59 smithi154 ceph-mon[36428]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:28:00.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:27:59 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1844316301' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T04:28:00.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T04:28:00.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T04:28:00.277 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T04:28:00.277 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-06-06T04:28:00.277 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-06-06T04:28:00.277 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2024-06-06T04:28:00.277 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-06-06T04:28:00.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T04:28:00.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T04:28:00.359 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-06-06T04:28:00.359 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T04:28:00.359 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-06-06T04:28:00.375 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-06-06T04:28:00.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T04:28:00.375 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-06-06T04:28:00.431 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2024-06-06T04:28:00.498 DEBUG:teuthology.orchestra.run.smithi045:> cat /proc/self/mounts | awk '{print $2}' 2024-06-06T04:28:00.555 INFO:teuthology.orchestra.run.smithi045.stdout:/proc 2024-06-06T04:28:00.555 INFO:teuthology.orchestra.run.smithi045.stdout:/sys 2024-06-06T04:28:00.555 INFO:teuthology.orchestra.run.smithi045.stdout:/dev 2024-06-06T04:28:00.555 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/security 2024-06-06T04:28:00.555 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/shm 2024-06-06T04:28:00.555 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/pts 2024-06-06T04:28:00.555 INFO:teuthology.orchestra.run.smithi045.stdout:/run 2024-06-06T04:28:00.555 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/cgroup 2024-06-06T04:28:00.555 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/pstore 2024-06-06T04:28:00.555 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/bpf 2024-06-06T04:28:00.556 INFO:teuthology.orchestra.run.smithi045.stdout:/ 2024-06-06T04:28:00.556 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/selinux 2024-06-06T04:28:00.556 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T04:28:00.556 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/debug 2024-06-06T04:28:00.556 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/mqueue 2024-06-06T04:28:00.556 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/hugepages 2024-06-06T04:28:00.556 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/tracing 2024-06-06T04:28:00.556 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-06T04:28:00.556 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/fuse/connections 2024-06-06T04:28:00.556 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/config 2024-06-06T04:28:00.556 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysctl.service 2024-06-06T04:28:00.556 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-06T04:28:00.556 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/nfs/rpc_pipefs 2024-06-06T04:28:00.557 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/1000 2024-06-06T04:28:00.557 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph 2024-06-06T04:28:00.557 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay 2024-06-06T04:28:00.557 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/57e443f29a64c959e8effc38e4275befcb284fbdb507cfd6aaf0c442915f8721/merged 2024-06-06T04:28:00.557 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/74431be50d65222972207a60a4c2f5a70aedb8a539ae8bef8bdc16892ed09952/merged 2024-06-06T04:28:00.557 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/0 2024-06-06T04:28:00.557 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T04:28:00.557 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/10bafd02f689ba63a05ed448439911fdc03cc7f3deaefc5465ce7072fcd2908b/merged 2024-06-06T04:28:00.557 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/5cdfe5a48f1929f19b88e34c68effb122ae479c8f4c4c1597a7975b5bdddc8f3/merged 2024-06-06T04:28:00.557 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/d3d8ff039eb3956da278e78baf060d983274d705302185cc3f75986841f98316/merged 2024-06-06T04:28:00.557 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/38c3e71ee0536db7c152dfeddf84a6291eb6cf535e4dbcb039208eb439e5b8a2/merged 2024-06-06T04:28:00.557 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/544f6f38c488d03f0f92d0f5428e3eaed2d25358f86ece6e89dbdc3ca6b8d4a2/merged 2024-06-06T04:28:00.557 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/f2f3e1307d03e6ba7f53088299078b6765c4e4d3aa7d67e51e32ef148eeab479/merged 2024-06-06T04:28:00.557 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/dde2a0e46c917da425a686247392814eb317a380d3855a146ba84659038f0cf2/merged 2024-06-06T04:28:00.557 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/3208d4a515a126b6e016421971bd1ac3649cb436a579a29fdc1720ff3d16253b/merged 2024-06-06T04:28:00.558 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns 2024-06-06T04:28:00.558 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T04:28:00.558 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T04:28:00.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:28:00.558 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2024-06-06T04:28:00.611 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-06T04:28:00.611 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-06-06T04:28:00.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2024-06-06T04:28:00.678 DEBUG:teuthology.orchestra.run.smithi045:> cat /proc/self/mounts | awk '{print $2}' 2024-06-06T04:28:00.735 INFO:teuthology.orchestra.run.smithi045.stdout:/proc 2024-06-06T04:28:00.735 INFO:teuthology.orchestra.run.smithi045.stdout:/sys 2024-06-06T04:28:00.735 INFO:teuthology.orchestra.run.smithi045.stdout:/dev 2024-06-06T04:28:00.735 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/security 2024-06-06T04:28:00.735 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/shm 2024-06-06T04:28:00.735 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/pts 2024-06-06T04:28:00.735 INFO:teuthology.orchestra.run.smithi045.stdout:/run 2024-06-06T04:28:00.735 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/cgroup 2024-06-06T04:28:00.735 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/pstore 2024-06-06T04:28:00.735 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/bpf 2024-06-06T04:28:00.735 INFO:teuthology.orchestra.run.smithi045.stdout:/ 2024-06-06T04:28:00.736 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/selinux 2024-06-06T04:28:00.736 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T04:28:00.736 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/debug 2024-06-06T04:28:00.736 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/mqueue 2024-06-06T04:28:00.736 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/hugepages 2024-06-06T04:28:00.736 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/tracing 2024-06-06T04:28:00.736 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-06T04:28:00.736 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/fuse/connections 2024-06-06T04:28:00.736 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/config 2024-06-06T04:28:00.736 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysctl.service 2024-06-06T04:28:00.736 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-06T04:28:00.736 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/nfs/rpc_pipefs 2024-06-06T04:28:00.736 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/1000 2024-06-06T04:28:00.736 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph 2024-06-06T04:28:00.737 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay 2024-06-06T04:28:00.737 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/57e443f29a64c959e8effc38e4275befcb284fbdb507cfd6aaf0c442915f8721/merged 2024-06-06T04:28:00.737 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/74431be50d65222972207a60a4c2f5a70aedb8a539ae8bef8bdc16892ed09952/merged 2024-06-06T04:28:00.737 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/0 2024-06-06T04:28:00.737 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T04:28:00.737 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/10bafd02f689ba63a05ed448439911fdc03cc7f3deaefc5465ce7072fcd2908b/merged 2024-06-06T04:28:00.737 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/5cdfe5a48f1929f19b88e34c68effb122ae479c8f4c4c1597a7975b5bdddc8f3/merged 2024-06-06T04:28:00.737 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/d3d8ff039eb3956da278e78baf060d983274d705302185cc3f75986841f98316/merged 2024-06-06T04:28:00.737 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/38c3e71ee0536db7c152dfeddf84a6291eb6cf535e4dbcb039208eb439e5b8a2/merged 2024-06-06T04:28:00.737 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/544f6f38c488d03f0f92d0f5428e3eaed2d25358f86ece6e89dbdc3ca6b8d4a2/merged 2024-06-06T04:28:00.737 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/f2f3e1307d03e6ba7f53088299078b6765c4e4d3aa7d67e51e32ef148eeab479/merged 2024-06-06T04:28:00.737 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/dde2a0e46c917da425a686247392814eb317a380d3855a146ba84659038f0cf2/merged 2024-06-06T04:28:00.737 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/3208d4a515a126b6e016421971bd1ac3649cb436a579a29fdc1720ff3d16253b/merged 2024-06-06T04:28:00.737 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns 2024-06-06T04:28:00.737 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T04:28:00.738 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T04:28:00.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:28:00.738 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2024-06-06T04:28:00.792 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:2024-06-06T04:28:00.791+0000 7fa926927480 -1 init, newargv = 0x559919b38930 newargc=15 2024-06-06T04:28:00.792 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[78548]: starting ceph client 2024-06-06T04:28:00.798 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[78548]: starting fuse 2024-06-06T04:28:01.892 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2024-06-06T04:28:01.926 DEBUG:teuthology.orchestra.run.smithi045:> cat /proc/self/mounts | awk '{print $2}' 2024-06-06T04:28:01.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:01 smithi045 ceph-mon[32299]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:28:01.985 INFO:teuthology.orchestra.run.smithi045.stdout:/proc 2024-06-06T04:28:01.985 INFO:teuthology.orchestra.run.smithi045.stdout:/sys 2024-06-06T04:28:01.985 INFO:teuthology.orchestra.run.smithi045.stdout:/dev 2024-06-06T04:28:01.985 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/security 2024-06-06T04:28:01.985 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/shm 2024-06-06T04:28:01.985 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/pts 2024-06-06T04:28:01.986 INFO:teuthology.orchestra.run.smithi045.stdout:/run 2024-06-06T04:28:01.986 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/cgroup 2024-06-06T04:28:01.986 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/pstore 2024-06-06T04:28:01.986 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/bpf 2024-06-06T04:28:01.986 INFO:teuthology.orchestra.run.smithi045.stdout:/ 2024-06-06T04:28:01.986 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/selinux 2024-06-06T04:28:01.986 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T04:28:01.986 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/debug 2024-06-06T04:28:01.986 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/mqueue 2024-06-06T04:28:01.986 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/hugepages 2024-06-06T04:28:01.986 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/tracing 2024-06-06T04:28:01.986 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-06T04:28:01.987 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/fuse/connections 2024-06-06T04:28:01.987 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/config 2024-06-06T04:28:01.987 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysctl.service 2024-06-06T04:28:01.987 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-06T04:28:01.987 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/nfs/rpc_pipefs 2024-06-06T04:28:01.987 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/1000 2024-06-06T04:28:01.987 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph 2024-06-06T04:28:01.987 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay 2024-06-06T04:28:01.987 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/57e443f29a64c959e8effc38e4275befcb284fbdb507cfd6aaf0c442915f8721/merged 2024-06-06T04:28:01.987 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/74431be50d65222972207a60a4c2f5a70aedb8a539ae8bef8bdc16892ed09952/merged 2024-06-06T04:28:01.987 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/0 2024-06-06T04:28:01.987 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T04:28:01.987 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/10bafd02f689ba63a05ed448439911fdc03cc7f3deaefc5465ce7072fcd2908b/merged 2024-06-06T04:28:01.987 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/5cdfe5a48f1929f19b88e34c68effb122ae479c8f4c4c1597a7975b5bdddc8f3/merged 2024-06-06T04:28:01.988 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/d3d8ff039eb3956da278e78baf060d983274d705302185cc3f75986841f98316/merged 2024-06-06T04:28:01.988 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/38c3e71ee0536db7c152dfeddf84a6291eb6cf535e4dbcb039208eb439e5b8a2/merged 2024-06-06T04:28:01.988 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/544f6f38c488d03f0f92d0f5428e3eaed2d25358f86ece6e89dbdc3ca6b8d4a2/merged 2024-06-06T04:28:01.988 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/f2f3e1307d03e6ba7f53088299078b6765c4e4d3aa7d67e51e32ef148eeab479/merged 2024-06-06T04:28:01.988 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/dde2a0e46c917da425a686247392814eb317a380d3855a146ba84659038f0cf2/merged 2024-06-06T04:28:01.988 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/3208d4a515a126b6e016421971bd1ac3649cb436a579a29fdc1720ff3d16253b/merged 2024-06-06T04:28:01.988 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns 2024-06-06T04:28:01.988 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T04:28:01.988 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T04:28:01.988 INFO:teuthology.orchestra.run.smithi045.stdout:/home/ubuntu/cephtest/mnt.admin 2024-06-06T04:28:01.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:28:01.989 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2024-06-06T04:28:02.047 INFO:teuthology.orchestra.run.smithi045.stdout:107 2024-06-06T04:28:02.048 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [107] 2024-06-06T04:28:02.048 DEBUG:teuthology.orchestra.run.smithi045:> sudo stdin-killer -- python3 -c ' 2024-06-06T04:28:02.048 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2024-06-06T04:28:02.048 DEBUG:teuthology.orchestra.run.smithi045:> import re 2024-06-06T04:28:02.048 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-06-06T04:28:02.048 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2024-06-06T04:28:02.048 DEBUG:teuthology.orchestra.run.smithi045:> 2024-06-06T04:28:02.048 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2024-06-06T04:28:02.048 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-06-06T04:28:02.048 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2024-06-06T04:28:02.049 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-06-06T04:28:02.049 DEBUG:teuthology.orchestra.run.smithi045:> 2024-06-06T04:28:02.049 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2024-06-06T04:28:02.049 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2024-06-06T04:28:02.049 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2024-06-06T04:28:02.049 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2024-06-06T04:28:02.049 DEBUG:teuthology.orchestra.run.smithi045:> 2024-06-06T04:28:02.049 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2024-06-06T04:28:02.049 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-06T04:28:02.049 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-06T04:28:02.049 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-06T04:28:02.049 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2024-06-06T04:28:02.049 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2024-06-06T04:28:02.049 DEBUG:teuthology.orchestra.run.smithi045:> return f 2024-06-06T04:28:02.049 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-06T04:28:02.050 DEBUG:teuthology.orchestra.run.smithi045:> 2024-06-06T04:28:02.050 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.admin")) 2024-06-06T04:28:02.050 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-06-06T04:28:02.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:01 smithi086 ceph-mon[36053]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:28:02.176 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.admin.78548.asok 2024-06-06T04:28:02.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T04:28:02.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:01 smithi154 ceph-mon[36428]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:28:02.188 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.78548.asok 2024-06-06T04:28:02.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:28:02.188 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.78548.asok status 2024-06-06T04:28:02.328 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:28:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2024-06-06T04:28:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-06T04:28:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T04:28:02.328 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "admin", 2024-06-06T04:28:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2024-06-06T04:28:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-06-06T04:28:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "78548", 2024-06-06T04:28:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2024-06-06T04:28:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:28:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2024-06-06T04:28:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2024-06-06T04:28:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 14721, 2024-06-06T04:28:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2024-06-06T04:28:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2024-06-06T04:28:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2024-06-06T04:28:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 14721 2024-06-06T04:28:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:28:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2024-06-06T04:28:02.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-06-06T04:28:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2024-06-06T04:28:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4177979992 2024-06-06T04:28:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:28:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:28:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2024-06-06T04:28:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-06-06T04:28:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2024-06-06T04:28:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4177979992 2024-06-06T04:28:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:28:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.14721 192.168.0.1:0/4177979992", 2024-06-06T04:28:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/4177979992", 2024-06-06T04:28:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2024-06-06T04:28:02.330 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 17, 2024-06-06T04:28:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 96, 2024-06-06T04:28:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2024-06-06T04:28:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2024-06-06T04:28:02.331 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2024-06-06T04:28:02.331 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:28:02.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:28:02.337 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2024-06-06T04:28:02.337 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-06-06T04:28:02.353 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2024-06-06T04:28:02.353 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-06-06T04:28:02.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:28:02.353 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-06-06T04:28:02.420 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-06-06T04:28:02.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-06T04:28:02.532 INFO:teuthology.orchestra.run.smithi045.stdout:# file: . 2024-06-06T04:28:02.532 INFO:teuthology.orchestra.run.smithi045.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-06-06T04:28:02.532 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:28:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T04:28:02.538 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:28:02.538 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T04:28:02.556 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-06T04:28:02.846 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:03.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:03 smithi045 ceph-mon[32299]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T04:28:04.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:03 smithi086 ceph-mon[36053]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T04:28:04.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:03 smithi154 ceph-mon[36428]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T04:28:04.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.450+0000 7f221ad1c700 1 -- 172.21.15.45:0/3406688129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2214100610 msgr2=0x7f2214100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.450+0000 7f221ad1c700 1 --2- 172.21.15.45:0/3406688129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2214100610 0x7f2214100a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f22000099c0 tx=0x7f2200009cd0 comp rx=0 tx=0).stop 2024-06-06T04:28:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.450+0000 7f221ad1c700 1 -- 172.21.15.45:0/3406688129 shutdown_connections 2024-06-06T04:28:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.450+0000 7f221ad1c700 1 --2- 172.21.15.45:0/3406688129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2214100fb0 0x7f221410f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.450+0000 7f221ad1c700 1 --2- 172.21.15.45:0/3406688129 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22141086c0 0x7f2214108aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.450+0000 7f221ad1c700 1 --2- 172.21.15.45:0/3406688129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2214100610 0x7f2214100a70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.450+0000 7f221ad1c700 1 -- 172.21.15.45:0/3406688129 >> 172.21.15.45:0/3406688129 conn(0x7f2214077590 msgr2=0x7f22140fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.450+0000 7f221ad1c700 1 -- 172.21.15.45:0/3406688129 shutdown_connections 2024-06-06T04:28:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.450+0000 7f221ad1c700 1 -- 172.21.15.45:0/3406688129 wait complete. 2024-06-06T04:28:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.451+0000 7f221ad1c700 1 Processor -- start 2024-06-06T04:28:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.451+0000 7f221ad1c700 1 -- start start 2024-06-06T04:28:04.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.451+0000 7f221ad1c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2214100610 0x7f221419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.451+0000 7f221ad1c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2214100fb0 0x7f221419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.451+0000 7f221ad1c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22141086c0 0x7f22141988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.451+0000 7f221ad1c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2214112610 con 0x7f2214100610 2024-06-06T04:28:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.451+0000 7f221ad1c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2214112490 con 0x7f22141086c0 2024-06-06T04:28:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.451+0000 7f221ad1c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2214112790 con 0x7f2214100fb0 2024-06-06T04:28:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.451+0000 7f2218ab8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2214100610 0x7f221419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.451+0000 7f2218ab8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2214100610 0x7f221419e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47228/0 (socket says 172.21.15.45:47228) 2024-06-06T04:28:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.451+0000 7f2218ab8700 1 -- 172.21.15.45:0/41621196 learned_addr learned my addr 172.21.15.45:0/41621196 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.451+0000 7f2213fff700 1 --2- 172.21.15.45:0/41621196 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2214100fb0 0x7f221419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.451+0000 7f22192b9700 1 --2- 172.21.15.45:0/41621196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22141086c0 0x7f22141988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.452+0000 7f2218ab8700 1 -- 172.21.15.45:0/41621196 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2214100fb0 msgr2=0x7f221419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.452+0000 7f2218ab8700 1 --2- 172.21.15.45:0/41621196 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2214100fb0 0x7f221419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.452+0000 7f2218ab8700 1 -- 172.21.15.45:0/41621196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22141086c0 msgr2=0x7f22141988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.452+0000 7f2218ab8700 1 --2- 172.21.15.45:0/41621196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22141086c0 0x7f22141988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:04.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.452+0000 7f2218ab8700 1 -- 172.21.15.45:0/41621196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2214199050 con 0x7f2214100610 2024-06-06T04:28:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.452+0000 7f2218ab8700 1 --2- 172.21.15.45:0/41621196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2214100610 0x7f221419e6e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f220800d980 tx=0x7f220800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.452+0000 7f2211ffb700 1 -- 172.21.15.45:0/41621196 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208017070 con 0x7f2214100610 2024-06-06T04:28:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.452+0000 7f221ad1c700 1 -- 172.21.15.45:0/41621196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2214199340 con 0x7f2214100610 2024-06-06T04:28:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.452+0000 7f2211ffb700 1 -- 172.21.15.45:0/41621196 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f220800fd40 con 0x7f2214100610 2024-06-06T04:28:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.452+0000 7f2211ffb700 1 -- 172.21.15.45:0/41621196 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220800e3b0 con 0x7f2214100610 2024-06-06T04:28:04.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.452+0000 7f221ad1c700 1 -- 172.21.15.45:0/41621196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2214101fb0 con 0x7f2214100610 2024-06-06T04:28:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.453+0000 7f221ad1c700 1 -- 172.21.15.45:0/41621196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2214101990 con 0x7f2214100610 2024-06-06T04:28:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.455+0000 7f2211ffb700 1 -- 172.21.15.45:0/41621196 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f220800f890 con 0x7f2214100610 2024-06-06T04:28:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.455+0000 7f2211ffb700 1 --2- 172.21.15.45:0/41621196 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f22040a2ca0 0x7f22040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.455+0000 7f2213fff700 1 --2- 172.21.15.45:0/41621196 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f22040a2ca0 0x7f22040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.456+0000 7f2211ffb700 1 -- 172.21.15.45:0/41621196 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f22080ced10 con 0x7f2214100610 2024-06-06T04:28:04.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.456+0000 7f2213fff700 1 --2- 172.21.15.45:0/41621196 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f22040a2ca0 0x7f22040a5160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2214107df0 tx=0x7f2200005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:04.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.460+0000 7f2211ffb700 1 -- 172.21.15.45:0/41621196 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f220809ba30 con 0x7f2214100610 2024-06-06T04:28:04.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.681+0000 7f221ad1c700 1 -- 172.21.15.45:0/41621196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2214199e00 con 0x7f2214100610 2024-06-06T04:28:04.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.682+0000 7f2211ffb700 1 -- 172.21.15.45:0/41621196 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v96) ==== 81+0+589 (secure 0 0 0) 0x7f2214199e00 con 0x7f2214100610 2024-06-06T04:28:04.683 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/36732849 2024-06-07T04:20:04.267269+0000 2024-06-06T04:28:04.683 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1500318775 2024-06-07T04:20:04.267269+0000 2024-06-06T04:28:04.684 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/3465796147 2024-06-07T04:20:04.267269+0000 2024-06-06T04:28:04.684 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/3465796147 2024-06-07T04:20:04.267269+0000 2024-06-06T04:28:04.684 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2041380391 2024-06-07T04:19:37.026199+0000 2024-06-06T04:28:04.684 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2504003341 2024-06-07T04:19:37.026199+0000 2024-06-06T04:28:04.684 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3798552599 2024-06-07T04:20:04.267269+0000 2024-06-06T04:28:04.684 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/498952822 2024-06-07T04:19:37.026199+0000 2024-06-06T04:28:04.684 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/1562511056 2024-06-07T04:19:37.026199+0000 2024-06-06T04:28:04.684 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/1562511056 2024-06-07T04:19:37.026199+0000 2024-06-06T04:28:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.684+0000 7f221ad1c700 1 -- 172.21.15.45:0/41621196 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f22040a2ca0 msgr2=0x7f22040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.684+0000 7f221ad1c700 1 --2- 172.21.15.45:0/41621196 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f22040a2ca0 0x7f22040a5160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2214107df0 tx=0x7f2200005fb0 comp rx=0 tx=0).stop 2024-06-06T04:28:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.684+0000 7f221ad1c700 1 -- 172.21.15.45:0/41621196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2214100610 msgr2=0x7f221419e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.684+0000 7f221ad1c700 1 --2- 172.21.15.45:0/41621196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2214100610 0x7f221419e6e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f220800d980 tx=0x7f220800dc90 comp rx=0 tx=0).stop 2024-06-06T04:28:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.684+0000 7f221ad1c700 1 -- 172.21.15.45:0/41621196 shutdown_connections 2024-06-06T04:28:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.684+0000 7f221ad1c700 1 --2- 172.21.15.45:0/41621196 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f22040a2ca0 0x7f22040a5160 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.684+0000 7f221ad1c700 1 --2- 172.21.15.45:0/41621196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22141086c0 0x7f22141988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.684+0000 7f221ad1c700 1 --2- 172.21.15.45:0/41621196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2214100610 0x7f221419e6e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.684+0000 7f221ad1c700 1 --2- 172.21.15.45:0/41621196 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2214100fb0 0x7f221419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.684+0000 7f221ad1c700 1 -- 172.21.15.45:0/41621196 >> 172.21.15.45:0/41621196 conn(0x7f2214077590 msgr2=0x7f22140fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.684+0000 7f221ad1c700 1 -- 172.21.15.45:0/41621196 shutdown_connections 2024-06-06T04:28:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:04.684+0000 7f221ad1c700 1 -- 172.21.15.45:0/41621196 wait complete. 2024-06-06T04:28:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:listed 10 entries 2024-06-06T04:28:04.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-06-06T04:28:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-06-06T04:28:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]: dispatch 2024-06-06T04:28:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-06-06T04:28:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-06-06T04:28:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-06-06T04:28:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]: dispatch 2024-06-06T04:28:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-06-06T04:28:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-06-06T04:28:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:04 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/41621196' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T04:28:05.082 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-06-06T04:28:05.082 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-06-06T04:28:05.082 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]: dispatch 2024-06-06T04:28:05.082 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-06-06T04:28:05.082 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-06-06T04:28:05.082 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-06-06T04:28:05.082 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]: dispatch 2024-06-06T04:28:05.082 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-06-06T04:28:05.082 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-06-06T04:28:05.083 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:04 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/41621196' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T04:28:05.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-06-06T04:28:05.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-06-06T04:28:05.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]: dispatch 2024-06-06T04:28:05.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-06-06T04:28:05.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-06-06T04:28:05.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-06-06T04:28:05.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]: dispatch 2024-06-06T04:28:05.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-06-06T04:28:05.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-06-06T04:28:05.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:04 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/41621196' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T04:28:05.329 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:28:05.329 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T04:28:05.348 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-06T04:28:05.638 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:05 smithi045 ceph-mon[32299]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T04:28:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-06-06T04:28:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-06-06T04:28:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]': finished 2024-06-06T04:28:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-06-06T04:28:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-06-06T04:28:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-06-06T04:28:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]': finished 2024-06-06T04:28:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-06-06T04:28:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-06-06T04:28:05.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:05 smithi045 ceph-mon[32299]: osdmap e97: 12 total, 12 up, 12 in 2024-06-06T04:28:06.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:05 smithi086 ceph-mon[36053]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T04:28:06.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-06-06T04:28:06.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-06-06T04:28:06.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]': finished 2024-06-06T04:28:06.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-06-06T04:28:06.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-06-06T04:28:06.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-06-06T04:28:06.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]': finished 2024-06-06T04:28:06.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-06-06T04:28:06.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-06-06T04:28:06.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:05 smithi086 ceph-mon[36053]: osdmap e97: 12 total, 12 up, 12 in 2024-06-06T04:28:06.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:05 smithi154 ceph-mon[36428]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T04:28:06.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-06-06T04:28:06.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-06-06T04:28:06.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]': finished 2024-06-06T04:28:06.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-06-06T04:28:06.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-06-06T04:28:06.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-06-06T04:28:06.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]': finished 2024-06-06T04:28:06.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-06-06T04:28:06.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-06-06T04:28:06.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:05 smithi154 ceph-mon[36428]: osdmap e97: 12 total, 12 up, 12 in 2024-06-06T04:28:06.883 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:06 smithi045 ceph-mon[32299]: osdmap e98: 12 total, 12 up, 12 in 2024-06-06T04:28:07.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:06 smithi086 ceph-mon[36053]: osdmap e98: 12 total, 12 up, 12 in 2024-06-06T04:28:07.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:06 smithi154 ceph-mon[36428]: osdmap e98: 12 total, 12 up, 12 in 2024-06-06T04:28:07.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.453+0000 7f73e8688700 1 -- 172.21.15.45:0/1382071555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e0101d50 msgr2=0x7f73e010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:07.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.453+0000 7f73e8688700 1 --2- 172.21.15.45:0/1382071555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e0101d50 0x7f73e010e240 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f73d800b0a0 tx=0x7f73d800b3b0 comp rx=0 tx=0).stop 2024-06-06T04:28:07.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.453+0000 7f73e8688700 1 -- 172.21.15.45:0/1382071555 shutdown_connections 2024-06-06T04:28:07.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.453+0000 7f73e8688700 1 --2- 172.21.15.45:0/1382071555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e0101d50 0x7f73e010e240 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:07.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.453+0000 7f73e8688700 1 --2- 172.21.15.45:0/1382071555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73e0107400 0x7f73e01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:07.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.453+0000 7f73e8688700 1 --2- 172.21.15.45:0/1382071555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e0101390 0x7f73e0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:07.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.453+0000 7f73e8688700 1 -- 172.21.15.45:0/1382071555 >> 172.21.15.45:0/1382071555 conn(0x7f73e0078340 msgr2=0x7f73e0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:07.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.453+0000 7f73e8688700 1 -- 172.21.15.45:0/1382071555 shutdown_connections 2024-06-06T04:28:07.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.453+0000 7f73e8688700 1 -- 172.21.15.45:0/1382071555 wait complete. 2024-06-06T04:28:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.454+0000 7f73e8688700 1 Processor -- start 2024-06-06T04:28:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.454+0000 7f73e8688700 1 -- start start 2024-06-06T04:28:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e8688700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73e0101390 0x7f73e0199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e8688700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e0101d50 0x7f73e0199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e8688700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e0107400 0x7f73e01a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e8688700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0111520 con 0x7f73e0101d50 2024-06-06T04:28:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e8688700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01113a0 con 0x7f73e0107400 2024-06-06T04:28:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e8688700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01116a0 con 0x7f73e0101390 2024-06-06T04:28:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e5c23700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e0101d50 0x7f73e0199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e5c23700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e0101d50 0x7f73e0199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47258/0 (socket says 172.21.15.45:47258) 2024-06-06T04:28:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e5c23700 1 -- 172.21.15.45:0/72780531 learned_addr learned my addr 172.21.15.45:0/72780531 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e6c25700 1 --2- 172.21.15.45:0/72780531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e0107400 0x7f73e01a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e6424700 1 --2- 172.21.15.45:0/72780531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73e0101390 0x7f73e0199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e5c23700 1 -- 172.21.15.45:0/72780531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73e0101390 msgr2=0x7f73e0199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e5c23700 1 --2- 172.21.15.45:0/72780531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73e0101390 0x7f73e0199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e5c23700 1 -- 172.21.15.45:0/72780531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e0107400 msgr2=0x7f73e01a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:07.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e5c23700 1 --2- 172.21.15.45:0/72780531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e0107400 0x7f73e01a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:07.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e5c23700 1 -- 172.21.15.45:0/72780531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e01a4ec0 con 0x7f73e0101d50 2024-06-06T04:28:07.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.455+0000 7f73e5c23700 1 --2- 172.21.15.45:0/72780531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e0101d50 0x7f73e0199e40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f73d000c920 tx=0x7f73d000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:07.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.456+0000 7f73d77fe700 1 -- 172.21.15.45:0/72780531 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0004a30 con 0x7f73e0101d50 2024-06-06T04:28:07.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.456+0000 7f73e8688700 1 -- 172.21.15.45:0/72780531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e01a5090 con 0x7f73e0101d50 2024-06-06T04:28:07.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.456+0000 7f73e8688700 1 -- 172.21.15.45:0/72780531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e01a55d0 con 0x7f73e0101d50 2024-06-06T04:28:07.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.456+0000 7f73d77fe700 1 -- 172.21.15.45:0/72780531 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73d00054d0 con 0x7f73e0101d50 2024-06-06T04:28:07.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.456+0000 7f73d77fe700 1 -- 172.21.15.45:0/72780531 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0007bd0 con 0x7f73e0101d50 2024-06-06T04:28:07.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.457+0000 7f73e8688700 1 -- 172.21.15.45:0/72780531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73e0069180 con 0x7f73e0101d50 2024-06-06T04:28:07.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.459+0000 7f73d77fe700 1 -- 172.21.15.45:0/72780531 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f73d0005000 con 0x7f73e0101d50 2024-06-06T04:28:07.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.459+0000 7f73d77fe700 1 --2- 172.21.15.45:0/72780531 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73cc0a2c20 0x7f73cc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:07.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.459+0000 7f73e6424700 1 --2- 172.21.15.45:0/72780531 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73cc0a2c20 0x7f73cc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:07.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.459+0000 7f73d77fe700 1 -- 172.21.15.45:0/72780531 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7f73d00cee70 con 0x7f73e0101d50 2024-06-06T04:28:07.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.460+0000 7f73e6424700 1 --2- 172.21.15.45:0/72780531 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73cc0a2c20 0x7f73cc0a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00abd0 tx=0x7f73dc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.464+0000 7f73d77fe700 1 -- 172.21.15.45:0/72780531 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f73d0014030 con 0x7f73e0101d50 2024-06-06T04:28:07.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.686+0000 7f73e8688700 1 -- 172.21.15.45:0/72780531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f73e006a730 con 0x7f73e0101d50 2024-06-06T04:28:07.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.686+0000 7f73d77fe700 1 -- 172.21.15.45:0/72780531 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v98) ==== 81+0+589 (secure 0 0 0) 0x7f73d00052f0 con 0x7f73e0101d50 2024-06-06T04:28:07.687 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/36732849 2024-06-07T04:20:04.267269+0000 2024-06-06T04:28:07.687 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1500318775 2024-06-07T04:20:04.267269+0000 2024-06-06T04:28:07.687 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/3465796147 2024-06-07T04:20:04.267269+0000 2024-06-06T04:28:07.687 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/3465796147 2024-06-07T04:20:04.267269+0000 2024-06-06T04:28:07.687 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2041380391 2024-06-07T04:19:37.026199+0000 2024-06-06T04:28:07.687 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2504003341 2024-06-07T04:19:37.026199+0000 2024-06-06T04:28:07.687 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3798552599 2024-06-07T04:20:04.267269+0000 2024-06-06T04:28:07.688 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/498952822 2024-06-07T04:19:37.026199+0000 2024-06-06T04:28:07.688 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/1562511056 2024-06-07T04:19:37.026199+0000 2024-06-06T04:28:07.688 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/1562511056 2024-06-07T04:19:37.026199+0000 2024-06-06T04:28:07.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.688+0000 7f73e8688700 1 -- 172.21.15.45:0/72780531 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73cc0a2c20 msgr2=0x7f73cc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:07.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.688+0000 7f73e8688700 1 --2- 172.21.15.45:0/72780531 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73cc0a2c20 0x7f73cc0a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00abd0 tx=0x7f73dc00a380 comp rx=0 tx=0).stop 2024-06-06T04:28:07.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.688+0000 7f73e8688700 1 -- 172.21.15.45:0/72780531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e0101d50 msgr2=0x7f73e0199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:07.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.688+0000 7f73e8688700 1 --2- 172.21.15.45:0/72780531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e0101d50 0x7f73e0199e40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f73d000c920 tx=0x7f73d000cc30 comp rx=0 tx=0).stop 2024-06-06T04:28:07.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.688+0000 7f73e8688700 1 -- 172.21.15.45:0/72780531 shutdown_connections 2024-06-06T04:28:07.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.688+0000 7f73e8688700 1 --2- 172.21.15.45:0/72780531 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73cc0a2c20 0x7f73cc0a50e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:07.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.689+0000 7f73e8688700 1 --2- 172.21.15.45:0/72780531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e0107400 0x7f73e01a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:07.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.689+0000 7f73e8688700 1 --2- 172.21.15.45:0/72780531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73e0101390 0x7f73e0199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:07.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.689+0000 7f73e8688700 1 --2- 172.21.15.45:0/72780531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e0101d50 0x7f73e0199e40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:07.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.689+0000 7f73e8688700 1 -- 172.21.15.45:0/72780531 >> 172.21.15.45:0/72780531 conn(0x7f73e0078340 msgr2=0x7f73e00ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:07.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.689+0000 7f73e8688700 1 -- 172.21.15.45:0/72780531 shutdown_connections 2024-06-06T04:28:07.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:07.689+0000 7f73e8688700 1 -- 172.21.15.45:0/72780531 wait complete. 2024-06-06T04:28:07.690 INFO:teuthology.orchestra.run.smithi045.stderr:listed 10 entries 2024-06-06T04:28:08.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:07 smithi045 ceph-mon[32299]: pgmap v329: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-06T04:28:08.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:07 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/72780531' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T04:28:08.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:07 smithi086 ceph-mon[36053]: pgmap v329: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-06T04:28:08.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:07 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/72780531' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T04:28:08.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:07 smithi154 ceph-mon[36428]: pgmap v329: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-06T04:28:08.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:07 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/72780531' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T04:28:08.358 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2024-06-06T04:28:08.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:28:08.358 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-06-06T04:28:08.387 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[78548]: fuse finished with error 0 and tester_r 0 2024-06-06T04:28:08.389 INFO:teuthology.orchestra.run:waiting for 300 2024-06-06T04:28:09.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:09 smithi045 ceph-mon[32299]: pgmap v330: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:10.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:09 smithi086 ceph-mon[36053]: pgmap v330: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:10.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:09 smithi154 ceph-mon[36428]: pgmap v330: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:12.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:11 smithi086 ceph-mon[36053]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:12.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:11 smithi154 ceph-mon[36428]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:12.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:11 smithi045 ceph-mon[32299]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:14.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:13 smithi086 ceph-mon[36053]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:14.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:13 smithi154 ceph-mon[36428]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:14.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:13 smithi045 ceph-mon[32299]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:14.491 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-06-06T04:28:14.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:28:14.492 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-06-06T04:28:14.528 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T04:28:14.828 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:16.078 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:15 smithi045 ceph-mon[32299]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 318 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:16.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:15 smithi086 ceph-mon[36053]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 318 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:16.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:15 smithi154 ceph-mon[36428]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 318 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:16.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.477+0000 7f89d8942700 1 -- 172.21.15.45:0/2953620458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89d4107760 msgr2=0x7f89d410f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:16.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.477+0000 7f89d8942700 1 --2- 172.21.15.45:0/2953620458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89d4107760 0x7f89d410f370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f89cc00b0d0 tx=0x7f89cc00b3e0 comp rx=0 tx=0).stop 2024-06-06T04:28:16.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.477+0000 7f89d8942700 1 -- 172.21.15.45:0/2953620458 shutdown_connections 2024-06-06T04:28:16.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.477+0000 7f89d8942700 1 --2- 172.21.15.45:0/2953620458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89d4107760 0x7f89d410f370 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:16.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.477+0000 7f89d8942700 1 --2- 172.21.15.45:0/2953620458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d4102770 0x7f89d4102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:16.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.477+0000 7f89d8942700 1 --2- 172.21.15.45:0/2953620458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4103120 0x7f89d4107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:16.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.477+0000 7f89d8942700 1 -- 172.21.15.45:0/2953620458 >> 172.21.15.45:0/2953620458 conn(0x7f89d4079560 msgr2=0x7f89d4079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:16.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.477+0000 7f89d8942700 1 -- 172.21.15.45:0/2953620458 shutdown_connections 2024-06-06T04:28:16.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.477+0000 7f89d8942700 1 -- 172.21.15.45:0/2953620458 wait complete. 2024-06-06T04:28:16.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.478+0000 7f89d8942700 1 Processor -- start 2024-06-06T04:28:16.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.478+0000 7f89d8942700 1 -- start start 2024-06-06T04:28:16.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d8942700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4102770 0x7f89d419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:16.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d8942700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89d4103120 0x7f89d419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:16.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d8942700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d4107760 0x7f89d4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:16.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d8942700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f89d4112690 con 0x7f89d4103120 2024-06-06T04:28:16.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d8942700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f89d4112510 con 0x7f89d4102770 2024-06-06T04:28:16.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d8942700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f89d4112810 con 0x7f89d4107760 2024-06-06T04:28:16.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89d4103120 0x7f89d419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:16.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4102770 0x7f89d419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:16.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89d4103120 0x7f89d419ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41966/0 (socket says 172.21.15.45:41966) 2024-06-06T04:28:16.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4102770 0x7f89d419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:46596/0 (socket says 172.21.15.45:46596) 2024-06-06T04:28:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d1d9b700 1 -- 172.21.15.45:0/3311668128 learned_addr learned my addr 172.21.15.45:0/3311668128 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d2d9d700 1 --2- 172.21.15.45:0/3311668128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d4107760 0x7f89d4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d1d9b700 1 -- 172.21.15.45:0/3311668128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d4107760 msgr2=0x7f89d4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d1d9b700 1 --2- 172.21.15.45:0/3311668128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d4107760 0x7f89d4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d1d9b700 1 -- 172.21.15.45:0/3311668128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4102770 msgr2=0x7f89d419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d1d9b700 1 --2- 172.21.15.45:0/3311668128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4102770 0x7f89d419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d1d9b700 1 -- 172.21.15.45:0/3311668128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d41990d0 con 0x7f89d4103120 2024-06-06T04:28:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d259c700 1 --2- 172.21.15.45:0/3311668128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4102770 0x7f89d419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:28:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89d1d9b700 1 --2- 172.21.15.45:0/3311668128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89d4103120 0x7f89d419ecf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f89c800d980 tx=0x7f89c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.479+0000 7f89c37fe700 1 -- 172.21.15.45:0/3311668128 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8017070 con 0x7f89d4103120 2024-06-06T04:28:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.480+0000 7f89c37fe700 1 -- 172.21.15.45:0/3311668128 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89c8009de0 con 0x7f89d4103120 2024-06-06T04:28:16.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.480+0000 7f89d8942700 1 -- 172.21.15.45:0/3311668128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d41993c0 con 0x7f89d4103120 2024-06-06T04:28:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.480+0000 7f89c37fe700 1 -- 172.21.15.45:0/3311668128 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c800f3c0 con 0x7f89d4103120 2024-06-06T04:28:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.480+0000 7f89d8942700 1 -- 172.21.15.45:0/3311668128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d40746c0 con 0x7f89d4103120 2024-06-06T04:28:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.481+0000 7f89d8942700 1 -- 172.21.15.45:0/3311668128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d4069180 con 0x7f89d4103120 2024-06-06T04:28:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.482+0000 7f89c37fe700 1 -- 172.21.15.45:0/3311668128 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f89c8009930 con 0x7f89d4103120 2024-06-06T04:28:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.482+0000 7f89c37fe700 1 --2- 172.21.15.45:0/3311668128 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f89bc0a2c20 0x7f89bc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.483+0000 7f89c37fe700 1 -- 172.21.15.45:0/3311668128 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7f89c80cee40 con 0x7f89d4103120 2024-06-06T04:28:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.483+0000 7f89d259c700 1 --2- 172.21.15.45:0/3311668128 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f89bc0a2c20 0x7f89bc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.483+0000 7f89d259c700 1 --2- 172.21.15.45:0/3311668128 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f89bc0a2c20 0x7f89bc0a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f89d4199ab0 tx=0x7f89c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:16.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.487+0000 7f89c37fe700 1 -- 172.21.15.45:0/3311668128 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f89c809ba60 con 0x7f89d4103120 2024-06-06T04:28:16.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.691+0000 7f89d8942700 1 -- 172.21.15.45:0/3311668128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89d402de60 con 0x7f89d4103120 2024-06-06T04:28:16.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.693+0000 7f89c37fe700 1 -- 172.21.15.45:0/3311668128 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24602 (secure 0 0 0) 0x7f89c809b360 con 0x7f89d4103120 2024-06-06T04:28:16.694 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:28:16.694 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":98,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","created":"2024-06-06T04:19:02.626916+0000","modified":"2024-06-06T04:28:05.713633+0000","last_up_change":"2024-06-06T04:26:17.390429+0000","last_in_change":"2024-06-06T04:25:56.938388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T04:22:28.797137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T04:27:17.750625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T04:27:21.486917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T04:27:31.029818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d3cee4a-5025-45b3-b217-082af37e79db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6803","nonce":2111759250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6805","nonce":2111759250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6809","nonce":2111759250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6807","nonce":2111759250}]},"public_addr":"172.21.15.45:6803/2111759250","cluster_addr":"172.21.15.45:6805/2111759250","heartbeat_back_addr":"172.21.15.45:6809/2111759250","heartbeat_front_addr":"172.21.15.45:6807/2111759250","state":["exists","up"]},{"osd":1,"uuid":"75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6801","nonce":3732225287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6803","nonce":3732225287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6807","nonce":3732225287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6805","nonce":3732225287}]},"public_addr":"172.21.15.86:6801/3732225287","cluster_addr":"172.21.15.86:6803/3732225287","heartbeat_back_addr":"172.21.15.86:6807/3732225287","heartbeat_front_addr":"172.21.15.86:6805/3732225287","state":["exists","up"]},{"osd":2,"uuid":"bb20565f-f9ae-47e3-9604-c69f737cff4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6801","nonce":2898783039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6803","nonce":2898783039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6807","nonce":2898783039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6805","nonce":2898783039}]},"public_addr":"172.21.15.154:6801/2898783039","cluster_addr":"172.21.15.154:6803/2898783039","heartbeat_back_addr":"172.21.15.154:6807/2898783039","heartbeat_front_addr":"172.21.15.154:6805/2898783039","state":["exists","up"]},{"osd":3,"uuid":"e1160799-7811-4326-9792-7e6c7d189c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6811","nonce":3436753135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6813","nonce":3436753135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6817","nonce":3436753135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6815","nonce":3436753135}]},"public_addr":"172.21.15.45:6811/3436753135","cluster_addr":"172.21.15.45:6813/3436753135","heartbeat_back_addr":"172.21.15.45:6817/3436753135","heartbeat_front_addr":"172.21.15.45:6815/3436753135","state":["exists","up"]},{"osd":4,"uuid":"6b389bc8-e19a-47f3-83ff-7b782d146d0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6809","nonce":3030235796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6811","nonce":3030235796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6815","nonce":3030235796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6813","nonce":3030235796}]},"public_addr":"172.21.15.86:6809/3030235796","cluster_addr":"172.21.15.86:6811/3030235796","heartbeat_back_addr":"172.21.15.86:6815/3030235796","heartbeat_front_addr":"172.21.15.86:6813/3030235796","state":["exists","up"]},{"osd":5,"uuid":"069de7b2-89bf-4150-bed7-7b8832573dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6809","nonce":2222268646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6811","nonce":2222268646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6815","nonce":2222268646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6813","nonce":2222268646}]},"public_addr":"172.21.15.154:6809/2222268646","cluster_addr":"172.21.15.154:6811/2222268646","heartbeat_back_addr":"172.21.15.154:6815/2222268646","heartbeat_front_addr":"172.21.15.154:6813/2222268646","state":["exists","up"]},{"osd":6,"uuid":"4c70b627-eafa-4e27-ba2a-4d8e1e335962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6819","nonce":384734939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6821","nonce":384734939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6825","nonce":384734939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6823","nonce":384734939}]},"public_addr":"172.21.15.45:6819/384734939","cluster_addr":"172.21.15.45:6821/384734939","heartbeat_back_addr":"172.21.15.45:6825/384734939","heartbeat_front_addr":"172.21.15.45:6823/384734939","state":["exists","up"]},{"osd":7,"uuid":"3b09b875-be52-4be0-b116-7fd9b0c26705","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6817","nonce":1119791005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6819","nonce":1119791005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6823","nonce":1119791005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6821","nonce":1119791005}]},"public_addr":"172.21.15.86:6817/1119791005","cluster_addr":"172.21.15.86:6819/1119791005","heartbeat_back_addr":"172.21.15.86:6823/1119791005","heartbeat_front_addr":"172.21.15.86:6821/1119791005","state":["exists","up"]},{"osd":8,"uuid":"cdf1dd1b-5392-4836-8e65-4c7c5b688d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6817","nonce":3054812496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6819","nonce":3054812496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6823","nonce":3054812496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6821","nonce":3054812496}]},"public_addr":"172.21.15.154:6817/3054812496","cluster_addr":"172.21.15.154:6819/3054812496","heartbeat_back_addr":"172.21.15.154:6823/3054812496","heartbeat_front_addr":"172.21.15.154:6821/3054812496","state":["exists","up"]},{"osd":9,"uuid":"acede92d-2cc1-4967-a357-7e5e201e99d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6827","nonce":1360655510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6829","nonce":1360655510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6833","nonce":1360655510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6831","nonce":1360655510}]},"public_addr":"172.21.15.45:6827/1360655510","cluster_addr":"172.21.15.45:6829/1360655510","heartbeat_back_addr":"172.21.15.45:6833/1360655510","heartbeat_front_addr":"172.21.15.45:6831/1360655510","state":["exists","up"]},{"osd":10,"uuid":"a2d0c488-1892-4da4-adbf-952c22df0ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6825","nonce":1701691877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6827","nonce":1701691877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6831","nonce":1701691877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6829","nonce":1701691877}]},"public_addr":"172.21.15.86:6825/1701691877","cluster_addr":"172.21.15.86:6827/1701691877","heartbeat_back_addr":"172.21.15.86:6831/1701691877","heartbeat_front_addr":"172.21.15.86:6829/1701691877","state":["exists","up"]},{"osd":11,"uuid":"6b349c4c-b672-480b-ac6a-8594dab0eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6825","nonce":2065648007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6827","nonce":2065648007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6831","nonce":2065648007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6829","nonce":2065648007}]},"public_addr":"172.21.15.154:6825/2065648007","cluster_addr":"172.21.15.154:6827/2065648007","heartbeat_back_addr":"172.21.15.154:6831/2065648007","heartbeat_front_addr":"172.21.15.154:6829/2065648007","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:21:37.147394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:00.776617+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:24.786849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:50.487455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:14.869272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:39.815158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:05.972619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:31.410063+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:56.795119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:23.114884+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:49.256504+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:26:15.251967+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/36732849":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/1500318775":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6801/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6800/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/2041380391":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/2504003341":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/3798552599":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/498952822":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6800/1562511056":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6801/1562511056":"2024-06-07T04:19:37.026199+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T04:28:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.696+0000 7f89d8942700 1 -- 172.21.15.45:0/3311668128 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f89bc0a2c20 msgr2=0x7f89bc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.696+0000 7f89d8942700 1 --2- 172.21.15.45:0/3311668128 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f89bc0a2c20 0x7f89bc0a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f89d4199ab0 tx=0x7f89c400b410 comp rx=0 tx=0).stop 2024-06-06T04:28:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.696+0000 7f89d8942700 1 -- 172.21.15.45:0/3311668128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89d4103120 msgr2=0x7f89d419ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.696+0000 7f89d8942700 1 --2- 172.21.15.45:0/3311668128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89d4103120 0x7f89d419ecf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f89c800d980 tx=0x7f89c800dc90 comp rx=0 tx=0).stop 2024-06-06T04:28:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.696+0000 7f89d8942700 1 -- 172.21.15.45:0/3311668128 shutdown_connections 2024-06-06T04:28:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.696+0000 7f89d8942700 1 --2- 172.21.15.45:0/3311668128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d4107760 0x7f89d4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.696+0000 7f89d8942700 1 --2- 172.21.15.45:0/3311668128 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f89bc0a2c20 0x7f89bc0a50e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:16.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.696+0000 7f89d8942700 1 --2- 172.21.15.45:0/3311668128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89d4102770 0x7f89d419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.696+0000 7f89d8942700 1 --2- 172.21.15.45:0/3311668128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89d4103120 0x7f89d419ecf0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.696+0000 7f89d8942700 1 -- 172.21.15.45:0/3311668128 >> 172.21.15.45:0/3311668128 conn(0x7f89d4079560 msgr2=0x7f89d41005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.696+0000 7f89d8942700 1 -- 172.21.15.45:0/3311668128 shutdown_connections 2024-06-06T04:28:16.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:16.696+0000 7f89d8942700 1 -- 172.21.15.45:0/3311668128 wait complete. 2024-06-06T04:28:16.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:16 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3311668128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:28:17.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:16 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3311668128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:28:17.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:16 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3311668128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:28:17.328 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T04:28:17.577 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:17 smithi045 ceph-mon[32299]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 288 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:18.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:17 smithi086 ceph-mon[36053]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 288 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:18.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:17 smithi154 ceph-mon[36428]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 288 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:18.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:18 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:28:19.009 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:18 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:28:19.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:18 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:28:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.250+0000 7fa0feb4f700 1 -- 172.21.15.45:0/1866275013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0f806ab30 msgr2=0x7fa0f810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.250+0000 7fa0feb4f700 1 --2- 172.21.15.45:0/1866275013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0f806ab30 0x7fa0f810f320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000b0a0 tx=0x7fa0f000b3b0 comp rx=0 tx=0).stop 2024-06-06T04:28:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.250+0000 7fa0feb4f700 1 -- 172.21.15.45:0/1866275013 shutdown_connections 2024-06-06T04:28:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.250+0000 7fa0feb4f700 1 --2- 172.21.15.45:0/1866275013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0f806ab30 0x7fa0f810f320 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.250+0000 7fa0feb4f700 1 --2- 172.21.15.45:0/1866275013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f8108500 0x7fa0f81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.250+0000 7fa0feb4f700 1 --2- 172.21.15.45:0/1866275013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0f806a170 0x7fa0f806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.250+0000 7fa0feb4f700 1 -- 172.21.15.45:0/1866275013 >> 172.21.15.45:0/1866275013 conn(0x7fa0f8078340 msgr2=0x7fa0f8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.250+0000 7fa0feb4f700 1 -- 172.21.15.45:0/1866275013 shutdown_connections 2024-06-06T04:28:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.250+0000 7fa0feb4f700 1 -- 172.21.15.45:0/1866275013 wait complete. 2024-06-06T04:28:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.251+0000 7fa0feb4f700 1 Processor -- start 2024-06-06T04:28:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.251+0000 7fa0feb4f700 1 -- start start 2024-06-06T04:28:19.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.251+0000 7fa0feb4f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0f806a170 0x7fa0f819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:19.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.251+0000 7fa0feb4f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f806ab30 0x7fa0f819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:19.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.251+0000 7fa0feb4f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0f8108500 0x7fa0f8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:19.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.251+0000 7fa0feb4f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f8112630 con 0x7fa0f8108500 2024-06-06T04:28:19.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.251+0000 7fa0feb4f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f81124b0 con 0x7fa0f806a170 2024-06-06T04:28:19.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.251+0000 7fa0feb4f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f81127b0 con 0x7fa0f806ab30 2024-06-06T04:28:19.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.251+0000 7fa0fd0ec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0f8108500 0x7fa0f8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:19.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.252+0000 7fa0fd0ec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0f8108500 0x7fa0f8198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46212/0 (socket says 172.21.15.45:46212) 2024-06-06T04:28:19.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.252+0000 7fa0fd0ec700 1 -- 172.21.15.45:0/4131608891 learned_addr learned my addr 172.21.15.45:0/4131608891 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:19.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.252+0000 7fa0fc8eb700 1 --2- 172.21.15.45:0/4131608891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0f806a170 0x7fa0f819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:19.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.252+0000 7fa0f7fff700 1 --2- 172.21.15.45:0/4131608891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f806ab30 0x7fa0f819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:19.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.252+0000 7fa0fd0ec700 1 -- 172.21.15.45:0/4131608891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f806ab30 msgr2=0x7fa0f819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:19.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.252+0000 7fa0fd0ec700 1 --2- 172.21.15.45:0/4131608891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f806ab30 0x7fa0f819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:19.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.252+0000 7fa0fd0ec700 1 -- 172.21.15.45:0/4131608891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0f806a170 msgr2=0x7fa0f819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:19.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.252+0000 7fa0fd0ec700 1 --2- 172.21.15.45:0/4131608891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0f806a170 0x7fa0f819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:19.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.252+0000 7fa0fd0ec700 1 -- 172.21.15.45:0/4131608891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f81990d0 con 0x7fa0f8108500 2024-06-06T04:28:19.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.252+0000 7fa0f7fff700 1 --2- 172.21.15.45:0/4131608891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f806ab30 0x7fa0f819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:28:19.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.252+0000 7fa0fc8eb700 1 --2- 172.21.15.45:0/4131608891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0f806a170 0x7fa0f819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:28:19.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.252+0000 7fa0fd0ec700 1 --2- 172.21.15.45:0/4131608891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0f8108500 0x7fa0f8198970 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000b780 tx=0x7fa0f0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:19.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.252+0000 7fa0f5ffb700 1 -- 172.21.15.45:0/4131608891 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0021070 con 0x7fa0f8108500 2024-06-06T04:28:19.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.252+0000 7fa0feb4f700 1 -- 172.21.15.45:0/4131608891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f8199360 con 0x7fa0f8108500 2024-06-06T04:28:19.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.252+0000 7fa0f5ffb700 1 -- 172.21.15.45:0/4131608891 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0f0007e30 con 0x7fa0f8108500 2024-06-06T04:28:19.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.252+0000 7fa0f5ffb700 1 -- 172.21.15.45:0/4131608891 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0004800 con 0x7fa0f8108500 2024-06-06T04:28:19.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.253+0000 7fa0feb4f700 1 -- 172.21.15.45:0/4131608891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f81a5680 con 0x7fa0f8108500 2024-06-06T04:28:19.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.254+0000 7fa0feb4f700 1 -- 172.21.15.45:0/4131608891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f810a9e0 con 0x7fa0f8108500 2024-06-06T04:28:19.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.255+0000 7fa0f5ffb700 1 -- 172.21.15.45:0/4131608891 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa0f0007980 con 0x7fa0f8108500 2024-06-06T04:28:19.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.256+0000 7fa0f5ffb700 1 --2- 172.21.15.45:0/4131608891 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa0e80a2ca0 0x7fa0e80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:19.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.256+0000 7fa0f5ffb700 1 -- 172.21.15.45:0/4131608891 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7fa0f000f020 con 0x7fa0f8108500 2024-06-06T04:28:19.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.256+0000 7fa0fc8eb700 1 --2- 172.21.15.45:0/4131608891 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa0e80a2ca0 0x7fa0e80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:19.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.257+0000 7fa0fc8eb700 1 --2- 172.21.15.45:0/4131608891 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa0e80a2ca0 0x7fa0e80a5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8199ab0 tx=0x7fa0ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.260+0000 7fa0f5ffb700 1 -- 172.21.15.45:0/4131608891 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa0f002a020 con 0x7fa0f8108500 2024-06-06T04:28:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.484+0000 7fa0feb4f700 1 -- 172.21.15.45:0/4131608891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0f8199ea0 con 0x7fa0f8108500 2024-06-06T04:28:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.485+0000 7fa0f5ffb700 1 -- 172.21.15.45:0/4131608891 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24602 (secure 0 0 0) 0x7fa0f009d8c0 con 0x7fa0f8108500 2024-06-06T04:28:19.486 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:28:19.487 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":98,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","created":"2024-06-06T04:19:02.626916+0000","modified":"2024-06-06T04:28:05.713633+0000","last_up_change":"2024-06-06T04:26:17.390429+0000","last_in_change":"2024-06-06T04:25:56.938388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T04:22:28.797137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T04:27:17.750625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T04:27:21.486917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T04:27:31.029818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d3cee4a-5025-45b3-b217-082af37e79db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6803","nonce":2111759250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6805","nonce":2111759250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6809","nonce":2111759250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6807","nonce":2111759250}]},"public_addr":"172.21.15.45:6803/2111759250","cluster_addr":"172.21.15.45:6805/2111759250","heartbeat_back_addr":"172.21.15.45:6809/2111759250","heartbeat_front_addr":"172.21.15.45:6807/2111759250","state":["exists","up"]},{"osd":1,"uuid":"75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6801","nonce":3732225287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6803","nonce":3732225287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6807","nonce":3732225287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6805","nonce":3732225287}]},"public_addr":"172.21.15.86:6801/3732225287","cluster_addr":"172.21.15.86:6803/3732225287","heartbeat_back_addr":"172.21.15.86:6807/3732225287","heartbeat_front_addr":"172.21.15.86:6805/3732225287","state":["exists","up"]},{"osd":2,"uuid":"bb20565f-f9ae-47e3-9604-c69f737cff4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6801","nonce":2898783039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6803","nonce":2898783039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6807","nonce":2898783039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6805","nonce":2898783039}]},"public_addr":"172.21.15.154:6801/2898783039","cluster_addr":"172.21.15.154:6803/2898783039","heartbeat_back_addr":"172.21.15.154:6807/2898783039","heartbeat_front_addr":"172.21.15.154:6805/2898783039","state":["exists","up"]},{"osd":3,"uuid":"e1160799-7811-4326-9792-7e6c7d189c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6811","nonce":3436753135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6813","nonce":3436753135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6817","nonce":3436753135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6815","nonce":3436753135}]},"public_addr":"172.21.15.45:6811/3436753135","cluster_addr":"172.21.15.45:6813/3436753135","heartbeat_back_addr":"172.21.15.45:6817/3436753135","heartbeat_front_addr":"172.21.15.45:6815/3436753135","state":["exists","up"]},{"osd":4,"uuid":"6b389bc8-e19a-47f3-83ff-7b782d146d0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6809","nonce":3030235796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6811","nonce":3030235796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6815","nonce":3030235796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6813","nonce":3030235796}]},"public_addr":"172.21.15.86:6809/3030235796","cluster_addr":"172.21.15.86:6811/3030235796","heartbeat_back_addr":"172.21.15.86:6815/3030235796","heartbeat_front_addr":"172.21.15.86:6813/3030235796","state":["exists","up"]},{"osd":5,"uuid":"069de7b2-89bf-4150-bed7-7b8832573dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6809","nonce":2222268646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6811","nonce":2222268646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6815","nonce":2222268646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6813","nonce":2222268646}]},"public_addr":"172.21.15.154:6809/2222268646","cluster_addr":"172.21.15.154:6811/2222268646","heartbeat_back_addr":"172.21.15.154:6815/2222268646","heartbeat_front_addr":"172.21.15.154:6813/2222268646","state":["exists","up"]},{"osd":6,"uuid":"4c70b627-eafa-4e27-ba2a-4d8e1e335962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6819","nonce":384734939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6821","nonce":384734939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6825","nonce":384734939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6823","nonce":384734939}]},"public_addr":"172.21.15.45:6819/384734939","cluster_addr":"172.21.15.45:6821/384734939","heartbeat_back_addr":"172.21.15.45:6825/384734939","heartbeat_front_addr":"172.21.15.45:6823/384734939","state":["exists","up"]},{"osd":7,"uuid":"3b09b875-be52-4be0-b116-7fd9b0c26705","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6817","nonce":1119791005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6819","nonce":1119791005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6823","nonce":1119791005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6821","nonce":1119791005}]},"public_addr":"172.21.15.86:6817/1119791005","cluster_addr":"172.21.15.86:6819/1119791005","heartbeat_back_addr":"172.21.15.86:6823/1119791005","heartbeat_front_addr":"172.21.15.86:6821/1119791005","state":["exists","up"]},{"osd":8,"uuid":"cdf1dd1b-5392-4836-8e65-4c7c5b688d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6817","nonce":3054812496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6819","nonce":3054812496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6823","nonce":3054812496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6821","nonce":3054812496}]},"public_addr":"172.21.15.154:6817/3054812496","cluster_addr":"172.21.15.154:6819/3054812496","heartbeat_back_addr":"172.21.15.154:6823/3054812496","heartbeat_front_addr":"172.21.15.154:6821/3054812496","state":["exists","up"]},{"osd":9,"uuid":"acede92d-2cc1-4967-a357-7e5e201e99d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6827","nonce":1360655510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6829","nonce":1360655510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6833","nonce":1360655510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6831","nonce":1360655510}]},"public_addr":"172.21.15.45:6827/1360655510","cluster_addr":"172.21.15.45:6829/1360655510","heartbeat_back_addr":"172.21.15.45:6833/1360655510","heartbeat_front_addr":"172.21.15.45:6831/1360655510","state":["exists","up"]},{"osd":10,"uuid":"a2d0c488-1892-4da4-adbf-952c22df0ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6825","nonce":1701691877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6827","nonce":1701691877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6831","nonce":1701691877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6829","nonce":1701691877}]},"public_addr":"172.21.15.86:6825/1701691877","cluster_addr":"172.21.15.86:6827/1701691877","heartbeat_back_addr":"172.21.15.86:6831/1701691877","heartbeat_front_addr":"172.21.15.86:6829/1701691877","state":["exists","up"]},{"osd":11,"uuid":"6b349c4c-b672-480b-ac6a-8594dab0eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6825","nonce":2065648007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6827","nonce":2065648007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6831","nonce":2065648007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6829","nonce":2065648007}]},"public_addr":"172.21.15.154:6825/2065648007","cluster_addr":"172.21.15.154:6827/2065648007","heartbeat_back_addr":"172.21.15.154:6831/2065648007","heartbeat_front_addr":"172.21.15.154:6829/2065648007","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:21:37.147394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:00.776617+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:24.786849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:50.487455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:14.869272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:39.815158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:05.972619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:31.410063+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:56.795119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:23.114884+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:49.256504+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:26:15.251967+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/36732849":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/1500318775":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6801/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6800/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/2041380391":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/2504003341":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/3798552599":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/498952822":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6800/1562511056":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6801/1562511056":"2024-06-07T04:19:37.026199+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T04:28:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.488+0000 7fa0feb4f700 1 -- 172.21.15.45:0/4131608891 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa0e80a2ca0 msgr2=0x7fa0e80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.488+0000 7fa0feb4f700 1 --2- 172.21.15.45:0/4131608891 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa0e80a2ca0 0x7fa0e80a5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8199ab0 tx=0x7fa0ec009260 comp rx=0 tx=0).stop 2024-06-06T04:28:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.488+0000 7fa0feb4f700 1 -- 172.21.15.45:0/4131608891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0f8108500 msgr2=0x7fa0f8198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.488+0000 7fa0feb4f700 1 --2- 172.21.15.45:0/4131608891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0f8108500 0x7fa0f8198970 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000b780 tx=0x7fa0f0004d00 comp rx=0 tx=0).stop 2024-06-06T04:28:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.488+0000 7fa0feb4f700 1 -- 172.21.15.45:0/4131608891 shutdown_connections 2024-06-06T04:28:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.488+0000 7fa0feb4f700 1 --2- 172.21.15.45:0/4131608891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0f8108500 0x7fa0f8198970 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.488+0000 7fa0feb4f700 1 --2- 172.21.15.45:0/4131608891 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa0e80a2ca0 0x7fa0e80a5160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.488+0000 7fa0feb4f700 1 --2- 172.21.15.45:0/4131608891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0f806a170 0x7fa0f819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.488+0000 7fa0feb4f700 1 --2- 172.21.15.45:0/4131608891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f806ab30 0x7fa0f819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.488+0000 7fa0feb4f700 1 -- 172.21.15.45:0/4131608891 >> 172.21.15.45:0/4131608891 conn(0x7fa0f8078340 msgr2=0x7fa0f81000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.488+0000 7fa0feb4f700 1 -- 172.21.15.45:0/4131608891 shutdown_connections 2024-06-06T04:28:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:19.488+0000 7fa0feb4f700 1 -- 172.21.15.45:0/4131608891 wait complete. 2024-06-06T04:28:20.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:19 smithi045 ceph-mon[32299]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:20.067 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:19 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/4131608891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:28:20.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:19 smithi086 ceph-mon[36053]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:20.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:19 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/4131608891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:28:20.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:19 smithi154 ceph-mon[36428]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:20.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:19 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/4131608891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:28:20.464 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-06-06T04:28:20.926 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:21.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:21 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:21.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:21 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:21.302 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:21 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:21.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:21 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:21.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:21 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:21.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:21 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:21.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:21 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:21.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:21 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:21.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:21 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:21.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:21 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:21.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:21 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:21.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:21 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:22.351 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:22 smithi045 ceph-mon[32299]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.590+0000 7fa5feeb0700 1 -- 172.21.15.45:0/4073106014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f8102900 msgr2=0x7fa5f810e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.590+0000 7fa5feeb0700 1 --2- 172.21.15.45:0/4073106014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f8102900 0x7fa5f810e310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000b0d0 tx=0x7fa5f000b3e0 comp rx=0 tx=0).stop 2024-06-06T04:28:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.590+0000 7fa5feeb0700 1 -- 172.21.15.45:0/4073106014 shutdown_connections 2024-06-06T04:28:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.590+0000 7fa5feeb0700 1 --2- 172.21.15.45:0/4073106014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f8102900 0x7fa5f810e310 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.590+0000 7fa5feeb0700 1 --2- 172.21.15.45:0/4073106014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f8107660 0x7fa5f8107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.590+0000 7fa5feeb0700 1 --2- 172.21.15.45:0/4073106014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5f8101f40 0x7fa5f81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.590+0000 7fa5feeb0700 1 -- 172.21.15.45:0/4073106014 >> 172.21.15.45:0/4073106014 conn(0x7fa5f80766b0 msgr2=0x7fa5f8076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.590+0000 7fa5feeb0700 1 -- 172.21.15.45:0/4073106014 shutdown_connections 2024-06-06T04:28:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.590+0000 7fa5feeb0700 1 -- 172.21.15.45:0/4073106014 wait complete. 2024-06-06T04:28:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.590+0000 7fa5feeb0700 1 Processor -- start 2024-06-06T04:28:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.590+0000 7fa5feeb0700 1 -- start start 2024-06-06T04:28:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5feeb0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f8101f40 0x7fa5f810eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5feeb0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5f8102900 0x7fa5f810f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5feeb0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f8107660 0x7fa5f810f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5feeb0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f81127e0 con 0x7fa5f8107660 2024-06-06T04:28:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5feeb0700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8112660 con 0x7fa5f8101f40 2024-06-06T04:28:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5feeb0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8112960 con 0x7fa5f8102900 2024-06-06T04:28:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5fd44d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f8107660 0x7fa5f810f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5fd44d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f8107660 0x7fa5f810f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46234/0 (socket says 172.21.15.45:46234) 2024-06-06T04:28:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5fd44d700 1 -- 172.21.15.45:0/3766171498 learned_addr learned my addr 172.21.15.45:0/3766171498 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5f7fff700 1 --2- 172.21.15.45:0/3766171498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5f8102900 0x7fa5f810f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5fcc4c700 1 --2- 172.21.15.45:0/3766171498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f8101f40 0x7fa5f810eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5fd44d700 1 -- 172.21.15.45:0/3766171498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5f8102900 msgr2=0x7fa5f810f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5fd44d700 1 --2- 172.21.15.45:0/3766171498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5f8102900 0x7fa5f810f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5fd44d700 1 -- 172.21.15.45:0/3766171498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f8101f40 msgr2=0x7fa5f810eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5fd44d700 1 --2- 172.21.15.45:0/3766171498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f8101f40 0x7fa5f810eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5fd44d700 1 -- 172.21.15.45:0/3766171498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f81a70f0 con 0x7fa5f8107660 2024-06-06T04:28:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5fcc4c700 1 --2- 172.21.15.45:0/3766171498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f8101f40 0x7fa5f810eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:28:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.591+0000 7fa5fd44d700 1 --2- 172.21.15.45:0/3766171498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f8107660 0x7fa5f810f820 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009fd0 tx=0x7fa5f0007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.592+0000 7fa5f5ffb700 1 -- 172.21.15.45:0/3766171498 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f001f070 con 0x7fa5f8107660 2024-06-06T04:28:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.592+0000 7fa5f5ffb700 1 -- 172.21.15.45:0/3766171498 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5f0003c00 con 0x7fa5f8107660 2024-06-06T04:28:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.592+0000 7fa5feeb0700 1 -- 172.21.15.45:0/3766171498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f81a7380 con 0x7fa5f8107660 2024-06-06T04:28:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.592+0000 7fa5f5ffb700 1 -- 172.21.15.45:0/3766171498 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f001b6a0 con 0x7fa5f8107660 2024-06-06T04:28:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.592+0000 7fa5feeb0700 1 -- 172.21.15.45:0/3766171498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f81a77d0 con 0x7fa5f8107660 2024-06-06T04:28:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.593+0000 7fa5feeb0700 1 -- 172.21.15.45:0/3766171498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5f8069180 con 0x7fa5f8107660 2024-06-06T04:28:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.594+0000 7fa5f5ffb700 1 -- 172.21.15.45:0/3766171498 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa5f0022070 con 0x7fa5f8107660 2024-06-06T04:28:22.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.595+0000 7fa5f5ffb700 1 --2- 172.21.15.45:0/3766171498 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa5e80a2c20 0x7fa5e80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:22.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.595+0000 7fa5f5ffb700 1 -- 172.21.15.45:0/3766171498 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7fa5f000f020 con 0x7fa5f8107660 2024-06-06T04:28:22.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.595+0000 7fa5fcc4c700 1 --2- 172.21.15.45:0/3766171498 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa5e80a2c20 0x7fa5e80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:22.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.596+0000 7fa5fcc4c700 1 --2- 172.21.15.45:0/3766171498 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa5e80a2c20 0x7fa5e80a50e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0099d0 tx=0x7fa5ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:22.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.600+0000 7fa5f5ffb700 1 -- 172.21.15.45:0/3766171498 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa5f0029020 con 0x7fa5f8107660 2024-06-06T04:28:22.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:22 smithi086 ceph-mon[36053]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:22.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:22 smithi154 ceph-mon[36428]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T04:28:22.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:22.877+0000 7fa5feeb0700 1 -- 172.21.15.45:0/3766171498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa5f802de60 con 0x7fa5f8107660 2024-06-06T04:28:23.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:23.685+0000 7fa5f5ffb700 1 -- 172.21.15.45:0/3766171498 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fa5f009d340 con 0x7fa5f8107660 2024-06-06T04:28:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:23.688+0000 7fa5feeb0700 1 -- 172.21.15.45:0/3766171498 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa5e80a2c20 msgr2=0x7fa5e80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:23.688+0000 7fa5feeb0700 1 --2- 172.21.15.45:0/3766171498 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa5e80a2c20 0x7fa5e80a50e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0099d0 tx=0x7fa5ec008040 comp rx=0 tx=0).stop 2024-06-06T04:28:23.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:23.688+0000 7fa5feeb0700 1 -- 172.21.15.45:0/3766171498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f8107660 msgr2=0x7fa5f810f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:23.688+0000 7fa5feeb0700 1 --2- 172.21.15.45:0/3766171498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f8107660 0x7fa5f810f820 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009fd0 tx=0x7fa5f0007a20 comp rx=0 tx=0).stop 2024-06-06T04:28:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:23.689+0000 7fa5feeb0700 1 -- 172.21.15.45:0/3766171498 shutdown_connections 2024-06-06T04:28:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:23.689+0000 7fa5feeb0700 1 --2- 172.21.15.45:0/3766171498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f8107660 0x7fa5f810f820 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:23.689+0000 7fa5feeb0700 1 --2- 172.21.15.45:0/3766171498 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa5e80a2c20 0x7fa5e80a50e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:23.689+0000 7fa5feeb0700 1 --2- 172.21.15.45:0/3766171498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5f8101f40 0x7fa5f810eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:23.689+0000 7fa5feeb0700 1 --2- 172.21.15.45:0/3766171498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5f8102900 0x7fa5f810f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:23.689+0000 7fa5feeb0700 1 -- 172.21.15.45:0/3766171498 >> 172.21.15.45:0/3766171498 conn(0x7fa5f80766b0 msgr2=0x7fa5f80fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:23.689+0000 7fa5feeb0700 1 -- 172.21.15.45:0/3766171498 shutdown_connections 2024-06-06T04:28:23.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:23.689+0000 7fa5feeb0700 1 -- 172.21.15.45:0/3766171498 wait complete. 2024-06-06T04:28:23.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:23 smithi045 ceph-mon[32299]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T04:28:23.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:23 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:23.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:23 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:23.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:23 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3766171498' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-06T04:28:23.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:23 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:28:23.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:23 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:28:23.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:23 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:24.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:23 smithi086 ceph-mon[36053]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T04:28:24.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:23 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:24.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:23 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:24.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:23 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3766171498' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-06T04:28:24.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:23 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:28:24.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:23 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:28:24.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:23 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:24.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:23 smithi154 ceph-mon[36428]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T04:28:24.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:23 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:24.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:23 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:24.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:23 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3766171498' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-06T04:28:24.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:23 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:28:24.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:23 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:28:24.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:23 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:28:24.364 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-06-06T04:28:24.364 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-06-06T04:28:24.622 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:24.887 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:24 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3766171498' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-06T04:28:24.887 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:24 smithi045 ceph-mon[32299]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T04:28:25.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:24 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3766171498' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-06T04:28:25.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:24 smithi086 ceph-mon[36053]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T04:28:25.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:24 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3766171498' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-06T04:28:25.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:24 smithi154 ceph-mon[36428]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T04:28:25.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:25 smithi045 ceph-mon[32299]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:28:26.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:25 smithi086 ceph-mon[36053]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:28:26.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:25 smithi154 ceph-mon[36428]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:28:26.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.325+0000 7f4cd7bb8700 1 -- 172.21.15.45:0/81507977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0102740 msgr2=0x7f4cd0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:26.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.325+0000 7f4cd7bb8700 1 --2- 172.21.15.45:0/81507977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0102740 0x7f4cd0102b20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0099c0 tx=0x7f4ccc009cd0 comp rx=0 tx=0).stop 2024-06-06T04:28:26.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.325+0000 7f4cd7bb8700 1 -- 172.21.15.45:0/81507977 shutdown_connections 2024-06-06T04:28:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.325+0000 7f4cd7bb8700 1 --2- 172.21.15.45:0/81507977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd0107840 0x7f4cd010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.325+0000 7f4cd7bb8700 1 --2- 172.21.15.45:0/81507977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0102740 0x7f4cd0102b20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.325+0000 7f4cd7bb8700 1 --2- 172.21.15.45:0/81507977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cd01030f0 0x7f4cd0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.325+0000 7f4cd7bb8700 1 -- 172.21.15.45:0/81507977 >> 172.21.15.45:0/81507977 conn(0x7f4cd00795f0 msgr2=0x7f4cd0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.325+0000 7f4cd7bb8700 1 -- 172.21.15.45:0/81507977 shutdown_connections 2024-06-06T04:28:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.325+0000 7f4cd7bb8700 1 -- 172.21.15.45:0/81507977 wait complete. 2024-06-06T04:28:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.326+0000 7f4cd7bb8700 1 Processor -- start 2024-06-06T04:28:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.326+0000 7f4cd7bb8700 1 -- start start 2024-06-06T04:28:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.326+0000 7f4cd7bb8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cd0102740 0x7f4cd019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.326+0000 7f4cd7bb8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd01030f0 0x7f4cd019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.326+0000 7f4cd7bb8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0107840 0x7f4cd0198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.326+0000 7f4cd7bb8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd0112680 con 0x7f4cd0107840 2024-06-06T04:28:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.326+0000 7f4cd7bb8700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd0112500 con 0x7f4cd0102740 2024-06-06T04:28:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.326+0000 7f4cd7bb8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd0112800 con 0x7f4cd01030f0 2024-06-06T04:28:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.326+0000 7f4cd5954700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cd0102740 0x7f4cd019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.326+0000 7f4cd5954700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cd0102740 0x7f4cd019e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:53366/0 (socket says 172.21.15.45:53366) 2024-06-06T04:28:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.326+0000 7f4cd5954700 1 -- 172.21.15.45:0/1383142358 learned_addr learned my addr 172.21.15.45:0/1383142358 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.327+0000 7f4cd6155700 1 --2- 172.21.15.45:0/1383142358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0107840 0x7f4cd0198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.327+0000 7f4cd5153700 1 --2- 172.21.15.45:0/1383142358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd01030f0 0x7f4cd019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.327+0000 7f4cd5954700 1 -- 172.21.15.45:0/1383142358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd01030f0 msgr2=0x7f4cd019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.327+0000 7f4cd5954700 1 --2- 172.21.15.45:0/1383142358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd01030f0 0x7f4cd019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.327+0000 7f4cd5954700 1 -- 172.21.15.45:0/1383142358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0107840 msgr2=0x7f4cd0198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.327+0000 7f4cd5954700 1 --2- 172.21.15.45:0/1383142358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0107840 0x7f4cd0198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.327+0000 7f4cd5954700 1 -- 172.21.15.45:0/1383142358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd0199080 con 0x7f4cd0102740 2024-06-06T04:28:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.327+0000 7f4cd5153700 1 --2- 172.21.15.45:0/1383142358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd01030f0 0x7f4cd019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:28:26.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.327+0000 7f4cd5954700 1 --2- 172.21.15.45:0/1383142358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cd0102740 0x7f4cd019e760 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc005910 tx=0x7f4ccc0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:26.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.327+0000 7f4cc6ffd700 1 -- 172.21.15.45:0/1383142358 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc026070 con 0x7f4cd0102740 2024-06-06T04:28:26.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.327+0000 7f4cc6ffd700 1 -- 172.21.15.45:0/1383142358 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ccc00f460 con 0x7f4cd0102740 2024-06-06T04:28:26.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.327+0000 7f4cd7bb8700 1 -- 172.21.15.45:0/1383142358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd0199310 con 0x7f4cd0102740 2024-06-06T04:28:26.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.327+0000 7f4cc6ffd700 1 -- 172.21.15.45:0/1383142358 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc009e60 con 0x7f4cd0102740 2024-06-06T04:28:26.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.327+0000 7f4cd7bb8700 1 -- 172.21.15.45:0/1383142358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd01a5630 con 0x7f4cd0102740 2024-06-06T04:28:26.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.328+0000 7f4cd7bb8700 1 -- 172.21.15.45:0/1383142358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd0069180 con 0x7f4cd0102740 2024-06-06T04:28:26.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.330+0000 7f4cc6ffd700 1 -- 172.21.15.45:0/1383142358 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4ccc029070 con 0x7f4cd0102740 2024-06-06T04:28:26.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.331+0000 7f4cc6ffd700 1 --2- 172.21.15.45:0/1383142358 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4cbc0a2c20 0x7f4cbc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:26.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.331+0000 7f4cd5153700 1 --2- 172.21.15.45:0/1383142358 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4cbc0a2c20 0x7f4cbc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:26.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.331+0000 7f4cc6ffd700 1 -- 172.21.15.45:0/1383142358 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7f4ccc0cfdd0 con 0x7f4cd0102740 2024-06-06T04:28:26.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.331+0000 7f4cd5153700 1 --2- 172.21.15.45:0/1383142358 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4cbc0a2c20 0x7f4cbc0a50e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0005950 tx=0x7f4cc00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:26.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.334+0000 7f4cc6ffd700 1 -- 172.21.15.45:0/1383142358 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4ccc030080 con 0x7f4cd0102740 2024-06-06T04:28:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.591+0000 7f4cd7bb8700 1 -- 172.21.15.45:0/1383142358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f4cd002dca0 con 0x7f4cd0102740 2024-06-06T04:28:26.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.701+0000 7f4cc6ffd700 1 -- 172.21.15.45:0/1383142358 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f4ccc013070 con 0x7f4cd0102740 2024-06-06T04:28:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.703+0000 7f4cd7bb8700 1 -- 172.21.15.45:0/1383142358 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4cbc0a2c20 msgr2=0x7f4cbc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.703+0000 7f4cd7bb8700 1 --2- 172.21.15.45:0/1383142358 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4cbc0a2c20 0x7f4cbc0a50e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0005950 tx=0x7f4cc00058e0 comp rx=0 tx=0).stop 2024-06-06T04:28:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.703+0000 7f4cd7bb8700 1 -- 172.21.15.45:0/1383142358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cd0102740 msgr2=0x7f4cd019e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.703+0000 7f4cd7bb8700 1 --2- 172.21.15.45:0/1383142358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cd0102740 0x7f4cd019e760 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc005910 tx=0x7f4ccc0045e0 comp rx=0 tx=0).stop 2024-06-06T04:28:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.703+0000 7f4cd7bb8700 1 -- 172.21.15.45:0/1383142358 shutdown_connections 2024-06-06T04:28:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.703+0000 7f4cd7bb8700 1 --2- 172.21.15.45:0/1383142358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd0107840 0x7f4cd0198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.703+0000 7f4cd7bb8700 1 --2- 172.21.15.45:0/1383142358 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4cbc0a2c20 0x7f4cbc0a50e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.703+0000 7f4cd7bb8700 1 --2- 172.21.15.45:0/1383142358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cd0102740 0x7f4cd019e760 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.703+0000 7f4cd7bb8700 1 --2- 172.21.15.45:0/1383142358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd01030f0 0x7f4cd019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.703+0000 7f4cd7bb8700 1 -- 172.21.15.45:0/1383142358 >> 172.21.15.45:0/1383142358 conn(0x7f4cd00795f0 msgr2=0x7f4cd0100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.703+0000 7f4cd7bb8700 1 -- 172.21.15.45:0/1383142358 shutdown_connections 2024-06-06T04:28:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:26.703+0000 7f4cd7bb8700 1 -- 172.21.15.45:0/1383142358 wait complete. 2024-06-06T04:28:26.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:26 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1383142358' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-06T04:28:26.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:26 smithi045 ceph-mon[32299]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-06T04:28:27.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:26 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1383142358' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-06T04:28:27.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:26 smithi086 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-06T04:28:27.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:26 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1383142358' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-06T04:28:27.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:26 smithi154 ceph-mon[36428]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-06T04:28:27.331 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-06-06T04:28:27.581 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:27.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:27 smithi045 ceph-mon[32299]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:28:27.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:27 smithi045 ceph-mon[32299]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-06T04:28:27.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:27 smithi045 ceph-mon[32299]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-06T04:28:27.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:27 smithi045 ceph-mon[32299]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-06T04:28:27.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:27 smithi045 ceph-mon[32299]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T04:28:27.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:27 smithi045 ceph-mon[32299]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-06T04:28:27.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:27 smithi045 ceph-mon[32299]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-06T04:28:28.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:27 smithi086 ceph-mon[36053]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:28:28.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:27 smithi086 ceph-mon[36053]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-06T04:28:28.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:27 smithi086 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-06T04:28:28.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:27 smithi086 ceph-mon[36053]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-06T04:28:28.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:27 smithi086 ceph-mon[36053]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T04:28:28.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:27 smithi086 ceph-mon[36053]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-06T04:28:28.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:27 smithi086 ceph-mon[36053]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-06T04:28:28.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:27 smithi154 ceph-mon[36428]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:28:28.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:27 smithi154 ceph-mon[36428]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-06T04:28:28.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:27 smithi154 ceph-mon[36428]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-06T04:28:28.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:27 smithi154 ceph-mon[36428]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-06T04:28:28.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:27 smithi154 ceph-mon[36428]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-06T04:28:28.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:27 smithi154 ceph-mon[36428]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-06T04:28:28.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:27 smithi154 ceph-mon[36428]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-06T04:28:28.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:28 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] up:active 2024-06-06T04:28:28.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:28 smithi045 ceph-mon[32299]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-06T04:28:28.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:28 smithi045 ceph-mon[32299]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-06T04:28:28.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:28 smithi045 ceph-mon[32299]: Cluster is now healthy 2024-06-06T04:28:28.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:28 smithi045 ceph-mon[32299]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-06T04:28:28.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:28 smithi045 ceph-mon[32299]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-06-06T04:28:28.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:28 smithi045 ceph-mon[32299]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-06-06T04:28:29.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:28 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] up:active 2024-06-06T04:28:29.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:28 smithi086 ceph-mon[36053]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-06T04:28:29.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:28 smithi086 ceph-mon[36053]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-06T04:28:29.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:28 smithi086 ceph-mon[36053]: Cluster is now healthy 2024-06-06T04:28:29.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:28 smithi086 ceph-mon[36053]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-06T04:28:29.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:28 smithi086 ceph-mon[36053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-06-06T04:28:29.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:28 smithi086 ceph-mon[36053]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-06-06T04:28:29.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:28 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] up:active 2024-06-06T04:28:29.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:28 smithi154 ceph-mon[36428]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-06T04:28:29.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:28 smithi154 ceph-mon[36428]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-06T04:28:29.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:28 smithi154 ceph-mon[36428]: Cluster is now healthy 2024-06-06T04:28:29.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:28 smithi154 ceph-mon[36428]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-06T04:28:29.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:28 smithi154 ceph-mon[36428]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-06-06T04:28:29.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:28 smithi154 ceph-mon[36428]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-06-06T04:28:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.265+0000 7f9236cb3700 1 -- 172.21.15.45:0/2983867868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9230108500 msgr2=0x7f92301088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.265+0000 7f9236cb3700 1 --2- 172.21.15.45:0/2983867868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9230108500 0x7f92301088e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9224009990 tx=0x7f9224009ca0 comp rx=0 tx=0).stop 2024-06-06T04:28:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.265+0000 7f9236cb3700 1 -- 172.21.15.45:0/2983867868 shutdown_connections 2024-06-06T04:28:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.265+0000 7f9236cb3700 1 --2- 172.21.15.45:0/2983867868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923006ab30 0x7f923010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.265+0000 7f9236cb3700 1 --2- 172.21.15.45:0/2983867868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9230108500 0x7f92301088e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.265+0000 7f9236cb3700 1 --2- 172.21.15.45:0/2983867868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f923006a170 0x7f923006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.265+0000 7f9236cb3700 1 -- 172.21.15.45:0/2983867868 >> 172.21.15.45:0/2983867868 conn(0x7f9230078340 msgr2=0x7f9230078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.265+0000 7f9236cb3700 1 -- 172.21.15.45:0/2983867868 shutdown_connections 2024-06-06T04:28:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.265+0000 7f9236cb3700 1 -- 172.21.15.45:0/2983867868 wait complete. 2024-06-06T04:28:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9236cb3700 1 Processor -- start 2024-06-06T04:28:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9236cb3700 1 -- start start 2024-06-06T04:28:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9236cb3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923006a170 0x7f923019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9236cb3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f923006ab30 0x7f923019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9236cb3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9230108500 0x7f9230198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9236cb3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9230112630 con 0x7f9230108500 2024-06-06T04:28:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9236cb3700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92301124b0 con 0x7f923006a170 2024-06-06T04:28:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9236cb3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f92301127b0 con 0x7f923006ab30 2024-06-06T04:28:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9235250700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9230108500 0x7f9230198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9235250700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9230108500 0x7f9230198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56080/0 (socket says 172.21.15.45:56080) 2024-06-06T04:28:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9235250700 1 -- 172.21.15.45:0/3790578340 learned_addr learned my addr 172.21.15.45:0/3790578340 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9234a4f700 1 --2- 172.21.15.45:0/3790578340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923006a170 0x7f923019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f922ffff700 1 --2- 172.21.15.45:0/3790578340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f923006ab30 0x7f923019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9235250700 1 -- 172.21.15.45:0/3790578340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f923006ab30 msgr2=0x7f923019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9235250700 1 --2- 172.21.15.45:0/3790578340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f923006ab30 0x7f923019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9235250700 1 -- 172.21.15.45:0/3790578340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923006a170 msgr2=0x7f923019e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9235250700 1 --2- 172.21.15.45:0/3790578340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923006a170 0x7f923019e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.266+0000 7f9235250700 1 -- 172.21.15.45:0/3790578340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92301990b0 con 0x7f9230108500 2024-06-06T04:28:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.267+0000 7f9235250700 1 --2- 172.21.15.45:0/3790578340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9230108500 0x7f9230198950 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9228007d70 tx=0x7f922800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.267+0000 7f922dffb700 1 -- 172.21.15.45:0/3790578340 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228016020 con 0x7f9230108500 2024-06-06T04:28:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.267+0000 7f9236cb3700 1 -- 172.21.15.45:0/3790578340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92301993a0 con 0x7f9230108500 2024-06-06T04:28:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.267+0000 7f922dffb700 1 -- 172.21.15.45:0/3790578340 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f922800ad40 con 0x7f9230108500 2024-06-06T04:28:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.267+0000 7f922dffb700 1 -- 172.21.15.45:0/3790578340 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92280115e0 con 0x7f9230108500 2024-06-06T04:28:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.267+0000 7f9236cb3700 1 -- 172.21.15.45:0/3790578340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92301a5610 con 0x7f9230108500 2024-06-06T04:28:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.268+0000 7f9236cb3700 1 -- 172.21.15.45:0/3790578340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f923010a9e0 con 0x7f9230108500 2024-06-06T04:28:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.269+0000 7f922dffb700 1 -- 172.21.15.45:0/3790578340 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9228019020 con 0x7f9230108500 2024-06-06T04:28:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.270+0000 7f922dffb700 1 --2- 172.21.15.45:0/3790578340 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f92200a2c20 0x7f92200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.270+0000 7f9234a4f700 1 --2- 172.21.15.45:0/3790578340 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f92200a2c20 0x7f92200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.270+0000 7f922dffb700 1 -- 172.21.15.45:0/3790578340 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7f92280cf300 con 0x7f9230108500 2024-06-06T04:28:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.270+0000 7f9234a4f700 1 --2- 172.21.15.45:0/3790578340 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f92200a2c20 0x7f92200a50e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f922400b5c0 tx=0x7f9224005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:29.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.274+0000 7f922dffb700 1 -- 172.21.15.45:0/3790578340 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9228020020 con 0x7f9230108500 2024-06-06T04:28:29.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.530+0000 7f9236cb3700 1 -- 172.21.15.45:0/3790578340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f923010a3c0 con 0x7f9230108500 2024-06-06T04:28:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.718+0000 7f922dffb700 1 -- 172.21.15.45:0/3790578340 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f922809bf20 con 0x7f9230108500 2024-06-06T04:28:29.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.721+0000 7f9236cb3700 1 -- 172.21.15.45:0/3790578340 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f92200a2c20 msgr2=0x7f92200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:29.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.721+0000 7f9236cb3700 1 --2- 172.21.15.45:0/3790578340 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f92200a2c20 0x7f92200a50e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f922400b5c0 tx=0x7f9224005960 comp rx=0 tx=0).stop 2024-06-06T04:28:29.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.721+0000 7f9236cb3700 1 -- 172.21.15.45:0/3790578340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9230108500 msgr2=0x7f9230198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:29.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.721+0000 7f9236cb3700 1 --2- 172.21.15.45:0/3790578340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9230108500 0x7f9230198950 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9228007d70 tx=0x7f922800a340 comp rx=0 tx=0).stop 2024-06-06T04:28:29.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.721+0000 7f9236cb3700 1 -- 172.21.15.45:0/3790578340 shutdown_connections 2024-06-06T04:28:29.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.721+0000 7f9236cb3700 1 --2- 172.21.15.45:0/3790578340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9230108500 0x7f9230198950 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:29.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.721+0000 7f9236cb3700 1 --2- 172.21.15.45:0/3790578340 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f92200a2c20 0x7f92200a50e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:29.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.721+0000 7f9236cb3700 1 --2- 172.21.15.45:0/3790578340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923006a170 0x7f923019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:29.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.721+0000 7f9236cb3700 1 --2- 172.21.15.45:0/3790578340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f923006ab30 0x7f923019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:29.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.721+0000 7f9236cb3700 1 -- 172.21.15.45:0/3790578340 >> 172.21.15.45:0/3790578340 conn(0x7f9230078340 msgr2=0x7f9230100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.721+0000 7f9236cb3700 1 -- 172.21.15.45:0/3790578340 shutdown_connections 2024-06-06T04:28:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:29.721+0000 7f9236cb3700 1 -- 172.21.15.45:0/3790578340 wait complete. 2024-06-06T04:28:29.724 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:28:29 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-b[65935]: 2024-06-06T04:28:29.446+0000 7f432ec43700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-06-06T04:28:29.724 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:28:29 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-b[65935]: 2024-06-06T04:28:29.446+0000 7f432ec43700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-06-06T04:28:29.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:29 smithi045 ceph-mon[32299]: pgmap v340: 129 pgs: 129 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-06-06T04:28:29.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:29 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] up:active 2024-06-06T04:28:29.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:29 smithi045 ceph-mon[32299]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-06-06T04:28:29.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:29 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3790578340' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-06T04:28:30.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:29 smithi086 ceph-mon[36053]: pgmap v340: 129 pgs: 129 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-06-06T04:28:30.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:29 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] up:active 2024-06-06T04:28:30.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:29 smithi086 ceph-mon[36053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-06-06T04:28:30.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:29 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3790578340' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-06T04:28:30.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:29 smithi154 ceph-mon[36428]: pgmap v340: 129 pgs: 129 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-06-06T04:28:30.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:29 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] up:active 2024-06-06T04:28:30.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:29 smithi154 ceph-mon[36428]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-06-06T04:28:30.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:29 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3790578340' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-06T04:28:30.314 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-06-06T04:28:30.566 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:30.956 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:30 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3790578340' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-06T04:28:30.956 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:30 smithi045 ceph-mon[32299]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-06-06T04:28:30.956 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:30 smithi045 ceph-mon[32299]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:28:31.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:30 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3790578340' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-06T04:28:31.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:30 smithi086 ceph-mon[36053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-06-06T04:28:31.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:30 smithi086 ceph-mon[36053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:28:31.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:30 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3790578340' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-06T04:28:31.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:30 smithi154 ceph-mon[36428]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-06-06T04:28:31.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:30 smithi154 ceph-mon[36428]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:28:31.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:31 smithi045 ceph-mon[32299]: pgmap v341: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-06-06T04:28:32.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:31 smithi086 ceph-mon[36053]: pgmap v341: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-06-06T04:28:32.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:31 smithi154 ceph-mon[36428]: pgmap v341: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-06-06T04:28:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.254+0000 7fdb7f0b5700 1 -- 172.21.15.45:0/2729937767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb78102030 msgr2=0x7fdb7810e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.254+0000 7fdb7f0b5700 1 --2- 172.21.15.45:0/2729937767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb78102030 0x7fdb7810e1f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000b0a0 tx=0x7fdb7000b3b0 comp rx=0 tx=0).stop 2024-06-06T04:28:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.254+0000 7fdb7f0b5700 1 -- 172.21.15.45:0/2729937767 shutdown_connections 2024-06-06T04:28:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.254+0000 7fdb7f0b5700 1 --2- 172.21.15.45:0/2729937767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb78102030 0x7fdb7810e1f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.254+0000 7fdb7f0b5700 1 --2- 172.21.15.45:0/2729937767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb781075a0 0x7fdb78107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.254+0000 7fdb7f0b5700 1 --2- 172.21.15.45:0/2729937767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb78101670 0x7fdb78101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.254+0000 7fdb7f0b5700 1 -- 172.21.15.45:0/2729937767 >> 172.21.15.45:0/2729937767 conn(0x7fdb78078220 msgr2=0x7fdb78078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.255+0000 7fdb7f0b5700 1 -- 172.21.15.45:0/2729937767 shutdown_connections 2024-06-06T04:28:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.255+0000 7fdb7f0b5700 1 -- 172.21.15.45:0/2729937767 wait complete. 2024-06-06T04:28:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.255+0000 7fdb7f0b5700 1 Processor -- start 2024-06-06T04:28:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.255+0000 7fdb7f0b5700 1 -- start start 2024-06-06T04:28:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.255+0000 7fdb7f0b5700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb78101670 0x7fdb78103440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.255+0000 7fdb7f0b5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb78102030 0x7fdb78103980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.255+0000 7fdb7f0b5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb781075a0 0x7fdb78103ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.255+0000 7fdb7f0b5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdb78111570 con 0x7fdb78102030 2024-06-06T04:28:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.255+0000 7fdb7f0b5700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781113f0 con 0x7fdb78101670 2024-06-06T04:28:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.255+0000 7fdb7f0b5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781116f0 con 0x7fdb781075a0 2024-06-06T04:28:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.256+0000 7fdb6ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb78102030 0x7fdb78103980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.256+0000 7fdb6ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb78102030 0x7fdb78103980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56102/0 (socket says 172.21.15.45:56102) 2024-06-06T04:28:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.256+0000 7fdb6ffff700 1 -- 172.21.15.45:0/3591332937 learned_addr learned my addr 172.21.15.45:0/3591332937 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.256+0000 7fdb7ce51700 1 --2- 172.21.15.45:0/3591332937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb78101670 0x7fdb78103440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.256+0000 7fdb7d652700 1 --2- 172.21.15.45:0/3591332937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb781075a0 0x7fdb78103ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.256+0000 7fdb6ffff700 1 -- 172.21.15.45:0/3591332937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb781075a0 msgr2=0x7fdb78103ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.256+0000 7fdb6ffff700 1 --2- 172.21.15.45:0/3591332937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb781075a0 0x7fdb78103ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.256+0000 7fdb6ffff700 1 -- 172.21.15.45:0/3591332937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb78101670 msgr2=0x7fdb78103440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.256+0000 7fdb6ffff700 1 --2- 172.21.15.45:0/3591332937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb78101670 0x7fdb78103440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.256+0000 7fdb6ffff700 1 -- 172.21.15.45:0/3591332937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb781a6fb0 con 0x7fdb78102030 2024-06-06T04:28:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.256+0000 7fdb7d652700 1 --2- 172.21.15.45:0/3591332937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb781075a0 0x7fdb78103ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:28:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.256+0000 7fdb6ffff700 1 --2- 172.21.15.45:0/3591332937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb78102030 0x7fdb78103980 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800d980 tx=0x7fdb6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.256+0000 7fdb6dffb700 1 -- 172.21.15.45:0/3591332937 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68017070 con 0x7fdb78102030 2024-06-06T04:28:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.256+0000 7fdb6dffb700 1 -- 172.21.15.45:0/3591332937 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb68005020 con 0x7fdb78102030 2024-06-06T04:28:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.256+0000 7fdb7f0b5700 1 -- 172.21.15.45:0/3591332937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb781a72a0 con 0x7fdb78102030 2024-06-06T04:28:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.256+0000 7fdb6dffb700 1 -- 172.21.15.45:0/3591332937 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68009b50 con 0x7fdb78102030 2024-06-06T04:28:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.257+0000 7fdb7f0b5700 1 -- 172.21.15.45:0/3591332937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb781a77e0 con 0x7fdb78102030 2024-06-06T04:28:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.258+0000 7fdb7f0b5700 1 -- 172.21.15.45:0/3591332937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb78102b10 con 0x7fdb78102030 2024-06-06T04:28:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.259+0000 7fdb6dffb700 1 -- 172.21.15.45:0/3591332937 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdb68009cf0 con 0x7fdb78102030 2024-06-06T04:28:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.260+0000 7fdb6dffb700 1 --2- 172.21.15.45:0/3591332937 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdb600a2ca0 0x7fdb600a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.260+0000 7fdb6dffb700 1 -- 172.21.15.45:0/3591332937 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7fdb680cf260 con 0x7fdb78102030 2024-06-06T04:28:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.260+0000 7fdb7ce51700 1 --2- 172.21.15.45:0/3591332937 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdb600a2ca0 0x7fdb600a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:32.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.260+0000 7fdb7ce51700 1 --2- 172.21.15.45:0/3591332937 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdb600a2ca0 0x7fdb600a5160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdb74005950 tx=0x7fdb7400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.264+0000 7fdb6dffb700 1 -- 172.21.15.45:0/3591332937 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdb68020020 con 0x7fdb78102030 2024-06-06T04:28:32.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.514+0000 7fdb7f0b5700 1 -- 172.21.15.45:0/3591332937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fdb781a7a40 con 0x7fdb78102030 2024-06-06T04:28:32.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.735+0000 7fdb6dffb700 1 -- 172.21.15.45:0/3591332937 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fdb6809be80 con 0x7fdb78102030 2024-06-06T04:28:32.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.738+0000 7fdb7f0b5700 1 -- 172.21.15.45:0/3591332937 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdb600a2ca0 msgr2=0x7fdb600a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:32.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.738+0000 7fdb7f0b5700 1 --2- 172.21.15.45:0/3591332937 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdb600a2ca0 0x7fdb600a5160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdb74005950 tx=0x7fdb7400b410 comp rx=0 tx=0).stop 2024-06-06T04:28:32.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.738+0000 7fdb7f0b5700 1 -- 172.21.15.45:0/3591332937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb78102030 msgr2=0x7fdb78103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:32.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.738+0000 7fdb7f0b5700 1 --2- 172.21.15.45:0/3591332937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb78102030 0x7fdb78103980 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800d980 tx=0x7fdb6800dc90 comp rx=0 tx=0).stop 2024-06-06T04:28:32.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.738+0000 7fdb7f0b5700 1 -- 172.21.15.45:0/3591332937 shutdown_connections 2024-06-06T04:28:32.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.738+0000 7fdb7f0b5700 1 --2- 172.21.15.45:0/3591332937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb781075a0 0x7fdb78103ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:32.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.738+0000 7fdb7f0b5700 1 --2- 172.21.15.45:0/3591332937 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdb600a2ca0 0x7fdb600a5160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:32.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.738+0000 7fdb7f0b5700 1 --2- 172.21.15.45:0/3591332937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb78101670 0x7fdb78103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:32.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.738+0000 7fdb7f0b5700 1 --2- 172.21.15.45:0/3591332937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb78102030 0x7fdb78103980 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:32.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.738+0000 7fdb7f0b5700 1 -- 172.21.15.45:0/3591332937 >> 172.21.15.45:0/3591332937 conn(0x7fdb78078220 msgr2=0x7fdb781055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:32.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.738+0000 7fdb7f0b5700 1 -- 172.21.15.45:0/3591332937 shutdown_connections 2024-06-06T04:28:32.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:32.738+0000 7fdb7f0b5700 1 -- 172.21.15.45:0/3591332937 wait complete. 2024-06-06T04:28:33.059 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:32 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3591332937' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-06T04:28:33.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:32 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3591332937' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-06T04:28:33.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:32 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3591332937' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-06T04:28:33.380 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-06-06T04:28:33.381 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-06-06T04:28:33.629 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:34.063 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:33 smithi045 ceph-mon[32299]: pgmap v342: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-06T04:28:34.064 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:33 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3591332937' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-06T04:28:34.064 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:33 smithi045 ceph-mon[32299]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:28:34.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:33 smithi086 ceph-mon[36053]: pgmap v342: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-06T04:28:34.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:33 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3591332937' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-06T04:28:34.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:33 smithi086 ceph-mon[36053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:28:34.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:33 smithi154 ceph-mon[36428]: pgmap v342: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-06T04:28:34.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:33 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3591332937' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-06T04:28:34.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:33 smithi154 ceph-mon[36428]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:28:35.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.174+0000 7f45cff1f700 1 -- 172.21.15.45:0/2104228208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c8101390 msgr2=0x7f45c8101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:35.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.174+0000 7f45cff1f700 1 --2- 172.21.15.45:0/2104228208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c8101390 0x7f45c8101810 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009990 tx=0x7f45b8009ca0 comp rx=0 tx=0).stop 2024-06-06T04:28:35.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.174+0000 7f45cff1f700 1 -- 172.21.15.45:0/2104228208 shutdown_connections 2024-06-06T04:28:35.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.174+0000 7f45cff1f700 1 --2- 172.21.15.45:0/2104228208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45c8101d50 0x7f45c810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:35.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.174+0000 7f45cff1f700 1 --2- 172.21.15.45:0/2104228208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45c8107400 0x7f45c81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:35.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.174+0000 7f45cff1f700 1 --2- 172.21.15.45:0/2104228208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c8101390 0x7f45c8101810 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:35.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.174+0000 7f45cff1f700 1 -- 172.21.15.45:0/2104228208 >> 172.21.15.45:0/2104228208 conn(0x7f45c8078340 msgr2=0x7f45c8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:35.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.174+0000 7f45cff1f700 1 -- 172.21.15.45:0/2104228208 shutdown_connections 2024-06-06T04:28:35.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.174+0000 7f45cff1f700 1 -- 172.21.15.45:0/2104228208 wait complete. 2024-06-06T04:28:35.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45cff1f700 1 Processor -- start 2024-06-06T04:28:35.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45cff1f700 1 -- start start 2024-06-06T04:28:35.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45cff1f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45c8101390 0x7f45c819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:35.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45cff1f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45c8101d50 0x7f45c819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:35.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45cff1f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c8107400 0x7f45c819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:35.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45cff1f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f45c81115a0 con 0x7f45c8107400 2024-06-06T04:28:35.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45cff1f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45c8111420 con 0x7f45c8101d50 2024-06-06T04:28:35.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45cff1f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f45c8111720 con 0x7f45c8101390 2024-06-06T04:28:35.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45ce4bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c8107400 0x7f45c819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:35.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45ce4bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c8107400 0x7f45c819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56130/0 (socket says 172.21.15.45:56130) 2024-06-06T04:28:35.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45ce4bc700 1 -- 172.21.15.45:0/1596949516 learned_addr learned my addr 172.21.15.45:0/1596949516 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:35.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45cd4ba700 1 --2- 172.21.15.45:0/1596949516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45c8101d50 0x7f45c819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:35.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45cdcbb700 1 --2- 172.21.15.45:0/1596949516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45c8101390 0x7f45c819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:35.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45ce4bc700 1 -- 172.21.15.45:0/1596949516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45c8101390 msgr2=0x7f45c819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:35.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45ce4bc700 1 --2- 172.21.15.45:0/1596949516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45c8101390 0x7f45c819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:35.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45ce4bc700 1 -- 172.21.15.45:0/1596949516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45c8101d50 msgr2=0x7f45c819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:35.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45ce4bc700 1 --2- 172.21.15.45:0/1596949516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45c8101d50 0x7f45c819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:35.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.175+0000 7f45ce4bc700 1 -- 172.21.15.45:0/1596949516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c819fdf0 con 0x7f45c8107400 2024-06-06T04:28:35.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.176+0000 7f45ce4bc700 1 --2- 172.21.15.45:0/1596949516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c8107400 0x7f45c819f6b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f45c0007ee0 tx=0x7f45c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:35.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.176+0000 7f45beffd700 1 -- 172.21.15.45:0/1596949516 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0004bb0 con 0x7f45c8107400 2024-06-06T04:28:35.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.176+0000 7f45beffd700 1 -- 172.21.15.45:0/1596949516 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45c0004d50 con 0x7f45c8107400 2024-06-06T04:28:35.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.176+0000 7f45beffd700 1 -- 172.21.15.45:0/1596949516 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c00058e0 con 0x7f45c8107400 2024-06-06T04:28:35.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.176+0000 7f45cff1f700 1 -- 172.21.15.45:0/1596949516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c81a00e0 con 0x7f45c8107400 2024-06-06T04:28:35.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.176+0000 7f45cff1f700 1 -- 172.21.15.45:0/1596949516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c81a7960 con 0x7f45c8107400 2024-06-06T04:28:35.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.177+0000 7f45cff1f700 1 -- 172.21.15.45:0/1596949516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c8069180 con 0x7f45c8107400 2024-06-06T04:28:35.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.178+0000 7f45beffd700 1 -- 172.21.15.45:0/1596949516 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f45c001a020 con 0x7f45c8107400 2024-06-06T04:28:35.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.179+0000 7f45beffd700 1 --2- 172.21.15.45:0/1596949516 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f45b40a2ca0 0x7f45b40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:35.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.179+0000 7f45beffd700 1 -- 172.21.15.45:0/1596949516 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7f45c00cea40 con 0x7f45c8107400 2024-06-06T04:28:35.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.179+0000 7f45cdcbb700 1 --2- 172.21.15.45:0/1596949516 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f45b40a2ca0 0x7f45b40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:35.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.179+0000 7f45cdcbb700 1 --2- 172.21.15.45:0/1596949516 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f45b40a2ca0 0x7f45b40a5160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f45c4005950 tx=0x7f45c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:35.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.184+0000 7f45beffd700 1 -- 172.21.15.45:0/1596949516 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f45c0010070 con 0x7f45c8107400 2024-06-06T04:28:35.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.419+0000 7f45cff1f700 1 -- 172.21.15.45:0/1596949516 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f45c806a8b0 con 0x7f45b40a2ca0 2024-06-06T04:28:35.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.447+0000 7f45beffd700 1 -- 172.21.15.45:0/1596949516 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f45c806a8b0 con 0x7f45b40a2ca0 2024-06-06T04:28:35.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.449+0000 7f45cff1f700 1 -- 172.21.15.45:0/1596949516 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f45b40a2ca0 msgr2=0x7f45b40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:35.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.449+0000 7f45cff1f700 1 --2- 172.21.15.45:0/1596949516 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f45b40a2ca0 0x7f45b40a5160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f45c4005950 tx=0x7f45c400b410 comp rx=0 tx=0).stop 2024-06-06T04:28:35.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.449+0000 7f45cff1f700 1 -- 172.21.15.45:0/1596949516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c8107400 msgr2=0x7f45c819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:35.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.449+0000 7f45cff1f700 1 --2- 172.21.15.45:0/1596949516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c8107400 0x7f45c819f6b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f45c0007ee0 tx=0x7f45c000d3b0 comp rx=0 tx=0).stop 2024-06-06T04:28:35.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.449+0000 7f45cff1f700 1 -- 172.21.15.45:0/1596949516 shutdown_connections 2024-06-06T04:28:35.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.449+0000 7f45cff1f700 1 --2- 172.21.15.45:0/1596949516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45c8107400 0x7f45c819f6b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:35.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.449+0000 7f45cff1f700 1 --2- 172.21.15.45:0/1596949516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45c8101390 0x7f45c819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:35.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.449+0000 7f45cff1f700 1 --2- 172.21.15.45:0/1596949516 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f45b40a2ca0 0x7f45b40a5160 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:35.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.449+0000 7f45cff1f700 1 --2- 172.21.15.45:0/1596949516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45c8101d50 0x7f45c819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:35.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.449+0000 7f45cff1f700 1 -- 172.21.15.45:0/1596949516 >> 172.21.15.45:0/1596949516 conn(0x7f45c8078340 msgr2=0x7f45c806a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:35.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.449+0000 7f45cff1f700 1 -- 172.21.15.45:0/1596949516 shutdown_connections 2024-06-06T04:28:35.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:35.449+0000 7f45cff1f700 1 -- 172.21.15.45:0/1596949516 wait complete. 2024-06-06T04:28:35.727 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:28:35 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:28:35.425+0000 7f120b098700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T04:28:35.727 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:28:35 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:28:35.425+0000 7f120b098700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T04:28:35.727 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:28:35 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:28:35.425+0000 7f120b098700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T04:28:35.727 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:28:35 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:28:35.425+0000 7f120b098700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T04:28:35.727 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:28:35 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:28:35.425+0000 7f120b098700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T04:28:36.000 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:35 smithi154 ceph-mon[36428]: pgmap v343: 129 pgs: 129 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T04:28:36.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:35 smithi086 ceph-mon[36053]: pgmap v343: 129 pgs: 129 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T04:28:36.147 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:35 smithi045 ceph-mon[32299]: pgmap v343: 129 pgs: 129 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-06T04:28:36.181 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-06-06T04:28:36.434 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:36.807 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:36 smithi045 ceph-mon[32299]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:28:37.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:36 smithi086 ceph-mon[36053]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:28:37.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:36 smithi154 ceph-mon[36428]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:28:37.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:37 smithi045 ceph-mon[32299]: pgmap v344: 129 pgs: 129 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-06T04:28:37.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:37 smithi045 ceph-mon[32299]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-06-06T04:28:38.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.094+0000 7f47b5d6e700 1 -- 172.21.15.45:0/3052577960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b0107840 msgr2=0x7f47b010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:38.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.094+0000 7f47b5d6e700 1 --2- 172.21.15.45:0/3052577960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b0107840 0x7f47b010f3c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f47a80099c0 tx=0x7f47a8009cd0 comp rx=0 tx=0).stop 2024-06-06T04:28:38.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.094+0000 7f47b5d6e700 1 -- 172.21.15.45:0/3052577960 shutdown_connections 2024-06-06T04:28:38.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.094+0000 7f47b5d6e700 1 --2- 172.21.15.45:0/3052577960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b0107840 0x7f47b010f3c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:38.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.094+0000 7f47b5d6e700 1 --2- 172.21.15.45:0/3052577960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b0102740 0x7f47b0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:38.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.094+0000 7f47b5d6e700 1 --2- 172.21.15.45:0/3052577960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47b01030f0 0x7f47b0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:38.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.094+0000 7f47b5d6e700 1 -- 172.21.15.45:0/3052577960 >> 172.21.15.45:0/3052577960 conn(0x7f47b00795f0 msgr2=0x7f47b0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:38.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.094+0000 7f47b5d6e700 1 -- 172.21.15.45:0/3052577960 shutdown_connections 2024-06-06T04:28:38.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.094+0000 7f47b5d6e700 1 -- 172.21.15.45:0/3052577960 wait complete. 2024-06-06T04:28:38.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.095+0000 7f47b5d6e700 1 Processor -- start 2024-06-06T04:28:38.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.095+0000 7f47b5d6e700 1 -- start start 2024-06-06T04:28:38.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.095+0000 7f47b5d6e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b0102740 0x7f47b019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:38.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.095+0000 7f47b5d6e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47b01030f0 0x7f47b019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:38.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.095+0000 7f47b5d6e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b0107840 0x7f47b0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:38.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.095+0000 7f47b5d6e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f47b01126b0 con 0x7f47b0102740 2024-06-06T04:28:38.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.095+0000 7f47b5d6e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47b0112530 con 0x7f47b0107840 2024-06-06T04:28:38.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.095+0000 7f47b5d6e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f47b0112830 con 0x7f47b01030f0 2024-06-06T04:28:38.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.095+0000 7f47af7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b0102740 0x7f47b019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:38.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.095+0000 7f47af7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b0102740 0x7f47b019e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40852/0 (socket says 172.21.15.45:40852) 2024-06-06T04:28:38.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.095+0000 7f47af7fe700 1 -- 172.21.15.45:0/3443288714 learned_addr learned my addr 172.21.15.45:0/3443288714 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:38.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.095+0000 7f47aeffd700 1 --2- 172.21.15.45:0/3443288714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47b01030f0 0x7f47b019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:38.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.095+0000 7f47affff700 1 --2- 172.21.15.45:0/3443288714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b0107840 0x7f47b0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:38.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.096+0000 7f47af7fe700 1 -- 172.21.15.45:0/3443288714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47b01030f0 msgr2=0x7f47b019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:38.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.096+0000 7f47af7fe700 1 --2- 172.21.15.45:0/3443288714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47b01030f0 0x7f47b019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:38.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.096+0000 7f47af7fe700 1 -- 172.21.15.45:0/3443288714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b0107840 msgr2=0x7f47b0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:38.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.096+0000 7f47af7fe700 1 --2- 172.21.15.45:0/3443288714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b0107840 0x7f47b0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:38.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.096+0000 7f47af7fe700 1 -- 172.21.15.45:0/3443288714 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b01990d0 con 0x7f47b0102740 2024-06-06T04:28:38.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.096+0000 7f47aeffd700 1 --2- 172.21.15.45:0/3443288714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47b01030f0 0x7f47b019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:28:38.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.096+0000 7f47af7fe700 1 --2- 172.21.15.45:0/3443288714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b0102740 0x7f47b019e760 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f47a000ba50 tx=0x7f47a000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:38.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.096+0000 7f47acff9700 1 -- 172.21.15.45:0/3443288714 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a0019070 con 0x7f47b0102740 2024-06-06T04:28:38.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.096+0000 7f47acff9700 1 -- 172.21.15.45:0/3443288714 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47a0014440 con 0x7f47b0102740 2024-06-06T04:28:38.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.096+0000 7f47b5d6e700 1 -- 172.21.15.45:0/3443288714 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b01993c0 con 0x7f47b0102740 2024-06-06T04:28:38.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.096+0000 7f47acff9700 1 -- 172.21.15.45:0/3443288714 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a00075d0 con 0x7f47b0102740 2024-06-06T04:28:38.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.096+0000 7f47b5d6e700 1 -- 172.21.15.45:0/3443288714 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b01a56b0 con 0x7f47b0102740 2024-06-06T04:28:38.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.097+0000 7f47b5d6e700 1 -- 172.21.15.45:0/3443288714 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b010aa80 con 0x7f47b0102740 2024-06-06T04:28:38.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.098+0000 7f47acff9700 1 -- 172.21.15.45:0/3443288714 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f47a001c070 con 0x7f47b0102740 2024-06-06T04:28:38.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.099+0000 7f47acff9700 1 --2- 172.21.15.45:0/3443288714 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f479c0a2d30 0x7f479c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:38.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.099+0000 7f47acff9700 1 -- 172.21.15.45:0/3443288714 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7f47a00ce860 con 0x7f47b0102740 2024-06-06T04:28:38.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.099+0000 7f47aeffd700 1 --2- 172.21.15.45:0/3443288714 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f479c0a2d30 0x7f479c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:38.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.100+0000 7f47aeffd700 1 --2- 172.21.15.45:0/3443288714 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f479c0a2d30 0x7f479c0a51f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f47b0102010 tx=0x7f479800b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:38.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.104+0000 7f47acff9700 1 -- 172.21.15.45:0/3443288714 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f47a009b3d0 con 0x7f47b0102740 2024-06-06T04:28:38.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:37 smithi086 ceph-mon[36053]: pgmap v344: 129 pgs: 129 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-06T04:28:38.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:37 smithi086 ceph-mon[36053]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-06-06T04:28:38.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:37 smithi154 ceph-mon[36428]: pgmap v344: 129 pgs: 129 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-06T04:28:38.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:37 smithi154 ceph-mon[36428]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-06-06T04:28:38.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.343+0000 7f47b5d6e700 1 -- 172.21.15.45:0/3443288714 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f47b00628d0 con 0x7f479c0a2d30 2024-06-06T04:28:38.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.373+0000 7f47acff9700 1 -- 172.21.15.45:0/3443288714 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f47b00628d0 con 0x7f479c0a2d30 2024-06-06T04:28:38.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.376+0000 7f47b5d6e700 1 -- 172.21.15.45:0/3443288714 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f479c0a2d30 msgr2=0x7f479c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:38.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.376+0000 7f47b5d6e700 1 --2- 172.21.15.45:0/3443288714 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f479c0a2d30 0x7f479c0a51f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f47b0102010 tx=0x7f479800b2c0 comp rx=0 tx=0).stop 2024-06-06T04:28:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.376+0000 7f47b5d6e700 1 -- 172.21.15.45:0/3443288714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b0102740 msgr2=0x7f47b019e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.376+0000 7f47b5d6e700 1 --2- 172.21.15.45:0/3443288714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b0102740 0x7f47b019e760 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f47a000ba50 tx=0x7f47a000be10 comp rx=0 tx=0).stop 2024-06-06T04:28:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.376+0000 7f47b5d6e700 1 -- 172.21.15.45:0/3443288714 shutdown_connections 2024-06-06T04:28:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.376+0000 7f47b5d6e700 1 --2- 172.21.15.45:0/3443288714 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f479c0a2d30 0x7f479c0a51f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.376+0000 7f47b5d6e700 1 --2- 172.21.15.45:0/3443288714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b0107840 0x7f47b0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.376+0000 7f47b5d6e700 1 --2- 172.21.15.45:0/3443288714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b0102740 0x7f47b019e760 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.376+0000 7f47b5d6e700 1 --2- 172.21.15.45:0/3443288714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47b01030f0 0x7f47b019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.376+0000 7f47b5d6e700 1 -- 172.21.15.45:0/3443288714 >> 172.21.15.45:0/3443288714 conn(0x7f47b00795f0 msgr2=0x7f47b01006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.376+0000 7f47b5d6e700 1 -- 172.21.15.45:0/3443288714 shutdown_connections 2024-06-06T04:28:38.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:38.376+0000 7f47b5d6e700 1 -- 172.21.15.45:0/3443288714 wait complete. 2024-06-06T04:28:39.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-06-06T04:28:39.250 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:39.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:39 smithi045 ceph-mon[32299]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:28:39.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:39 smithi045 ceph-mon[32299]: pgmap v345: 129 pgs: 129 active+clean; 591 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-06-06T04:28:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:39 smithi086 ceph-mon[36053]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:28:40.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:39 smithi086 ceph-mon[36053]: pgmap v345: 129 pgs: 129 active+clean; 591 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-06-06T04:28:40.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:39 smithi154 ceph-mon[36428]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:28:40.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:39 smithi154 ceph-mon[36428]: pgmap v345: 129 pgs: 129 active+clean; 591 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-06-06T04:28:40.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.980+0000 7f3895f28700 1 -- 172.21.15.45:0/3456510821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38901030f0 msgr2=0x7f3890107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:40.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.980+0000 7f3895f28700 1 --2- 172.21.15.45:0/3456510821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38901030f0 0x7f3890107110 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f38840099c0 tx=0x7f3884009cd0 comp rx=0 tx=0).stop 2024-06-06T04:28:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.980+0000 7f3895f28700 1 -- 172.21.15.45:0/3456510821 shutdown_connections 2024-06-06T04:28:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.980+0000 7f3895f28700 1 --2- 172.21.15.45:0/3456510821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3890107840 0x7f389010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.980+0000 7f3895f28700 1 --2- 172.21.15.45:0/3456510821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3890102740 0x7f3890102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.980+0000 7f3895f28700 1 --2- 172.21.15.45:0/3456510821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38901030f0 0x7f3890107110 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.980+0000 7f3895f28700 1 -- 172.21.15.45:0/3456510821 >> 172.21.15.45:0/3456510821 conn(0x7f38900795f0 msgr2=0x7f3890079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.980+0000 7f3895f28700 1 -- 172.21.15.45:0/3456510821 shutdown_connections 2024-06-06T04:28:40.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.980+0000 7f3895f28700 1 -- 172.21.15.45:0/3456510821 wait complete. 2024-06-06T04:28:40.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.981+0000 7f3895f28700 1 Processor -- start 2024-06-06T04:28:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.981+0000 7f3895f28700 1 -- start start 2024-06-06T04:28:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.981+0000 7f3895f28700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3890102740 0x7f3890074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.981+0000 7f3895f28700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38901030f0 0x7f3890074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.981+0000 7f3895f28700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3890107840 0x7f3890075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.981+0000 7f3895f28700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3890112600 con 0x7f38901030f0 2024-06-06T04:28:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.981+0000 7f3895f28700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3890112480 con 0x7f3890102740 2024-06-06T04:28:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.981+0000 7f3895f28700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3890112780 con 0x7f3890107840 2024-06-06T04:28:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.981+0000 7f388effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38901030f0 0x7f3890074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.981+0000 7f388effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38901030f0 0x7f3890074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40866/0 (socket says 172.21.15.45:40866) 2024-06-06T04:28:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.981+0000 7f388effd700 1 -- 172.21.15.45:0/425052283 learned_addr learned my addr 172.21.15.45:0/425052283 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.981+0000 7f388f7fe700 1 --2- 172.21.15.45:0/425052283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3890102740 0x7f3890074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.982+0000 7f388ffff700 1 --2- 172.21.15.45:0/425052283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3890107840 0x7f3890075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.982+0000 7f388effd700 1 -- 172.21.15.45:0/425052283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3890107840 msgr2=0x7f3890075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.982+0000 7f388effd700 1 --2- 172.21.15.45:0/425052283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3890107840 0x7f3890075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:40.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.982+0000 7f388effd700 1 -- 172.21.15.45:0/425052283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3890102740 msgr2=0x7f3890074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:40.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.982+0000 7f388effd700 1 --2- 172.21.15.45:0/425052283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3890102740 0x7f3890074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:40.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.982+0000 7f388effd700 1 -- 172.21.15.45:0/425052283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38901a6fa0 con 0x7f38901030f0 2024-06-06T04:28:40.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.982+0000 7f388effd700 1 --2- 172.21.15.45:0/425052283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38901030f0 0x7f3890074cd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3884006010 tx=0x7f3884009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:40.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.982+0000 7f388cff9700 1 -- 172.21.15.45:0/425052283 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884027070 con 0x7f38901030f0 2024-06-06T04:28:40.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.982+0000 7f388cff9700 1 -- 172.21.15.45:0/425052283 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3884003f50 con 0x7f38901030f0 2024-06-06T04:28:40.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.982+0000 7f3895f28700 1 -- 172.21.15.45:0/425052283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38901a7230 con 0x7f38901030f0 2024-06-06T04:28:40.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.982+0000 7f388cff9700 1 -- 172.21.15.45:0/425052283 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884022cd0 con 0x7f38901030f0 2024-06-06T04:28:40.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.982+0000 7f3895f28700 1 -- 172.21.15.45:0/425052283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38901a7680 con 0x7f38901030f0 2024-06-06T04:28:40.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.983+0000 7f3895f28700 1 -- 172.21.15.45:0/425052283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3890069180 con 0x7f38901030f0 2024-06-06T04:28:40.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.984+0000 7f388cff9700 1 -- 172.21.15.45:0/425052283 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f388402a070 con 0x7f38901030f0 2024-06-06T04:28:40.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.985+0000 7f388cff9700 1 --2- 172.21.15.45:0/425052283 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f38780a2cb0 0x7f38780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:40.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.985+0000 7f388f7fe700 1 --2- 172.21.15.45:0/425052283 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f38780a2cb0 0x7f38780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:40.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.985+0000 7f388cff9700 1 -- 172.21.15.45:0/425052283 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7f38840d0380 con 0x7f38901030f0 2024-06-06T04:28:40.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.985+0000 7f388f7fe700 1 --2- 172.21.15.45:0/425052283 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f38780a2cb0 0x7f38780a5170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3880005950 tx=0x7f388000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:40.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:40.989+0000 7f388cff9700 1 -- 172.21.15.45:0/425052283 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f388405e850 con 0x7f38901030f0 2024-06-06T04:28:41.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:41.234+0000 7f3895f28700 1 -- 172.21.15.45:0/425052283 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f38900628d0 con 0x7f38780a2cb0 2024-06-06T04:28:41.240 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/qa/sv_0/d0b2bf13-bd83-48bd-8e31-022ceebb759d 2024-06-06T04:28:41.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:41.240+0000 7f388cff9700 1 -- 172.21.15.45:0/425052283 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f38900628d0 con 0x7f38780a2cb0 2024-06-06T04:28:41.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:41.242+0000 7f3895f28700 1 -- 172.21.15.45:0/425052283 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f38780a2cb0 msgr2=0x7f38780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:41.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:41.242+0000 7f3895f28700 1 --2- 172.21.15.45:0/425052283 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f38780a2cb0 0x7f38780a5170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3880005950 tx=0x7f388000b410 comp rx=0 tx=0).stop 2024-06-06T04:28:41.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:41.242+0000 7f3895f28700 1 -- 172.21.15.45:0/425052283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38901030f0 msgr2=0x7f3890074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:41.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:41.242+0000 7f3895f28700 1 --2- 172.21.15.45:0/425052283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38901030f0 0x7f3890074cd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3884006010 tx=0x7f3884009f80 comp rx=0 tx=0).stop 2024-06-06T04:28:41.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:41.242+0000 7f3895f28700 1 -- 172.21.15.45:0/425052283 shutdown_connections 2024-06-06T04:28:41.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:41.242+0000 7f3895f28700 1 --2- 172.21.15.45:0/425052283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3890107840 0x7f3890075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:41.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:41.242+0000 7f3895f28700 1 --2- 172.21.15.45:0/425052283 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f38780a2cb0 0x7f38780a5170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:41.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:41.242+0000 7f3895f28700 1 --2- 172.21.15.45:0/425052283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3890102740 0x7f3890074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:41.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:41.242+0000 7f3895f28700 1 --2- 172.21.15.45:0/425052283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38901030f0 0x7f3890074cd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:41.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:41.242+0000 7f3895f28700 1 -- 172.21.15.45:0/425052283 >> 172.21.15.45:0/425052283 conn(0x7f38900795f0 msgr2=0x7f3890100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:41.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:41.242+0000 7f3895f28700 1 -- 172.21.15.45:0/425052283 shutdown_connections 2024-06-06T04:28:41.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:41.243+0000 7f3895f28700 1 -- 172.21.15.45:0/425052283 wait complete. 2024-06-06T04:28:41.957 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:41 smithi045 ceph-mon[32299]: pgmap v346: 129 pgs: 129 active+clean; 591 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-06T04:28:41.957 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:41 smithi045 ceph-mon[32299]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:28:41.989 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-06-06T04:28:42.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:41 smithi086 ceph-mon[36053]: pgmap v346: 129 pgs: 129 active+clean; 591 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-06T04:28:42.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:41 smithi086 ceph-mon[36053]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:28:42.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:41 smithi154 ceph-mon[36428]: pgmap v346: 129 pgs: 129 active+clean; 591 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-06T04:28:42.182 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:41 smithi154 ceph-mon[36428]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:28:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:43.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.977+0000 7f736c9f4700 1 -- 172.21.15.45:0/4095946735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368101390 msgr2=0x7f7368101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:43.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.977+0000 7f736c9f4700 1 --2- 172.21.15.45:0/4095946735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368101390 0x7f7368101810 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f735c009990 tx=0x7f735c009ca0 comp rx=0 tx=0).stop 2024-06-06T04:28:43.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.977+0000 7f736c9f4700 1 -- 172.21.15.45:0/4095946735 shutdown_connections 2024-06-06T04:28:43.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.977+0000 7f736c9f4700 1 --2- 172.21.15.45:0/4095946735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7368101d50 0x7f736810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:43.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.977+0000 7f736c9f4700 1 --2- 172.21.15.45:0/4095946735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7368107400 0x7f73681077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:43.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.977+0000 7f736c9f4700 1 --2- 172.21.15.45:0/4095946735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368101390 0x7f7368101810 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:43.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.977+0000 7f736c9f4700 1 -- 172.21.15.45:0/4095946735 >> 172.21.15.45:0/4095946735 conn(0x7f7368078340 msgr2=0x7f7368078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:43.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.977+0000 7f736c9f4700 1 -- 172.21.15.45:0/4095946735 shutdown_connections 2024-06-06T04:28:43.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.977+0000 7f736c9f4700 1 -- 172.21.15.45:0/4095946735 wait complete. 2024-06-06T04:28:43.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.978+0000 7f736c9f4700 1 Processor -- start 2024-06-06T04:28:43.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.978+0000 7f736c9f4700 1 -- start start 2024-06-06T04:28:43.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.978+0000 7f736c9f4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368101390 0x7f736819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:43.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.978+0000 7f736c9f4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7368101d50 0x7f736819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:43.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.978+0000 7f736c9f4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7368107400 0x7f7368198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:43.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.979+0000 7f736c9f4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7368111570 con 0x7f7368101390 2024-06-06T04:28:43.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.979+0000 7f736c9f4700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73681113f0 con 0x7f7368107400 2024-06-06T04:28:43.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.979+0000 7f736c9f4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73681116f0 con 0x7f7368101d50 2024-06-06T04:28:43.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.979+0000 7f736659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368101390 0x7f736819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:43.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.979+0000 7f736659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368101390 0x7f736819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40876/0 (socket says 172.21.15.45:40876) 2024-06-06T04:28:43.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.979+0000 7f736659c700 1 -- 172.21.15.45:0/3993755763 learned_addr learned my addr 172.21.15.45:0/3993755763 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:43.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.979+0000 7f7366d9d700 1 --2- 172.21.15.45:0/3993755763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7368107400 0x7f7368198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:43.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.979+0000 7f7365d9b700 1 --2- 172.21.15.45:0/3993755763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7368101d50 0x7f736819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:43.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.979+0000 7f736659c700 1 -- 172.21.15.45:0/3993755763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7368101d50 msgr2=0x7f736819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:43.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.979+0000 7f736659c700 1 --2- 172.21.15.45:0/3993755763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7368101d50 0x7f736819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:43.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.979+0000 7f736659c700 1 -- 172.21.15.45:0/3993755763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7368107400 msgr2=0x7f7368198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:43.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.979+0000 7f736659c700 1 --2- 172.21.15.45:0/3993755763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7368107400 0x7f7368198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:43.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.979+0000 7f736659c700 1 -- 172.21.15.45:0/3993755763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73681990c0 con 0x7f7368101390 2024-06-06T04:28:43.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.979+0000 7f736659c700 1 --2- 172.21.15.45:0/3993755763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368101390 0x7f736819e7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f735800d950 tx=0x7f735800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:43.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.979+0000 7f73577fe700 1 -- 172.21.15.45:0/3993755763 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7358016070 con 0x7f7368101390 2024-06-06T04:28:43.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.979+0000 7f736c9f4700 1 -- 172.21.15.45:0/3993755763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73681993b0 con 0x7f7368101390 2024-06-06T04:28:43.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.980+0000 7f73577fe700 1 -- 172.21.15.45:0/3993755763 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f735800fd40 con 0x7f7368101390 2024-06-06T04:28:43.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.980+0000 7f73577fe700 1 -- 172.21.15.45:0/3993755763 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735800e3b0 con 0x7f7368101390 2024-06-06T04:28:43.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.980+0000 7f736c9f4700 1 -- 172.21.15.45:0/3993755763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7368102fa0 con 0x7f7368101390 2024-06-06T04:28:43.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.980+0000 7f736c9f4700 1 -- 172.21.15.45:0/3993755763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7368069180 con 0x7f7368101390 2024-06-06T04:28:43.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.982+0000 7f73577fe700 1 -- 172.21.15.45:0/3993755763 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7358019070 con 0x7f7368101390 2024-06-06T04:28:43.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.982+0000 7f73577fe700 1 --2- 172.21.15.45:0/3993755763 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73500a2cb0 0x7f73500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:43.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.983+0000 7f7365d9b700 1 --2- 172.21.15.45:0/3993755763 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73500a2cb0 0x7f73500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:43.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.983+0000 7f73577fe700 1 -- 172.21.15.45:0/3993755763 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7f73580cef10 con 0x7f7368101390 2024-06-06T04:28:43.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.983+0000 7f7365d9b700 1 --2- 172.21.15.45:0/3993755763 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73500a2cb0 0x7f73500a5170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f73680788b0 tx=0x7f735c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:43.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:43.986+0000 7f73577fe700 1 -- 172.21.15.45:0/3993755763 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f735809b9d0 con 0x7f7368101390 2024-06-06T04:28:44.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:43 smithi086 ceph-mon[36053]: pgmap v347: 129 pgs: 129 active+clean; 609 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-06-06T04:28:44.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:43 smithi154 ceph-mon[36428]: pgmap v347: 129 pgs: 129 active+clean; 609 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-06-06T04:28:44.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:43 smithi045 ceph-mon[32299]: pgmap v347: 129 pgs: 129 active+clean; 609 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-06-06T04:28:44.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:44.225+0000 7f736c9f4700 1 -- 172.21.15.45:0/3993755763 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f73680628d0 con 0x7f73500a2cb0 2024-06-06T04:28:44.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:44.252+0000 7f73577fe700 1 -- 172.21.15.45:0/3993755763 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f73680628d0 con 0x7f73500a2cb0 2024-06-06T04:28:44.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:44.254+0000 7f736c9f4700 1 -- 172.21.15.45:0/3993755763 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73500a2cb0 msgr2=0x7f73500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:44.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:44.254+0000 7f736c9f4700 1 --2- 172.21.15.45:0/3993755763 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73500a2cb0 0x7f73500a5170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f73680788b0 tx=0x7f735c005fd0 comp rx=0 tx=0).stop 2024-06-06T04:28:44.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:44.254+0000 7f736c9f4700 1 -- 172.21.15.45:0/3993755763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368101390 msgr2=0x7f736819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:44.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:44.254+0000 7f736c9f4700 1 --2- 172.21.15.45:0/3993755763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368101390 0x7f736819e7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f735800d950 tx=0x7f735800dc60 comp rx=0 tx=0).stop 2024-06-06T04:28:44.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:44.254+0000 7f736c9f4700 1 -- 172.21.15.45:0/3993755763 shutdown_connections 2024-06-06T04:28:44.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:44.254+0000 7f736c9f4700 1 --2- 172.21.15.45:0/3993755763 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73500a2cb0 0x7f73500a5170 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:44.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:44.254+0000 7f736c9f4700 1 --2- 172.21.15.45:0/3993755763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7368107400 0x7f7368198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:44.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:44.254+0000 7f736c9f4700 1 --2- 172.21.15.45:0/3993755763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368101390 0x7f736819e7a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:44.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:44.254+0000 7f736c9f4700 1 --2- 172.21.15.45:0/3993755763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7368101d50 0x7f736819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:44.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:44.254+0000 7f736c9f4700 1 -- 172.21.15.45:0/3993755763 >> 172.21.15.45:0/3993755763 conn(0x7f7368078340 msgr2=0x7f7368100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:44.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:44.254+0000 7f736c9f4700 1 -- 172.21.15.45:0/3993755763 shutdown_connections 2024-06-06T04:28:44.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:44.254+0000 7f736c9f4700 1 -- 172.21.15.45:0/3993755763 wait complete. 2024-06-06T04:28:44.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:44 smithi045 ceph-mon[32299]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:28:45.054 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-06-06T04:28:45.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:44 smithi086 ceph-mon[36053]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:28:45.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:44 smithi154 ceph-mon[36428]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:28:45.305 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:45.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:45 smithi045 ceph-mon[32299]: pgmap v348: 129 pgs: 129 active+clean; 609 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-06-06T04:28:46.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:45 smithi086 ceph-mon[36053]: pgmap v348: 129 pgs: 129 active+clean; 609 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-06-06T04:28:46.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:45 smithi154 ceph-mon[36428]: pgmap v348: 129 pgs: 129 active+clean; 609 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-06-06T04:28:47.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.064+0000 7fb439280700 1 -- 172.21.15.45:0/2218492357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4341084f0 msgr2=0x7fb4341088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:47.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.064+0000 7fb439280700 1 --2- 172.21.15.45:0/2218492357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4341084f0 0x7fb4341088d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb424009990 tx=0x7fb424009ca0 comp rx=0 tx=0).stop 2024-06-06T04:28:47.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.064+0000 7fb439280700 1 -- 172.21.15.45:0/2218492357 shutdown_connections 2024-06-06T04:28:47.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.064+0000 7fb439280700 1 --2- 172.21.15.45:0/2218492357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434102e80 0x7fb43410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:47.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.064+0000 7fb439280700 1 --2- 172.21.15.45:0/2218492357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4341084f0 0x7fb4341088d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:47.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.064+0000 7fb439280700 1 --2- 172.21.15.45:0/2218492357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4341024e0 0x7fb434102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:47.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.064+0000 7fb439280700 1 -- 172.21.15.45:0/2218492357 >> 172.21.15.45:0/2218492357 conn(0x7fb434079530 msgr2=0x7fb434079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:47.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.064+0000 7fb439280700 1 -- 172.21.15.45:0/2218492357 shutdown_connections 2024-06-06T04:28:47.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.064+0000 7fb439280700 1 -- 172.21.15.45:0/2218492357 wait complete. 2024-06-06T04:28:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.065+0000 7fb439280700 1 Processor -- start 2024-06-06T04:28:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.065+0000 7fb439280700 1 -- start start 2024-06-06T04:28:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb439280700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4341024e0 0x7fb43419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb439280700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434102e80 0x7fb43419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb439280700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4341084f0 0x7fb434198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb439280700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb434112600 con 0x7fb4341024e0 2024-06-06T04:28:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb439280700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb434112480 con 0x7fb434102e80 2024-06-06T04:28:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb439280700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb434112780 con 0x7fb4341084f0 2024-06-06T04:28:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb432ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4341024e0 0x7fb43419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb432ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4341024e0 0x7fb43419e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40900/0 (socket says 172.21.15.45:40900) 2024-06-06T04:28:47.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb432ffd700 1 -- 172.21.15.45:0/3478341903 learned_addr learned my addr 172.21.15.45:0/3478341903 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb42a7fc700 1 --2- 172.21.15.45:0/3478341903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434102e80 0x7fb43419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb4337fe700 1 --2- 172.21.15.45:0/3478341903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4341084f0 0x7fb434198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb432ffd700 1 -- 172.21.15.45:0/3478341903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4341084f0 msgr2=0x7fb434198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb432ffd700 1 --2- 172.21.15.45:0/3478341903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4341084f0 0x7fb434198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb432ffd700 1 -- 172.21.15.45:0/3478341903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434102e80 msgr2=0x7fb43419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb432ffd700 1 --2- 172.21.15.45:0/3478341903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434102e80 0x7fb43419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb432ffd700 1 -- 172.21.15.45:0/3478341903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb434199190 con 0x7fb4341024e0 2024-06-06T04:28:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb42a7fc700 1 --2- 172.21.15.45:0/3478341903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434102e80 0x7fb43419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:28:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.066+0000 7fb432ffd700 1 --2- 172.21.15.45:0/3478341903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4341024e0 0x7fb43419e870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb424000c00 tx=0x7fb424003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.067+0000 7fb430ff9700 1 -- 172.21.15.45:0/3478341903 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424027070 con 0x7fb4341024e0 2024-06-06T04:28:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.067+0000 7fb430ff9700 1 -- 172.21.15.45:0/3478341903 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb42400bdc0 con 0x7fb4341024e0 2024-06-06T04:28:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.067+0000 7fb439280700 1 -- 172.21.15.45:0/3478341903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb434199420 con 0x7fb4341024e0 2024-06-06T04:28:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.067+0000 7fb430ff9700 1 -- 172.21.15.45:0/3478341903 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42400f460 con 0x7fb4341024e0 2024-06-06T04:28:47.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.067+0000 7fb439280700 1 -- 172.21.15.45:0/3478341903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4341a5740 con 0x7fb4341024e0 2024-06-06T04:28:47.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.068+0000 7fb439280700 1 -- 172.21.15.45:0/3478341903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb434103ba0 con 0x7fb4341024e0 2024-06-06T04:28:47.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.069+0000 7fb430ff9700 1 -- 172.21.15.45:0/3478341903 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb42402a070 con 0x7fb4341024e0 2024-06-06T04:28:47.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.069+0000 7fb430ff9700 1 --2- 172.21.15.45:0/3478341903 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4200a2cb0 0x7fb4200a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:47.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.070+0000 7fb42a7fc700 1 --2- 172.21.15.45:0/3478341903 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4200a2cb0 0x7fb4200a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:47.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.070+0000 7fb430ff9700 1 -- 172.21.15.45:0/3478341903 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7fb4240d0410 con 0x7fb4341024e0 2024-06-06T04:28:47.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.070+0000 7fb42a7fc700 1 --2- 172.21.15.45:0/3478341903 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4200a2cb0 0x7fb4200a5170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb434079c50 tx=0x7fb41c00b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.074+0000 7fb430ff9700 1 -- 172.21.15.45:0/3478341903 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb424013070 con 0x7fb4341024e0 2024-06-06T04:28:47.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.316+0000 7fb439280700 1 -- 172.21.15.45:0/3478341903 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb434199f60 con 0x7fb4200a2cb0 2024-06-06T04:28:47.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.322+0000 7fb430ff9700 1 -- 172.21.15.45:0/3478341903 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb434199f60 con 0x7fb4200a2cb0 2024-06-06T04:28:47.336 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/qa/sv_1/a616e4d2-eddf-48e8-8a99-98b3c2139f06 2024-06-06T04:28:47.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.324+0000 7fb439280700 1 -- 172.21.15.45:0/3478341903 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4200a2cb0 msgr2=0x7fb4200a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:47.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.324+0000 7fb439280700 1 --2- 172.21.15.45:0/3478341903 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4200a2cb0 0x7fb4200a5170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb434079c50 tx=0x7fb41c00b2c0 comp rx=0 tx=0).stop 2024-06-06T04:28:47.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.324+0000 7fb439280700 1 -- 172.21.15.45:0/3478341903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4341024e0 msgr2=0x7fb43419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:47.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.324+0000 7fb439280700 1 --2- 172.21.15.45:0/3478341903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4341024e0 0x7fb43419e870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb424000c00 tx=0x7fb424003b30 comp rx=0 tx=0).stop 2024-06-06T04:28:47.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.324+0000 7fb439280700 1 -- 172.21.15.45:0/3478341903 shutdown_connections 2024-06-06T04:28:47.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.324+0000 7fb439280700 1 --2- 172.21.15.45:0/3478341903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4341084f0 0x7fb434198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:47.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.324+0000 7fb439280700 1 --2- 172.21.15.45:0/3478341903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4341024e0 0x7fb43419e870 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:47.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.324+0000 7fb439280700 1 --2- 172.21.15.45:0/3478341903 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4200a2cb0 0x7fb4200a5170 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:47.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.324+0000 7fb439280700 1 --2- 172.21.15.45:0/3478341903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb434102e80 0x7fb43419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:47.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.324+0000 7fb439280700 1 -- 172.21.15.45:0/3478341903 >> 172.21.15.45:0/3478341903 conn(0x7fb434079530 msgr2=0x7fb434100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:47.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.324+0000 7fb439280700 1 -- 172.21.15.45:0/3478341903 shutdown_connections 2024-06-06T04:28:47.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:47.324+0000 7fb439280700 1 -- 172.21.15.45:0/3478341903 wait complete. 2024-06-06T04:28:47.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:47 smithi045 ceph-mon[32299]: pgmap v349: 129 pgs: 129 active+clean; 609 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-06-06T04:28:47.995 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:28:48.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:47 smithi086 ceph-mon[36053]: pgmap v349: 129 pgs: 129 active+clean; 609 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-06-06T04:28:48.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:47 smithi154 ceph-mon[36428]: pgmap v349: 129 pgs: 129 active+clean; 609 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-06-06T04:28:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:48.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:48 smithi045 ceph-mon[32299]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:28:49.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:48 smithi086 ceph-mon[36053]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:28:49.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:48 smithi154 ceph-mon[36428]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:28:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.951+0000 7fa2dad2b700 1 -- 172.21.15.45:0/1210935623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d4100e30 msgr2=0x7fa2d4101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.951+0000 7fa2dad2b700 1 --2- 172.21.15.45:0/1210935623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d4100e30 0x7fa2d4101290 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009990 tx=0x7fa2c4009ca0 comp rx=0 tx=0).stop 2024-06-06T04:28:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.951+0000 7fa2dad2b700 1 -- 172.21.15.45:0/1210935623 shutdown_connections 2024-06-06T04:28:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.951+0000 7fa2dad2b700 1 --2- 172.21.15.45:0/1210935623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2d41017d0 0x7fa2d410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.951+0000 7fa2dad2b700 1 --2- 172.21.15.45:0/1210935623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d4107570 0x7fa2d4107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.951+0000 7fa2dad2b700 1 --2- 172.21.15.45:0/1210935623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d4100e30 0x7fa2d4101290 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.951+0000 7fa2dad2b700 1 -- 172.21.15.45:0/1210935623 >> 172.21.15.45:0/1210935623 conn(0x7fa2d4076680 msgr2=0x7fa2d4076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.951+0000 7fa2dad2b700 1 -- 172.21.15.45:0/1210935623 shutdown_connections 2024-06-06T04:28:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.951+0000 7fa2dad2b700 1 -- 172.21.15.45:0/1210935623 wait complete. 2024-06-06T04:28:49.953 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:49 smithi045 ceph-mon[32299]: pgmap v350: 129 pgs: 129 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-06-06T04:28:49.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.951+0000 7fa2dad2b700 1 Processor -- start 2024-06-06T04:28:49.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.951+0000 7fa2dad2b700 1 -- start start 2024-06-06T04:28:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2dad2b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2d4100e30 0x7fa2d41a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2dad2b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d41017d0 0x7fa2d41a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2dad2b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d4107570 0x7fa2d419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2dad2b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4112610 con 0x7fa2d41017d0 2024-06-06T04:28:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2dad2b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4112490 con 0x7fa2d4107570 2024-06-06T04:28:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2dad2b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4112790 con 0x7fa2d4100e30 2024-06-06T04:28:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2cbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d41017d0 0x7fa2d41a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2cbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d41017d0 0x7fa2d41a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34808/0 (socket says 172.21.15.45:34808) 2024-06-06T04:28:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2cbfff700 1 -- 172.21.15.45:0/135608539 learned_addr learned my addr 172.21.15.45:0/135608539 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2d8ac7700 1 --2- 172.21.15.45:0/135608539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2d4100e30 0x7fa2d41a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2d92c8700 1 --2- 172.21.15.45:0/135608539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d4107570 0x7fa2d419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2cbfff700 1 -- 172.21.15.45:0/135608539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2d4100e30 msgr2=0x7fa2d41a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:49.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2cbfff700 1 --2- 172.21.15.45:0/135608539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2d4100e30 0x7fa2d41a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:49.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2cbfff700 1 -- 172.21.15.45:0/135608539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d4107570 msgr2=0x7fa2d419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:49.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2cbfff700 1 --2- 172.21.15.45:0/135608539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d4107570 0x7fa2d419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:49.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2cbfff700 1 -- 172.21.15.45:0/135608539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d419b1d0 con 0x7fa2d41017d0 2024-06-06T04:28:49.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.952+0000 7fa2d92c8700 1 --2- 172.21.15.45:0/135608539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d4107570 0x7fa2d419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:28:49.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.953+0000 7fa2cbfff700 1 --2- 172.21.15.45:0/135608539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d41017d0 0x7fa2d41a0df0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4000c00 tx=0x7fa2c4003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:49.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.953+0000 7fa2c9ffb700 1 -- 172.21.15.45:0/135608539 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4027070 con 0x7fa2d41017d0 2024-06-06T04:28:49.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.953+0000 7fa2c9ffb700 1 -- 172.21.15.45:0/135608539 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2c400bdc0 con 0x7fa2d41017d0 2024-06-06T04:28:49.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.953+0000 7fa2dad2b700 1 -- 172.21.15.45:0/135608539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d419b460 con 0x7fa2d41017d0 2024-06-06T04:28:49.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.953+0000 7fa2c9ffb700 1 -- 172.21.15.45:0/135608539 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c400f460 con 0x7fa2d41017d0 2024-06-06T04:28:49.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.953+0000 7fa2dad2b700 1 -- 172.21.15.45:0/135608539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d41a7780 con 0x7fa2d41017d0 2024-06-06T04:28:49.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.954+0000 7fa2dad2b700 1 -- 172.21.15.45:0/135608539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d406a700 con 0x7fa2d41017d0 2024-06-06T04:28:49.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.955+0000 7fa2c9ffb700 1 -- 172.21.15.45:0/135608539 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa2c402a070 con 0x7fa2d41017d0 2024-06-06T04:28:49.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.956+0000 7fa2c9ffb700 1 --2- 172.21.15.45:0/135608539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa2bc0a2cb0 0x7fa2bc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:49.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.956+0000 7fa2c9ffb700 1 -- 172.21.15.45:0/135608539 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7fa2c40d04b0 con 0x7fa2d41017d0 2024-06-06T04:28:49.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.956+0000 7fa2d8ac7700 1 --2- 172.21.15.45:0/135608539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa2bc0a2cb0 0x7fa2bc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:49.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.957+0000 7fa2d8ac7700 1 --2- 172.21.15.45:0/135608539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa2bc0a2cb0 0x7fa2bc0a5170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa2d419bbb0 tx=0x7fa2d000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:49.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:49.960+0000 7fa2c9ffb700 1 -- 172.21.15.45:0/135608539 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa2c405e850 con 0x7fa2d41017d0 2024-06-06T04:28:50.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:49 smithi086 ceph-mon[36053]: pgmap v350: 129 pgs: 129 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-06-06T04:28:50.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:49 smithi154 ceph-mon[36428]: pgmap v350: 129 pgs: 129 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-06-06T04:28:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:50.212+0000 7fa2dad2b700 1 -- 172.21.15.45:0/135608539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2d41a7950 con 0x7fa2d41017d0 2024-06-06T04:28:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:50.213+0000 7fa2c9ffb700 1 -- 172.21.15.45:0/135608539 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11858 (secure 0 0 0) 0x7fa2c4013070 con 0x7fa2d41017d0 2024-06-06T04:28:50.213 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:28:50.213 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T04:27:24.462897+0000","modified":"2024-06-06T04:28:32.515629+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24463,24473,24451]},"id":1}]} 2024-06-06T04:28:50.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:50.215+0000 7fa2dad2b700 1 -- 172.21.15.45:0/135608539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa2bc0a2cb0 msgr2=0x7fa2bc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:50.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:50.215+0000 7fa2dad2b700 1 --2- 172.21.15.45:0/135608539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa2bc0a2cb0 0x7fa2bc0a5170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa2d419bbb0 tx=0x7fa2d000a3b0 comp rx=0 tx=0).stop 2024-06-06T04:28:50.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:50.215+0000 7fa2dad2b700 1 -- 172.21.15.45:0/135608539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d41017d0 msgr2=0x7fa2d41a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:50.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:50.215+0000 7fa2dad2b700 1 --2- 172.21.15.45:0/135608539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d41017d0 0x7fa2d41a0df0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4000c00 tx=0x7fa2c4003a80 comp rx=0 tx=0).stop 2024-06-06T04:28:50.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:50.215+0000 7fa2dad2b700 1 -- 172.21.15.45:0/135608539 shutdown_connections 2024-06-06T04:28:50.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:50.215+0000 7fa2dad2b700 1 --2- 172.21.15.45:0/135608539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa2bc0a2cb0 0x7fa2bc0a5170 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:50.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:50.215+0000 7fa2dad2b700 1 --2- 172.21.15.45:0/135608539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2d4107570 0x7fa2d419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:50.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:50.215+0000 7fa2dad2b700 1 --2- 172.21.15.45:0/135608539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2d4100e30 0x7fa2d41a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:50.215+0000 7fa2dad2b700 1 --2- 172.21.15.45:0/135608539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2d41017d0 0x7fa2d41a0df0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:50.215+0000 7fa2dad2b700 1 -- 172.21.15.45:0/135608539 >> 172.21.15.45:0/135608539 conn(0x7fa2d4076680 msgr2=0x7fa2d40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:50.215+0000 7fa2dad2b700 1 -- 172.21.15.45:0/135608539 shutdown_connections 2024-06-06T04:28:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:50.215+0000 7fa2dad2b700 1 -- 172.21.15.45:0/135608539 wait complete. 2024-06-06T04:28:50.217 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-06-06T04:28:50.902 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T04:28:51.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:50 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/135608539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:28:51.151 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:51.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:50 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/135608539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:28:51.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:50 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/135608539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:28:51.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:51 smithi045 ceph-mon[32299]: pgmap v351: 129 pgs: 129 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-06-06T04:28:52.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:51 smithi086 ceph-mon[36053]: pgmap v351: 129 pgs: 129 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-06-06T04:28:52.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:51 smithi154 ceph-mon[36428]: pgmap v351: 129 pgs: 129 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-06-06T04:28:52.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.818+0000 7fbd18b3e700 1 -- 172.21.15.45:0/1986705596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd14101370 msgr2=0x7fbd141017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:52.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.818+0000 7fbd18b3e700 1 --2- 172.21.15.45:0/1986705596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd14101370 0x7fbd141017f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009990 tx=0x7fbd08009ca0 comp rx=0 tx=0).stop 2024-06-06T04:28:52.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.818+0000 7fbd18b3e700 1 -- 172.21.15.45:0/1986705596 shutdown_connections 2024-06-06T04:28:52.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.818+0000 7fbd18b3e700 1 --2- 172.21.15.45:0/1986705596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd14101d30 0x7fbd1410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:52.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.818+0000 7fbd18b3e700 1 --2- 172.21.15.45:0/1986705596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd14107420 0x7fbd14107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:52.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.818+0000 7fbd18b3e700 1 --2- 172.21.15.45:0/1986705596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd14101370 0x7fbd141017f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:52.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.818+0000 7fbd18b3e700 1 -- 172.21.15.45:0/1986705596 >> 172.21.15.45:0/1986705596 conn(0x7fbd14078340 msgr2=0x7fbd14078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:52.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.818+0000 7fbd18b3e700 1 -- 172.21.15.45:0/1986705596 shutdown_connections 2024-06-06T04:28:52.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.818+0000 7fbd18b3e700 1 -- 172.21.15.45:0/1986705596 wait complete. 2024-06-06T04:28:52.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.819+0000 7fbd18b3e700 1 Processor -- start 2024-06-06T04:28:52.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.819+0000 7fbd18b3e700 1 -- start start 2024-06-06T04:28:52.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.819+0000 7fbd18b3e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd14101370 0x7fbd1419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:52.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.819+0000 7fbd18b3e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd14101d30 0x7fbd1419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:52.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.819+0000 7fbd18b3e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd14107420 0x7fbd14198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:52.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.819+0000 7fbd18b3e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbd14111510 con 0x7fbd14101370 2024-06-06T04:28:52.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.819+0000 7fbd18b3e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd14111390 con 0x7fbd14107420 2024-06-06T04:28:52.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.819+0000 7fbd18b3e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbd14111690 con 0x7fbd14101d30 2024-06-06T04:28:52.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.819+0000 7fbd1259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd14101370 0x7fbd1419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:52.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.819+0000 7fbd1259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd14101370 0x7fbd1419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34834/0 (socket says 172.21.15.45:34834) 2024-06-06T04:28:52.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.819+0000 7fbd1259c700 1 -- 172.21.15.45:0/465252763 learned_addr learned my addr 172.21.15.45:0/465252763 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:52.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.819+0000 7fbd12d9d700 1 --2- 172.21.15.45:0/465252763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd14107420 0x7fbd14198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:52.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.819+0000 7fbd11d9b700 1 --2- 172.21.15.45:0/465252763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd14101d30 0x7fbd1419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:52.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.820+0000 7fbd12d9d700 1 -- 172.21.15.45:0/465252763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd14101d30 msgr2=0x7fbd1419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:52.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.820+0000 7fbd12d9d700 1 --2- 172.21.15.45:0/465252763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd14101d30 0x7fbd1419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:52.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.820+0000 7fbd12d9d700 1 -- 172.21.15.45:0/465252763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd14101370 msgr2=0x7fbd1419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:52.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.820+0000 7fbd12d9d700 1 --2- 172.21.15.45:0/465252763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd14101370 0x7fbd1419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:52.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.820+0000 7fbd12d9d700 1 -- 172.21.15.45:0/465252763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd141990d0 con 0x7fbd14107420 2024-06-06T04:28:52.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.820+0000 7fbd11d9b700 1 --2- 172.21.15.45:0/465252763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd14101d30 0x7fbd1419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:28:52.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.820+0000 7fbd1259c700 1 --2- 172.21.15.45:0/465252763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd14101370 0x7fbd1419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:28:52.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.820+0000 7fbd12d9d700 1 --2- 172.21.15.45:0/465252763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd14107420 0x7fbd14198970 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c007f10 tx=0x7fbd0c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:52.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.820+0000 7fbd037fe700 1 -- 172.21.15.45:0/465252763 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c004db0 con 0x7fbd14107420 2024-06-06T04:28:52.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.820+0000 7fbd18b3e700 1 -- 172.21.15.45:0/465252763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd141993c0 con 0x7fbd14107420 2024-06-06T04:28:52.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.820+0000 7fbd037fe700 1 -- 172.21.15.45:0/465252763 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd0c009e20 con 0x7fbd14107420 2024-06-06T04:28:52.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.820+0000 7fbd037fe700 1 -- 172.21.15.45:0/465252763 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c015400 con 0x7fbd14107420 2024-06-06T04:28:52.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.820+0000 7fbd18b3e700 1 -- 172.21.15.45:0/465252763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd141a5680 con 0x7fbd14107420 2024-06-06T04:28:52.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.821+0000 7fbd18b3e700 1 -- 172.21.15.45:0/465252763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd14069180 con 0x7fbd14107420 2024-06-06T04:28:52.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.822+0000 7fbd037fe700 1 -- 172.21.15.45:0/465252763 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbd0c00db20 con 0x7fbd14107420 2024-06-06T04:28:52.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.823+0000 7fbd037fe700 1 --2- 172.21.15.45:0/465252763 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbcfc0a2cb0 0x7fbcfc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:52.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.823+0000 7fbd1259c700 1 --2- 172.21.15.45:0/465252763 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbcfc0a2cb0 0x7fbcfc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:52.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.823+0000 7fbd037fe700 1 -- 172.21.15.45:0/465252763 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7fbd0c0cf1b0 con 0x7fbd14107420 2024-06-06T04:28:52.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.824+0000 7fbd1259c700 1 --2- 172.21.15.45:0/465252763 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbcfc0a2cb0 0x7fbcfc0a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbd14199ab0 tx=0x7fbd0400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:52.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:52.827+0000 7fbd037fe700 1 -- 172.21.15.45:0/465252763 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbd0c010070 con 0x7fbd14107420 2024-06-06T04:28:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:53.036+0000 7fbd18b3e700 1 -- 172.21.15.45:0/465252763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd141a5880 con 0x7fbd14107420 2024-06-06T04:28:53.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:53.038+0000 7fbd037fe700 1 -- 172.21.15.45:0/465252763 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24602 (secure 0 0 0) 0x7fbd0c09bc10 con 0x7fbd14107420 2024-06-06T04:28:53.039 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:28:53.039 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":98,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","created":"2024-06-06T04:19:02.626916+0000","modified":"2024-06-06T04:28:05.713633+0000","last_up_change":"2024-06-06T04:26:17.390429+0000","last_in_change":"2024-06-06T04:25:56.938388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T04:22:28.797137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T04:27:17.750625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T04:27:21.486917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T04:27:31.029818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d3cee4a-5025-45b3-b217-082af37e79db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6803","nonce":2111759250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6805","nonce":2111759250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6809","nonce":2111759250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6807","nonce":2111759250}]},"public_addr":"172.21.15.45:6803/2111759250","cluster_addr":"172.21.15.45:6805/2111759250","heartbeat_back_addr":"172.21.15.45:6809/2111759250","heartbeat_front_addr":"172.21.15.45:6807/2111759250","state":["exists","up"]},{"osd":1,"uuid":"75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6801","nonce":3732225287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6803","nonce":3732225287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6807","nonce":3732225287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6805","nonce":3732225287}]},"public_addr":"172.21.15.86:6801/3732225287","cluster_addr":"172.21.15.86:6803/3732225287","heartbeat_back_addr":"172.21.15.86:6807/3732225287","heartbeat_front_addr":"172.21.15.86:6805/3732225287","state":["exists","up"]},{"osd":2,"uuid":"bb20565f-f9ae-47e3-9604-c69f737cff4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6801","nonce":2898783039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6803","nonce":2898783039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6807","nonce":2898783039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6805","nonce":2898783039}]},"public_addr":"172.21.15.154:6801/2898783039","cluster_addr":"172.21.15.154:6803/2898783039","heartbeat_back_addr":"172.21.15.154:6807/2898783039","heartbeat_front_addr":"172.21.15.154:6805/2898783039","state":["exists","up"]},{"osd":3,"uuid":"e1160799-7811-4326-9792-7e6c7d189c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6811","nonce":3436753135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6813","nonce":3436753135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6817","nonce":3436753135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6815","nonce":3436753135}]},"public_addr":"172.21.15.45:6811/3436753135","cluster_addr":"172.21.15.45:6813/3436753135","heartbeat_back_addr":"172.21.15.45:6817/3436753135","heartbeat_front_addr":"172.21.15.45:6815/3436753135","state":["exists","up"]},{"osd":4,"uuid":"6b389bc8-e19a-47f3-83ff-7b782d146d0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6809","nonce":3030235796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6811","nonce":3030235796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6815","nonce":3030235796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6813","nonce":3030235796}]},"public_addr":"172.21.15.86:6809/3030235796","cluster_addr":"172.21.15.86:6811/3030235796","heartbeat_back_addr":"172.21.15.86:6815/3030235796","heartbeat_front_addr":"172.21.15.86:6813/3030235796","state":["exists","up"]},{"osd":5,"uuid":"069de7b2-89bf-4150-bed7-7b8832573dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6809","nonce":2222268646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6811","nonce":2222268646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6815","nonce":2222268646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6813","nonce":2222268646}]},"public_addr":"172.21.15.154:6809/2222268646","cluster_addr":"172.21.15.154:6811/2222268646","heartbeat_back_addr":"172.21.15.154:6815/2222268646","heartbeat_front_addr":"172.21.15.154:6813/2222268646","state":["exists","up"]},{"osd":6,"uuid":"4c70b627-eafa-4e27-ba2a-4d8e1e335962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6819","nonce":384734939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6821","nonce":384734939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6825","nonce":384734939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6823","nonce":384734939}]},"public_addr":"172.21.15.45:6819/384734939","cluster_addr":"172.21.15.45:6821/384734939","heartbeat_back_addr":"172.21.15.45:6825/384734939","heartbeat_front_addr":"172.21.15.45:6823/384734939","state":["exists","up"]},{"osd":7,"uuid":"3b09b875-be52-4be0-b116-7fd9b0c26705","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6817","nonce":1119791005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6819","nonce":1119791005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6823","nonce":1119791005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6821","nonce":1119791005}]},"public_addr":"172.21.15.86:6817/1119791005","cluster_addr":"172.21.15.86:6819/1119791005","heartbeat_back_addr":"172.21.15.86:6823/1119791005","heartbeat_front_addr":"172.21.15.86:6821/1119791005","state":["exists","up"]},{"osd":8,"uuid":"cdf1dd1b-5392-4836-8e65-4c7c5b688d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6817","nonce":3054812496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6819","nonce":3054812496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6823","nonce":3054812496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6821","nonce":3054812496}]},"public_addr":"172.21.15.154:6817/3054812496","cluster_addr":"172.21.15.154:6819/3054812496","heartbeat_back_addr":"172.21.15.154:6823/3054812496","heartbeat_front_addr":"172.21.15.154:6821/3054812496","state":["exists","up"]},{"osd":9,"uuid":"acede92d-2cc1-4967-a357-7e5e201e99d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6827","nonce":1360655510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6829","nonce":1360655510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6833","nonce":1360655510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6831","nonce":1360655510}]},"public_addr":"172.21.15.45:6827/1360655510","cluster_addr":"172.21.15.45:6829/1360655510","heartbeat_back_addr":"172.21.15.45:6833/1360655510","heartbeat_front_addr":"172.21.15.45:6831/1360655510","state":["exists","up"]},{"osd":10,"uuid":"a2d0c488-1892-4da4-adbf-952c22df0ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6825","nonce":1701691877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6827","nonce":1701691877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6831","nonce":1701691877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6829","nonce":1701691877}]},"public_addr":"172.21.15.86:6825/1701691877","cluster_addr":"172.21.15.86:6827/1701691877","heartbeat_back_addr":"172.21.15.86:6831/1701691877","heartbeat_front_addr":"172.21.15.86:6829/1701691877","state":["exists","up"]},{"osd":11,"uuid":"6b349c4c-b672-480b-ac6a-8594dab0eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6825","nonce":2065648007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6827","nonce":2065648007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6831","nonce":2065648007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6829","nonce":2065648007}]},"public_addr":"172.21.15.154:6825/2065648007","cluster_addr":"172.21.15.154:6827/2065648007","heartbeat_back_addr":"172.21.15.154:6831/2065648007","heartbeat_front_addr":"172.21.15.154:6829/2065648007","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:21:37.147394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:00.776617+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:24.786849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:50.487455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:14.869272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:39.815158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:05.972619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:31.410063+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:56.795119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:23.114884+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:49.256504+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:26:15.251967+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/36732849":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/1500318775":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6801/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6800/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/2041380391":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/2504003341":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/3798552599":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/498952822":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6800/1562511056":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6801/1562511056":"2024-06-07T04:19:37.026199+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T04:28:53.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:53.040+0000 7fbd18b3e700 1 -- 172.21.15.45:0/465252763 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbcfc0a2cb0 msgr2=0x7fbcfc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:53.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:53.040+0000 7fbd18b3e700 1 --2- 172.21.15.45:0/465252763 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbcfc0a2cb0 0x7fbcfc0a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbd14199ab0 tx=0x7fbd0400b500 comp rx=0 tx=0).stop 2024-06-06T04:28:53.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:53.040+0000 7fbd18b3e700 1 -- 172.21.15.45:0/465252763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd14107420 msgr2=0x7fbd14198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:53.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:53.040+0000 7fbd18b3e700 1 --2- 172.21.15.45:0/465252763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd14107420 0x7fbd14198970 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c007f10 tx=0x7fbd0c00d3b0 comp rx=0 tx=0).stop 2024-06-06T04:28:53.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:53.040+0000 7fbd18b3e700 1 -- 172.21.15.45:0/465252763 shutdown_connections 2024-06-06T04:28:53.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:53.040+0000 7fbd18b3e700 1 --2- 172.21.15.45:0/465252763 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbcfc0a2cb0 0x7fbcfc0a5170 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:53.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:53.040+0000 7fbd18b3e700 1 --2- 172.21.15.45:0/465252763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd14107420 0x7fbd14198970 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:53.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:53.040+0000 7fbd18b3e700 1 --2- 172.21.15.45:0/465252763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd14101370 0x7fbd1419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:53.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:53.040+0000 7fbd18b3e700 1 --2- 172.21.15.45:0/465252763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd14101d30 0x7fbd1419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:53.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:53.040+0000 7fbd18b3e700 1 -- 172.21.15.45:0/465252763 >> 172.21.15.45:0/465252763 conn(0x7fbd14078340 msgr2=0x7fbd1406aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:53.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:53.040+0000 7fbd18b3e700 1 -- 172.21.15.45:0/465252763 shutdown_connections 2024-06-06T04:28:53.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:53.040+0000 7fbd18b3e700 1 -- 172.21.15.45:0/465252763 wait complete. 2024-06-06T04:28:53.720 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-06T04:28:53.721 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T04:28:53.975 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:53.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:53 smithi045 ceph-mon[32299]: pgmap v352: 129 pgs: 129 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-06-06T04:28:53.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:53 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/465252763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:28:54.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:53 smithi086 ceph-mon[36053]: pgmap v352: 129 pgs: 129 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-06-06T04:28:54.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:53 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/465252763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:28:54.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:53 smithi154 ceph-mon[36428]: pgmap v352: 129 pgs: 129 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-06-06T04:28:54.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:53 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/465252763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:28:55.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.667+0000 7fc5aa10a700 1 -- 172.21.15.45:0/1160397590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a4101d40 msgr2=0x7fc5a410e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.667+0000 7fc5aa10a700 1 --2- 172.21.15.45:0/1160397590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a4101d40 0x7fc5a410e230 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00b0a0 tx=0x7fc59c00b3b0 comp rx=0 tx=0).stop 2024-06-06T04:28:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.667+0000 7fc5aa10a700 1 -- 172.21.15.45:0/1160397590 shutdown_connections 2024-06-06T04:28:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.667+0000 7fc5aa10a700 1 --2- 172.21.15.45:0/1160397590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a4101d40 0x7fc5a410e230 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.667+0000 7fc5aa10a700 1 --2- 172.21.15.45:0/1160397590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5a41073f0 0x7fc5a41077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.667+0000 7fc5aa10a700 1 --2- 172.21.15.45:0/1160397590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5a4101380 0x7fc5a4101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.667+0000 7fc5aa10a700 1 -- 172.21.15.45:0/1160397590 >> 172.21.15.45:0/1160397590 conn(0x7fc5a4078300 msgr2=0x7fc5a4078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.668+0000 7fc5aa10a700 1 -- 172.21.15.45:0/1160397590 shutdown_connections 2024-06-06T04:28:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.668+0000 7fc5aa10a700 1 -- 172.21.15.45:0/1160397590 wait complete. 2024-06-06T04:28:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.668+0000 7fc5aa10a700 1 Processor -- start 2024-06-06T04:28:55.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5aa10a700 1 -- start start 2024-06-06T04:28:55.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5aa10a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a4101380 0x7fc5a41998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:55.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5aa10a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5a4101d40 0x7fc5a4199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:55.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5aa10a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5a41073f0 0x7fc5a41a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:55.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5aa10a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4111510 con 0x7fc5a4101380 2024-06-06T04:28:55.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5aa10a700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4111390 con 0x7fc5a41073f0 2024-06-06T04:28:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5aa10a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4111690 con 0x7fc5a4101d40 2024-06-06T04:28:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5a37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a4101380 0x7fc5a41998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5a37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a4101380 0x7fc5a41998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34858/0 (socket says 172.21.15.45:34858) 2024-06-06T04:28:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5a37fe700 1 -- 172.21.15.45:0/2765562858 learned_addr learned my addr 172.21.15.45:0/2765562858 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5a2ffd700 1 --2- 172.21.15.45:0/2765562858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5a4101d40 0x7fc5a4199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5a3fff700 1 --2- 172.21.15.45:0/2765562858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5a41073f0 0x7fc5a41a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5a37fe700 1 -- 172.21.15.45:0/2765562858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5a4101d40 msgr2=0x7fc5a4199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5a37fe700 1 --2- 172.21.15.45:0/2765562858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5a4101d40 0x7fc5a4199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5a37fe700 1 -- 172.21.15.45:0/2765562858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5a41073f0 msgr2=0x7fc5a41a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5a37fe700 1 --2- 172.21.15.45:0/2765562858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5a41073f0 0x7fc5a41a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.669+0000 7fc5a37fe700 1 -- 172.21.15.45:0/2765562858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a41a4eb0 con 0x7fc5a4101380 2024-06-06T04:28:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.670+0000 7fc5a37fe700 1 --2- 172.21.15.45:0/2765562858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a4101380 0x7fc5a41998f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc59400c920 tx=0x7fc59400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.670+0000 7fc5a0ff9700 1 -- 172.21.15.45:0/2765562858 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594007b00 con 0x7fc5a4101380 2024-06-06T04:28:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.670+0000 7fc5a0ff9700 1 -- 172.21.15.45:0/2765562858 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc594004ce0 con 0x7fc5a4101380 2024-06-06T04:28:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.670+0000 7fc5aa10a700 1 -- 172.21.15.45:0/2765562858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a41a50e0 con 0x7fc5a4101380 2024-06-06T04:28:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.670+0000 7fc5a0ff9700 1 -- 172.21.15.45:0/2765562858 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594005550 con 0x7fc5a4101380 2024-06-06T04:28:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.670+0000 7fc5aa10a700 1 -- 172.21.15.45:0/2765562858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a41a5640 con 0x7fc5a4101380 2024-06-06T04:28:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.671+0000 7fc5aa10a700 1 -- 172.21.15.45:0/2765562858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a4069180 con 0x7fc5a4101380 2024-06-06T04:28:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.672+0000 7fc5a0ff9700 1 -- 172.21.15.45:0/2765562858 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc594004750 con 0x7fc5a4101380 2024-06-06T04:28:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.673+0000 7fc5a0ff9700 1 --2- 172.21.15.45:0/2765562858 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc58c0a2d30 0x7fc58c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.673+0000 7fc5a2ffd700 1 --2- 172.21.15.45:0/2765562858 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc58c0a2d30 0x7fc58c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.673+0000 7fc5a0ff9700 1 -- 172.21.15.45:0/2765562858 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7fc5940cf390 con 0x7fc5a4101380 2024-06-06T04:28:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.673+0000 7fc5a2ffd700 1 --2- 172.21.15.45:0/2765562858 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc58c0a2d30 0x7fc58c0a51f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40788b0 tx=0x7fc598009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:55.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.677+0000 7fc5a0ff9700 1 -- 172.21.15.45:0/2765562858 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc594014030 con 0x7fc5a4101380 2024-06-06T04:28:55.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.881+0000 7fc5aa10a700 1 -- 172.21.15.45:0/2765562858 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc5a40628d0 con 0x7fc58c0a2d30 2024-06-06T04:28:55.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.892+0000 7fc5a0ff9700 1 -- 172.21.15.45:0/2765562858 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340616 (secure 0 0 0) 0x7fc5a40628d0 con 0x7fc58c0a2d30 2024-06-06T04:28:55.897 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:28:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.895+0000 7fc5aa10a700 1 -- 172.21.15.45:0/2765562858 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc58c0a2d30 msgr2=0x7fc58c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.895+0000 7fc5aa10a700 1 --2- 172.21.15.45:0/2765562858 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc58c0a2d30 0x7fc58c0a51f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40788b0 tx=0x7fc598009260 comp rx=0 tx=0).stop 2024-06-06T04:28:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.895+0000 7fc5aa10a700 1 -- 172.21.15.45:0/2765562858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a4101380 msgr2=0x7fc5a41998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.895+0000 7fc5aa10a700 1 --2- 172.21.15.45:0/2765562858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a4101380 0x7fc5a41998f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc59400c920 tx=0x7fc59400cc30 comp rx=0 tx=0).stop 2024-06-06T04:28:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.895+0000 7fc5aa10a700 1 -- 172.21.15.45:0/2765562858 shutdown_connections 2024-06-06T04:28:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.895+0000 7fc5aa10a700 1 --2- 172.21.15.45:0/2765562858 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fc58c0a2d30 0x7fc58c0a51f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.895+0000 7fc5aa10a700 1 --2- 172.21.15.45:0/2765562858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5a41073f0 0x7fc5a41a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.895+0000 7fc5aa10a700 1 --2- 172.21.15.45:0/2765562858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a4101380 0x7fc5a41998f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.895+0000 7fc5aa10a700 1 --2- 172.21.15.45:0/2765562858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5a4101d40 0x7fc5a4199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.895+0000 7fc5aa10a700 1 -- 172.21.15.45:0/2765562858 >> 172.21.15.45:0/2765562858 conn(0x7fc5a4078300 msgr2=0x7fc5a40fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.896+0000 7fc5aa10a700 1 -- 172.21.15.45:0/2765562858 shutdown_connections 2024-06-06T04:28:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:55.896+0000 7fc5aa10a700 1 -- 172.21.15.45:0/2765562858 wait complete. 2024-06-06T04:28:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-06-06T04:28:56.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:55 smithi086 ceph-mon[36053]: pgmap v353: 129 pgs: 129 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-06-06T04:28:56.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:55 smithi154 ceph-mon[36428]: pgmap v353: 129 pgs: 129 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-06-06T04:28:56.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:55 smithi045 ceph-mon[32299]: pgmap v353: 129 pgs: 129 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-06-06T04:28:56.727 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-06-06T04:28:54.380524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650760,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":287,"num_read_kb":255,"num_write":345,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374732,"kb_used_data":13100,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567092,"statfs":{"total":1151940427776,"available":1151556702208,"internally_reserved":0,"allocated":13414400,"data_stored":8576054,"data_compressed":426471,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27070,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":72,"num_read_kb":63,"num_write":18,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003696"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720845+0000","last_change":"2024-06-06T04:27:22.574900+0000","last_active":"2024-06-06T04:28:05.720845+0000","last_peered":"2024-06-06T04:28:05.720845+0000","last_clean":"2024-06-06T04:28:05.720845+0000","last_became_active":"2024-06-06T04:27:22.574389+0000","last_became_peered":"2024-06-06T04:27:22.574389+0000","last_unstale":"2024-06-06T04:28:05.720845+0000","last_undegraded":"2024-06-06T04:28:05.720845+0000","last_fullsized":"2024-06-06T04:28:05.720845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:39.246463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720674+0000","last_change":"2024-06-06T04:27:22.565543+0000","last_active":"2024-06-06T04:28:05.720674+0000","last_peered":"2024-06-06T04:28:05.720674+0000","last_clean":"2024-06-06T04:28:05.720674+0000","last_became_active":"2024-06-06T04:27:22.563833+0000","last_became_peered":"2024-06-06T04:27:22.563833+0000","last_unstale":"2024-06-06T04:28:05.720674+0000","last_undegraded":"2024-06-06T04:28:05.720674+0000","last_fullsized":"2024-06-06T04:28:05.720674+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:26:17.810782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720208+0000","last_change":"2024-06-06T04:27:22.570107+0000","last_active":"2024-06-06T04:28:05.720208+0000","last_peered":"2024-06-06T04:28:05.720208+0000","last_clean":"2024-06-06T04:28:05.720208+0000","last_became_active":"2024-06-06T04:27:22.569788+0000","last_became_peered":"2024-06-06T04:27:22.569788+0000","last_unstale":"2024-06-06T04:28:05.720208+0000","last_undegraded":"2024-06-06T04:28:05.720208+0000","last_fullsized":"2024-06-06T04:28:05.720208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:48:20.454782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717897+0000","last_change":"2024-06-06T04:27:22.559431+0000","last_active":"2024-06-06T04:28:05.717897+0000","last_peered":"2024-06-06T04:28:05.717897+0000","last_clean":"2024-06-06T04:28:05.717897+0000","last_became_active":"2024-06-06T04:27:22.558961+0000","last_became_peered":"2024-06-06T04:27:22.558961+0000","last_unstale":"2024-06-06T04:28:05.717897+0000","last_undegraded":"2024-06-06T04:28:05.717897+0000","last_fullsized":"2024-06-06T04:28:05.717897+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:59.445283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719982+0000","last_change":"2024-06-06T04:27:22.561598+0000","last_active":"2024-06-06T04:28:05.719982+0000","last_peered":"2024-06-06T04:28:05.719982+0000","last_clean":"2024-06-06T04:28:05.719982+0000","last_became_active":"2024-06-06T04:27:22.561198+0000","last_became_peered":"2024-06-06T04:27:22.561198+0000","last_unstale":"2024-06-06T04:28:05.719982+0000","last_undegraded":"2024-06-06T04:28:05.719982+0000","last_fullsized":"2024-06-06T04:28:05.719982+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:20:28.314941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719457+0000","last_change":"2024-06-06T04:27:22.569395+0000","last_active":"2024-06-06T04:28:05.719457+0000","last_peered":"2024-06-06T04:28:05.719457+0000","last_clean":"2024-06-06T04:28:05.719457+0000","last_became_active":"2024-06-06T04:27:22.569054+0000","last_became_peered":"2024-06-06T04:27:22.569054+0000","last_unstale":"2024-06-06T04:28:05.719457+0000","last_undegraded":"2024-06-06T04:28:05.719457+0000","last_fullsized":"2024-06-06T04:28:05.719457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:03:56.630677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724921+0000","last_change":"2024-06-06T04:27:22.577121+0000","last_active":"2024-06-06T04:28:05.724921+0000","last_peered":"2024-06-06T04:28:05.724921+0000","last_clean":"2024-06-06T04:28:05.724921+0000","last_became_active":"2024-06-06T04:27:22.575574+0000","last_became_peered":"2024-06-06T04:27:22.575574+0000","last_unstale":"2024-06-06T04:28:05.724921+0000","last_undegraded":"2024-06-06T04:28:05.724921+0000","last_fullsized":"2024-06-06T04:28:05.724921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:05:16.799335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.716776+0000","last_change":"2024-06-06T04:27:22.570051+0000","last_active":"2024-06-06T04:28:05.716776+0000","last_peered":"2024-06-06T04:28:05.716776+0000","last_clean":"2024-06-06T04:28:05.716776+0000","last_became_active":"2024-06-06T04:27:22.569640+0000","last_became_peered":"2024-06-06T04:27:22.569640+0000","last_unstale":"2024-06-06T04:28:05.716776+0000","last_undegraded":"2024-06-06T04:28:05.716776+0000","last_fullsized":"2024-06-06T04:28:05.716776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:48:00.744167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.721268+0000","last_change":"2024-06-06T04:27:22.574900+0000","last_active":"2024-06-06T04:28:05.721268+0000","last_peered":"2024-06-06T04:28:05.721268+0000","last_clean":"2024-06-06T04:28:05.721268+0000","last_became_active":"2024-06-06T04:27:22.574570+0000","last_became_peered":"2024-06-06T04:27:22.574570+0000","last_unstale":"2024-06-06T04:28:05.721268+0000","last_undegraded":"2024-06-06T04:28:05.721268+0000","last_fullsized":"2024-06-06T04:28:05.721268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:54.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720075+0000","last_change":"2024-06-06T04:27:22.570581+0000","last_active":"2024-06-06T04:28:05.720075+0000","last_peered":"2024-06-06T04:28:05.720075+0000","last_clean":"2024-06-06T04:28:05.720075+0000","last_became_active":"2024-06-06T04:27:22.570243+0000","last_became_peered":"2024-06-06T04:27:22.570243+0000","last_unstale":"2024-06-06T04:28:05.720075+0000","last_undegraded":"2024-06-06T04:28:05.720075+0000","last_fullsized":"2024-06-06T04:28:05.720075+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:27.087527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725926+0000","last_change":"2024-06-06T04:27:22.571162+0000","last_active":"2024-06-06T04:28:05.725926+0000","last_peered":"2024-06-06T04:28:05.725926+0000","last_clean":"2024-06-06T04:28:05.725926+0000","last_became_active":"2024-06-06T04:27:22.570843+0000","last_became_peered":"2024-06-06T04:27:22.570843+0000","last_unstale":"2024-06-06T04:28:05.725926+0000","last_undegraded":"2024-06-06T04:28:05.725926+0000","last_fullsized":"2024-06-06T04:28:05.725926+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:43:05.590240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717804+0000","last_change":"2024-06-06T04:27:22.567420+0000","last_active":"2024-06-06T04:28:05.717804+0000","last_peered":"2024-06-06T04:28:05.717804+0000","last_clean":"2024-06-06T04:28:05.717804+0000","last_became_active":"2024-06-06T04:27:22.566750+0000","last_became_peered":"2024-06-06T04:27:22.566750+0000","last_unstale":"2024-06-06T04:28:05.717804+0000","last_undegraded":"2024-06-06T04:28:05.717804+0000","last_fullsized":"2024-06-06T04:28:05.717804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:21.669965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719673+0000","last_change":"2024-06-06T04:27:22.566831+0000","last_active":"2024-06-06T04:28:05.719673+0000","last_peered":"2024-06-06T04:28:05.719673+0000","last_clean":"2024-06-06T04:28:05.719673+0000","last_became_active":"2024-06-06T04:27:22.566255+0000","last_became_peered":"2024-06-06T04:27:22.566255+0000","last_unstale":"2024-06-06T04:28:05.719673+0000","last_undegraded":"2024-06-06T04:28:05.719673+0000","last_fullsized":"2024-06-06T04:28:05.719673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:40:22.486674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720828+0000","last_change":"2024-06-06T04:27:22.564596+0000","last_active":"2024-06-06T04:28:05.720828+0000","last_peered":"2024-06-06T04:28:05.720828+0000","last_clean":"2024-06-06T04:28:05.720828+0000","last_became_active":"2024-06-06T04:27:22.563694+0000","last_became_peered":"2024-06-06T04:27:22.563694+0000","last_unstale":"2024-06-06T04:28:05.720828+0000","last_undegraded":"2024-06-06T04:28:05.720828+0000","last_fullsized":"2024-06-06T04:28:05.720828+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:18.568266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.721206+0000","last_change":"2024-06-06T04:27:22.559395+0000","last_active":"2024-06-06T04:28:05.721206+0000","last_peered":"2024-06-06T04:28:05.721206+0000","last_clean":"2024-06-06T04:28:05.721206+0000","last_became_active":"2024-06-06T04:27:22.559030+0000","last_became_peered":"2024-06-06T04:27:22.559030+0000","last_unstale":"2024-06-06T04:28:05.721206+0000","last_undegraded":"2024-06-06T04:28:05.721206+0000","last_fullsized":"2024-06-06T04:28:05.721206+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:08.562184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719969+0000","last_change":"2024-06-06T04:27:22.569657+0000","last_active":"2024-06-06T04:28:05.719969+0000","last_peered":"2024-06-06T04:28:05.719969+0000","last_clean":"2024-06-06T04:28:05.719969+0000","last_became_active":"2024-06-06T04:27:22.569359+0000","last_became_peered":"2024-06-06T04:27:22.569359+0000","last_unstale":"2024-06-06T04:28:05.719969+0000","last_undegraded":"2024-06-06T04:28:05.719969+0000","last_fullsized":"2024-06-06T04:28:05.719969+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:42:20.197715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720550+0000","last_change":"2024-06-06T04:27:22.574661+0000","last_active":"2024-06-06T04:28:05.720550+0000","last_peered":"2024-06-06T04:28:05.720550+0000","last_clean":"2024-06-06T04:28:05.720550+0000","last_became_active":"2024-06-06T04:27:22.574310+0000","last_became_peered":"2024-06-06T04:27:22.574310+0000","last_unstale":"2024-06-06T04:28:05.720550+0000","last_undegraded":"2024-06-06T04:28:05.720550+0000","last_fullsized":"2024-06-06T04:28:05.720550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:45:18.772270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.895238+0000","last_change":"2024-06-06T04:27:22.565575+0000","last_active":"2024-06-06T04:28:05.895238+0000","last_peered":"2024-06-06T04:28:05.895238+0000","last_clean":"2024-06-06T04:28:05.895238+0000","last_became_active":"2024-06-06T04:27:22.565256+0000","last_became_peered":"2024-06-06T04:27:22.565256+0000","last_unstale":"2024-06-06T04:28:05.895238+0000","last_undegraded":"2024-06-06T04:28:05.895238+0000","last_fullsized":"2024-06-06T04:28:05.895238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:49.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.718703+0000","last_change":"2024-06-06T04:27:22.559606+0000","last_active":"2024-06-06T04:28:05.718703+0000","last_peered":"2024-06-06T04:28:05.718703+0000","last_clean":"2024-06-06T04:28:05.718703+0000","last_became_active":"2024-06-06T04:27:22.557881+0000","last_became_peered":"2024-06-06T04:27:22.557881+0000","last_unstale":"2024-06-06T04:28:05.718703+0000","last_undegraded":"2024-06-06T04:28:05.718703+0000","last_fullsized":"2024-06-06T04:28:05.718703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:31:06.269785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717560+0000","last_change":"2024-06-06T04:27:32.626612+0000","last_active":"2024-06-06T04:28:05.717560+0000","last_peered":"2024-06-06T04:28:05.717560+0000","last_clean":"2024-06-06T04:28:05.717560+0000","last_became_active":"2024-06-06T04:27:32.626113+0000","last_became_peered":"2024-06-06T04:27:32.626113+0000","last_unstale":"2024-06-06T04:28:05.717560+0000","last_undegraded":"2024-06-06T04:28:05.717560+0000","last_fullsized":"2024-06-06T04:28:05.717560+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.318738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":50,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:27.721774+0000","last_change":"2024-06-06T04:27:19.302744+0000","last_active":"2024-06-06T04:28:27.721774+0000","last_peered":"2024-06-06T04:28:27.721774+0000","last_clean":"2024-06-06T04:28:27.721774+0000","last_became_active":"2024-06-06T04:27:19.302389+0000","last_became_peered":"2024-06-06T04:27:19.302389+0000","last_unstale":"2024-06-06T04:28:27.721774+0000","last_undegraded":"2024-06-06T04:28:27.721774+0000","last_fullsized":"2024-06-06T04:28:27.721774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:49.872618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717319+0000","last_change":"2024-06-06T04:27:22.570220+0000","last_active":"2024-06-06T04:28:05.717319+0000","last_peered":"2024-06-06T04:28:05.717319+0000","last_clean":"2024-06-06T04:28:05.717319+0000","last_became_active":"2024-06-06T04:27:22.569835+0000","last_became_peered":"2024-06-06T04:27:22.569835+0000","last_unstale":"2024-06-06T04:28:05.717319+0000","last_undegraded":"2024-06-06T04:28:05.717319+0000","last_fullsized":"2024-06-06T04:28:05.717319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:54:57.372561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717188+0000","last_change":"2024-06-06T04:27:32.622407+0000","last_active":"2024-06-06T04:28:05.717188+0000","last_peered":"2024-06-06T04:28:05.717188+0000","last_clean":"2024-06-06T04:28:05.717188+0000","last_became_active":"2024-06-06T04:27:32.621868+0000","last_became_peered":"2024-06-06T04:27:32.621868+0000","last_unstale":"2024-06-06T04:28:05.717188+0000","last_undegraded":"2024-06-06T04:28:05.717188+0000","last_fullsized":"2024-06-06T04:28:05.717188+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:50:05.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717512+0000","last_change":"2024-06-06T04:27:19.516235+0000","last_active":"2024-06-06T04:28:05.717512+0000","last_peered":"2024-06-06T04:28:05.717512+0000","last_clean":"2024-06-06T04:28:05.717512+0000","last_became_active":"2024-06-06T04:27:19.515849+0000","last_became_peered":"2024-06-06T04:27:19.515849+0000","last_unstale":"2024-06-06T04:28:05.717512+0000","last_undegraded":"2024-06-06T04:28:05.717512+0000","last_fullsized":"2024-06-06T04:28:05.717512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:52:42.722103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720330+0000","last_change":"2024-06-06T04:27:22.570797+0000","last_active":"2024-06-06T04:28:05.720330+0000","last_peered":"2024-06-06T04:28:05.720330+0000","last_clean":"2024-06-06T04:28:05.720330+0000","last_became_active":"2024-06-06T04:27:22.570305+0000","last_became_peered":"2024-06-06T04:27:22.570305+0000","last_unstale":"2024-06-06T04:28:05.720330+0000","last_undegraded":"2024-06-06T04:28:05.720330+0000","last_fullsized":"2024-06-06T04:28:05.720330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:04:14.545306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717388+0000","last_change":"2024-06-06T04:27:32.623241+0000","last_active":"2024-06-06T04:28:05.717388+0000","last_peered":"2024-06-06T04:28:05.717388+0000","last_clean":"2024-06-06T04:28:05.717388+0000","last_became_active":"2024-06-06T04:27:32.622751+0000","last_became_peered":"2024-06-06T04:27:32.622751+0000","last_unstale":"2024-06-06T04:28:05.717388+0000","last_undegraded":"2024-06-06T04:28:05.717388+0000","last_fullsized":"2024-06-06T04:28:05.717388+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:24:53.446329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:27.719217+0000","last_change":"2024-06-06T04:28:05.733350+0000","last_active":"2024-06-06T04:28:27.719217+0000","last_peered":"2024-06-06T04:28:27.719217+0000","last_clean":"2024-06-06T04:28:27.719217+0000","last_became_active":"2024-06-06T04:28:05.726556+0000","last_became_peered":"2024-06-06T04:28:05.726556+0000","last_unstale":"2024-06-06T04:28:27.719217+0000","last_undegraded":"2024-06-06T04:28:27.719217+0000","last_fullsized":"2024-06-06T04:28:27.719217+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:26.636625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720742+0000","last_change":"2024-06-06T04:27:22.571731+0000","last_active":"2024-06-06T04:28:05.720742+0000","last_peered":"2024-06-06T04:28:05.720742+0000","last_clean":"2024-06-06T04:28:05.720742+0000","last_became_active":"2024-06-06T04:27:22.571398+0000","last_became_peered":"2024-06-06T04:27:22.571398+0000","last_unstale":"2024-06-06T04:28:05.720742+0000","last_undegraded":"2024-06-06T04:28:05.720742+0000","last_fullsized":"2024-06-06T04:28:05.720742+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:22:45.144427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717172+0000","last_change":"2024-06-06T04:27:32.618465+0000","last_active":"2024-06-06T04:28:05.717172+0000","last_peered":"2024-06-06T04:28:05.717172+0000","last_clean":"2024-06-06T04:28:05.717172+0000","last_became_active":"2024-06-06T04:27:32.618066+0000","last_became_peered":"2024-06-06T04:27:32.618066+0000","last_unstale":"2024-06-06T04:28:05.717172+0000","last_undegraded":"2024-06-06T04:28:05.717172+0000","last_fullsized":"2024-06-06T04:28:05.717172+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:54:23.988687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.730858+0000","last_change":"2024-06-06T04:28:05.730381+0000","last_active":"2024-06-06T04:28:05.730858+0000","last_peered":"2024-06-06T04:28:05.730858+0000","last_clean":"2024-06-06T04:28:05.730858+0000","last_became_active":"2024-06-06T04:28:05.726552+0000","last_became_peered":"2024-06-06T04:28:05.726552+0000","last_unstale":"2024-06-06T04:28:05.730858+0000","last_undegraded":"2024-06-06T04:28:05.730858+0000","last_fullsized":"2024-06-06T04:28:05.730858+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:52.903042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.726700+0000","last_change":"2024-06-06T04:27:22.573769+0000","last_active":"2024-06-06T04:28:05.726700+0000","last_peered":"2024-06-06T04:28:05.726700+0000","last_clean":"2024-06-06T04:28:05.726700+0000","last_became_active":"2024-06-06T04:27:22.572674+0000","last_became_peered":"2024-06-06T04:27:22.572674+0000","last_unstale":"2024-06-06T04:28:05.726700+0000","last_undegraded":"2024-06-06T04:28:05.726700+0000","last_fullsized":"2024-06-06T04:28:05.726700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:57:22.653052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.718601+0000","last_change":"2024-06-06T04:27:32.617934+0000","last_active":"2024-06-06T04:28:05.718601+0000","last_peered":"2024-06-06T04:28:05.718601+0000","last_clean":"2024-06-06T04:28:05.718601+0000","last_became_active":"2024-06-06T04:27:32.617537+0000","last_became_peered":"2024-06-06T04:27:32.617537+0000","last_unstale":"2024-06-06T04:28:05.718601+0000","last_undegraded":"2024-06-06T04:28:05.718601+0000","last_fullsized":"2024-06-06T04:28:05.718601+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:21:48.319434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'5","reported_seq":69,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:49.762915+0000","last_change":"2024-06-06T04:27:19.306236+0000","last_active":"2024-06-06T04:28:49.762915+0000","last_peered":"2024-06-06T04:28:49.762915+0000","last_clean":"2024-06-06T04:28:49.762915+0000","last_became_active":"2024-06-06T04:27:19.305850+0000","last_became_peered":"2024-06-06T04:27:19.305850+0000","last_unstale":"2024-06-06T04:28:49.762915+0000","last_undegraded":"2024-06-06T04:28:49.762915+0000","last_fullsized":"2024-06-06T04:28:49.762915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:58:47.310040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.721086+0000","last_change":"2024-06-06T04:27:22.567440+0000","last_active":"2024-06-06T04:28:05.721086+0000","last_peered":"2024-06-06T04:28:05.721086+0000","last_clean":"2024-06-06T04:28:05.721086+0000","last_became_active":"2024-06-06T04:27:22.566732+0000","last_became_peered":"2024-06-06T04:27:22.566732+0000","last_unstale":"2024-06-06T04:28:05.721086+0000","last_undegraded":"2024-06-06T04:28:05.721086+0000","last_fullsized":"2024-06-06T04:28:05.721086+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:48:46.270103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.718261+0000","last_change":"2024-06-06T04:27:32.626554+0000","last_active":"2024-06-06T04:28:05.718261+0000","last_peered":"2024-06-06T04:28:05.718261+0000","last_clean":"2024-06-06T04:28:05.718261+0000","last_became_active":"2024-06-06T04:27:32.626108+0000","last_became_peered":"2024-06-06T04:27:32.626108+0000","last_unstale":"2024-06-06T04:28:05.718261+0000","last_undegraded":"2024-06-06T04:28:05.718261+0000","last_fullsized":"2024-06-06T04:28:05.718261+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:37:43.811005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:27.718526+0000","last_change":"2024-06-06T04:27:19.306821+0000","last_active":"2024-06-06T04:28:27.718526+0000","last_peered":"2024-06-06T04:28:27.718526+0000","last_clean":"2024-06-06T04:28:27.718526+0000","last_became_active":"2024-06-06T04:27:19.306489+0000","last_became_peered":"2024-06-06T04:27:19.306489+0000","last_unstale":"2024-06-06T04:28:27.718526+0000","last_undegraded":"2024-06-06T04:28:27.718526+0000","last_fullsized":"2024-06-06T04:28:27.718526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:31:28.973112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.726240+0000","last_change":"2024-06-06T04:27:22.570617+0000","last_active":"2024-06-06T04:28:05.726240+0000","last_peered":"2024-06-06T04:28:05.726240+0000","last_clean":"2024-06-06T04:28:05.726240+0000","last_became_active":"2024-06-06T04:27:22.570271+0000","last_became_peered":"2024-06-06T04:27:22.570271+0000","last_unstale":"2024-06-06T04:28:05.726240+0000","last_undegraded":"2024-06-06T04:28:05.726240+0000","last_fullsized":"2024-06-06T04:28:05.726240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:45:33.890164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720406+0000","last_change":"2024-06-06T04:27:32.628132+0000","last_active":"2024-06-06T04:28:05.720406+0000","last_peered":"2024-06-06T04:28:05.720406+0000","last_clean":"2024-06-06T04:28:05.720406+0000","last_became_active":"2024-06-06T04:27:32.627658+0000","last_became_peered":"2024-06-06T04:27:32.627658+0000","last_unstale":"2024-06-06T04:28:05.720406+0000","last_undegraded":"2024-06-06T04:28:05.720406+0000","last_fullsized":"2024-06-06T04:28:05.720406+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:24.817079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:27.718694+0000","last_change":"2024-06-06T04:27:19.300510+0000","last_active":"2024-06-06T04:28:27.718694+0000","last_peered":"2024-06-06T04:28:27.718694+0000","last_clean":"2024-06-06T04:28:27.718694+0000","last_became_active":"2024-06-06T04:27:19.300150+0000","last_became_peered":"2024-06-06T04:27:19.300150+0000","last_unstale":"2024-06-06T04:28:27.718694+0000","last_undegraded":"2024-06-06T04:28:27.718694+0000","last_fullsized":"2024-06-06T04:28:27.718694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:38:25.084690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.716894+0000","last_change":"2024-06-06T04:27:22.563864+0000","last_active":"2024-06-06T04:28:05.716894+0000","last_peered":"2024-06-06T04:28:05.716894+0000","last_clean":"2024-06-06T04:28:05.716894+0000","last_became_active":"2024-06-06T04:27:22.563555+0000","last_became_peered":"2024-06-06T04:27:22.563555+0000","last_unstale":"2024-06-06T04:28:05.716894+0000","last_undegraded":"2024-06-06T04:28:05.716894+0000","last_fullsized":"2024-06-06T04:28:05.716894+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:39:33.707404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720390+0000","last_change":"2024-06-06T04:27:32.617627+0000","last_active":"2024-06-06T04:28:05.720390+0000","last_peered":"2024-06-06T04:28:05.720390+0000","last_clean":"2024-06-06T04:28:05.720390+0000","last_became_active":"2024-06-06T04:27:32.617187+0000","last_became_peered":"2024-06-06T04:27:32.617187+0000","last_unstale":"2024-06-06T04:28:05.720390+0000","last_undegraded":"2024-06-06T04:28:05.720390+0000","last_fullsized":"2024-06-06T04:28:05.720390+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:24:52.554176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724506+0000","last_change":"2024-06-06T04:27:19.304316+0000","last_active":"2024-06-06T04:28:05.724506+0000","last_peered":"2024-06-06T04:28:05.724506+0000","last_clean":"2024-06-06T04:28:05.724506+0000","last_became_active":"2024-06-06T04:27:19.303952+0000","last_became_peered":"2024-06-06T04:27:19.303952+0000","last_unstale":"2024-06-06T04:28:05.724506+0000","last_undegraded":"2024-06-06T04:28:05.724506+0000","last_fullsized":"2024-06-06T04:28:05.724506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:26.886916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.894742+0000","last_change":"2024-06-06T04:27:22.561824+0000","last_active":"2024-06-06T04:28:05.894742+0000","last_peered":"2024-06-06T04:28:05.894742+0000","last_clean":"2024-06-06T04:28:05.894742+0000","last_became_active":"2024-06-06T04:27:22.561495+0000","last_became_peered":"2024-06-06T04:27:22.561495+0000","last_unstale":"2024-06-06T04:28:05.894742+0000","last_undegraded":"2024-06-06T04:28:05.894742+0000","last_fullsized":"2024-06-06T04:28:05.894742+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:40:31.314627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.716867+0000","last_change":"2024-06-06T04:27:32.624540+0000","last_active":"2024-06-06T04:28:05.716867+0000","last_peered":"2024-06-06T04:28:05.716867+0000","last_clean":"2024-06-06T04:28:05.716867+0000","last_became_active":"2024-06-06T04:27:32.624036+0000","last_became_peered":"2024-06-06T04:27:32.624036+0000","last_unstale":"2024-06-06T04:28:05.716867+0000","last_undegraded":"2024-06-06T04:28:05.716867+0000","last_fullsized":"2024-06-06T04:28:05.716867+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:16.983576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":63,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:49.762365+0000","last_change":"2024-06-06T04:27:19.304614+0000","last_active":"2024-06-06T04:28:49.762365+0000","last_peered":"2024-06-06T04:28:49.762365+0000","last_clean":"2024-06-06T04:28:49.762365+0000","last_became_active":"2024-06-06T04:27:19.304126+0000","last_became_peered":"2024-06-06T04:27:19.304126+0000","last_unstale":"2024-06-06T04:28:49.762365+0000","last_undegraded":"2024-06-06T04:28:49.762365+0000","last_fullsized":"2024-06-06T04:28:49.762365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:07:39.908630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720767+0000","last_change":"2024-06-06T04:27:22.572534+0000","last_active":"2024-06-06T04:28:05.720767+0000","last_peered":"2024-06-06T04:28:05.720767+0000","last_clean":"2024-06-06T04:28:05.720767+0000","last_became_active":"2024-06-06T04:27:22.572214+0000","last_became_peered":"2024-06-06T04:27:22.572214+0000","last_unstale":"2024-06-06T04:28:05.720767+0000","last_undegraded":"2024-06-06T04:28:05.720767+0000","last_fullsized":"2024-06-06T04:28:05.720767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:13.176045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724101+0000","last_change":"2024-06-06T04:28:05.723450+0000","last_active":"2024-06-06T04:28:05.724101+0000","last_peered":"2024-06-06T04:28:05.724101+0000","last_clean":"2024-06-06T04:28:05.724101+0000","last_became_active":"2024-06-06T04:28:05.722928+0000","last_became_peered":"2024-06-06T04:28:05.722928+0000","last_unstale":"2024-06-06T04:28:05.724101+0000","last_undegraded":"2024-06-06T04:28:05.724101+0000","last_fullsized":"2024-06-06T04:28:05.724101+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:53.093276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:26.712609+0000","last_change":"2024-06-06T04:27:19.308465+0000","last_active":"2024-06-06T04:28:26.712609+0000","last_peered":"2024-06-06T04:28:26.712609+0000","last_clean":"2024-06-06T04:28:26.712609+0000","last_became_active":"2024-06-06T04:27:19.308077+0000","last_became_peered":"2024-06-06T04:27:19.308077+0000","last_unstale":"2024-06-06T04:28:26.712609+0000","last_undegraded":"2024-06-06T04:28:26.712609+0000","last_fullsized":"2024-06-06T04:28:26.712609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:37:48.282613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720627+0000","last_change":"2024-06-06T04:27:22.574141+0000","last_active":"2024-06-06T04:28:05.720627+0000","last_peered":"2024-06-06T04:28:05.720627+0000","last_clean":"2024-06-06T04:28:05.720627+0000","last_became_active":"2024-06-06T04:27:22.573679+0000","last_became_peered":"2024-06-06T04:27:22.573679+0000","last_unstale":"2024-06-06T04:28:05.720627+0000","last_undegraded":"2024-06-06T04:28:05.720627+0000","last_fullsized":"2024-06-06T04:28:05.720627+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:27:25.177990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717527+0000","last_change":"2024-06-06T04:27:32.624148+0000","last_active":"2024-06-06T04:28:05.717527+0000","last_peered":"2024-06-06T04:28:05.717527+0000","last_clean":"2024-06-06T04:28:05.717527+0000","last_became_active":"2024-06-06T04:27:32.623448+0000","last_became_peered":"2024-06-06T04:27:32.623448+0000","last_unstale":"2024-06-06T04:28:05.717527+0000","last_undegraded":"2024-06-06T04:28:05.717527+0000","last_fullsized":"2024-06-06T04:28:05.717527+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:02.037296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:26.713006+0000","last_change":"2024-06-06T04:27:19.516875+0000","last_active":"2024-06-06T04:28:26.713006+0000","last_peered":"2024-06-06T04:28:26.713006+0000","last_clean":"2024-06-06T04:28:26.713006+0000","last_became_active":"2024-06-06T04:27:19.516393+0000","last_became_peered":"2024-06-06T04:27:19.516393+0000","last_unstale":"2024-06-06T04:28:26.713006+0000","last_undegraded":"2024-06-06T04:28:26.713006+0000","last_fullsized":"2024-06-06T04:28:26.713006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:20.079218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724043+0000","last_change":"2024-06-06T04:28:05.724043+0000","last_active":"2024-06-06T04:28:05.724043+0000","last_peered":"2024-06-06T04:28:05.724043+0000","last_clean":"2024-06-06T04:28:05.724043+0000","last_became_active":"2024-06-06T04:28:05.723622+0000","last_became_peered":"2024-06-06T04:28:05.723622+0000","last_unstale":"2024-06-06T04:28:05.724043+0000","last_undegraded":"2024-06-06T04:28:05.724043+0000","last_fullsized":"2024-06-06T04:28:05.724043+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:42.162400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.894796+0000","last_change":"2024-06-06T04:27:32.626366+0000","last_active":"2024-06-06T04:28:05.894796+0000","last_peered":"2024-06-06T04:28:05.894796+0000","last_clean":"2024-06-06T04:28:05.894796+0000","last_became_active":"2024-06-06T04:27:32.625847+0000","last_became_peered":"2024-06-06T04:27:32.625847+0000","last_unstale":"2024-06-06T04:28:05.894796+0000","last_undegraded":"2024-06-06T04:28:05.894796+0000","last_fullsized":"2024-06-06T04:28:05.894796+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:47:44.167531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"98'30","reported_seq":119,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:50.764138+0000","last_change":"2024-06-06T04:27:19.303581+0000","last_active":"2024-06-06T04:28:50.764138+0000","last_peered":"2024-06-06T04:28:50.764138+0000","last_clean":"2024-06-06T04:28:50.764138+0000","last_became_active":"2024-06-06T04:27:19.303234+0000","last_became_peered":"2024-06-06T04:27:19.303234+0000","last_unstale":"2024-06-06T04:28:50.764138+0000","last_undegraded":"2024-06-06T04:28:50.764138+0000","last_fullsized":"2024-06-06T04:28:50.764138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:37.721588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56158,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":51,"num_read_kb":81,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719957+0000","last_change":"2024-06-06T04:27:22.567177+0000","last_active":"2024-06-06T04:28:05.719957+0000","last_peered":"2024-06-06T04:28:05.719957+0000","last_clean":"2024-06-06T04:28:05.719957+0000","last_became_active":"2024-06-06T04:27:22.566751+0000","last_became_peered":"2024-06-06T04:27:22.566751+0000","last_unstale":"2024-06-06T04:28:05.719957+0000","last_undegraded":"2024-06-06T04:28:05.719957+0000","last_fullsized":"2024-06-06T04:28:05.719957+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:37:15.914505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.727434+0000","last_change":"2024-06-06T04:27:32.619319+0000","last_active":"2024-06-06T04:28:05.727434+0000","last_peered":"2024-06-06T04:28:05.727434+0000","last_clean":"2024-06-06T04:28:05.727434+0000","last_became_active":"2024-06-06T04:27:32.618843+0000","last_became_peered":"2024-06-06T04:27:32.618843+0000","last_unstale":"2024-06-06T04:28:05.727434+0000","last_undegraded":"2024-06-06T04:28:05.727434+0000","last_fullsized":"2024-06-06T04:28:05.727434+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:32:20.071512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:27.720338+0000","last_change":"2024-06-06T04:27:19.304731+0000","last_active":"2024-06-06T04:28:27.720338+0000","last_peered":"2024-06-06T04:28:27.720338+0000","last_clean":"2024-06-06T04:28:27.720338+0000","last_became_active":"2024-06-06T04:27:19.304351+0000","last_became_peered":"2024-06-06T04:27:19.304351+0000","last_unstale":"2024-06-06T04:28:27.720338+0000","last_undegraded":"2024-06-06T04:28:27.720338+0000","last_fullsized":"2024-06-06T04:28:27.720338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:16:14.136633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719354+0000","last_change":"2024-06-06T04:27:22.573223+0000","last_active":"2024-06-06T04:28:05.719354+0000","last_peered":"2024-06-06T04:28:05.719354+0000","last_clean":"2024-06-06T04:28:05.719354+0000","last_became_active":"2024-06-06T04:27:22.572884+0000","last_became_peered":"2024-06-06T04:27:22.572884+0000","last_unstale":"2024-06-06T04:28:05.719354+0000","last_undegraded":"2024-06-06T04:28:05.719354+0000","last_fullsized":"2024-06-06T04:28:05.719354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:42:11.819847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.721922+0000","last_change":"2024-06-06T04:27:32.628645+0000","last_active":"2024-06-06T04:28:05.721922+0000","last_peered":"2024-06-06T04:28:05.721922+0000","last_clean":"2024-06-06T04:28:05.721922+0000","last_became_active":"2024-06-06T04:27:32.627586+0000","last_became_peered":"2024-06-06T04:27:32.627586+0000","last_unstale":"2024-06-06T04:28:05.721922+0000","last_undegraded":"2024-06-06T04:28:05.721922+0000","last_fullsized":"2024-06-06T04:28:05.721922+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:46:53.094511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:26.714189+0000","last_change":"2024-06-06T04:27:19.303805+0000","last_active":"2024-06-06T04:28:26.714189+0000","last_peered":"2024-06-06T04:28:26.714189+0000","last_clean":"2024-06-06T04:28:26.714189+0000","last_became_active":"2024-06-06T04:27:19.303391+0000","last_became_peered":"2024-06-06T04:27:19.303391+0000","last_unstale":"2024-06-06T04:28:26.714189+0000","last_undegraded":"2024-06-06T04:28:26.714189+0000","last_fullsized":"2024-06-06T04:28:26.714189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:30:27.336567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725582+0000","last_change":"2024-06-06T04:27:22.575148+0000","last_active":"2024-06-06T04:28:05.725582+0000","last_peered":"2024-06-06T04:28:05.725582+0000","last_clean":"2024-06-06T04:28:05.725582+0000","last_became_active":"2024-06-06T04:27:22.574591+0000","last_became_peered":"2024-06-06T04:27:22.574591+0000","last_unstale":"2024-06-06T04:28:05.725582+0000","last_undegraded":"2024-06-06T04:28:05.725582+0000","last_fullsized":"2024-06-06T04:28:05.725582+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:35:23.404644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.728254+0000","last_change":"2024-06-06T04:28:05.726734+0000","last_active":"2024-06-06T04:28:05.728254+0000","last_peered":"2024-06-06T04:28:05.728254+0000","last_clean":"2024-06-06T04:28:05.728254+0000","last_became_active":"2024-06-06T04:28:05.726238+0000","last_became_peered":"2024-06-06T04:28:05.726238+0000","last_unstale":"2024-06-06T04:28:05.728254+0000","last_undegraded":"2024-06-06T04:28:05.728254+0000","last_fullsized":"2024-06-06T04:28:05.728254+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:47.841475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"98'4","reported_seq":69,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:50.764034+0000","last_change":"2024-06-06T04:27:19.305571+0000","last_active":"2024-06-06T04:28:50.764034+0000","last_peered":"2024-06-06T04:28:50.764034+0000","last_clean":"2024-06-06T04:28:50.764034+0000","last_became_active":"2024-06-06T04:27:19.305183+0000","last_became_peered":"2024-06-06T04:27:19.305183+0000","last_unstale":"2024-06-06T04:28:50.764034+0000","last_undegraded":"2024-06-06T04:28:50.764034+0000","last_fullsized":"2024-06-06T04:28:50.764034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:51.205209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724269+0000","last_change":"2024-06-06T04:27:22.571135+0000","last_active":"2024-06-06T04:28:05.724269+0000","last_peered":"2024-06-06T04:28:05.724269+0000","last_clean":"2024-06-06T04:28:05.724269+0000","last_became_active":"2024-06-06T04:27:22.570352+0000","last_became_peered":"2024-06-06T04:27:22.570352+0000","last_unstale":"2024-06-06T04:28:05.724269+0000","last_undegraded":"2024-06-06T04:28:05.724269+0000","last_fullsized":"2024-06-06T04:28:05.724269+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:41:47.059281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724892+0000","last_change":"2024-06-06T04:28:05.724892+0000","last_active":"2024-06-06T04:28:05.724892+0000","last_peered":"2024-06-06T04:28:05.724892+0000","last_clean":"2024-06-06T04:28:05.724892+0000","last_became_active":"2024-06-06T04:28:05.724420+0000","last_became_peered":"2024-06-06T04:28:05.724420+0000","last_unstale":"2024-06-06T04:28:05.724892+0000","last_undegraded":"2024-06-06T04:28:05.724892+0000","last_fullsized":"2024-06-06T04:28:05.724892+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:39.879481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":55,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:27.721685+0000","last_change":"2024-06-06T04:27:19.516687+0000","last_active":"2024-06-06T04:28:27.721685+0000","last_peered":"2024-06-06T04:28:27.721685+0000","last_clean":"2024-06-06T04:28:27.721685+0000","last_became_active":"2024-06-06T04:27:19.516271+0000","last_became_peered":"2024-06-06T04:27:19.516271+0000","last_unstale":"2024-06-06T04:28:27.721685+0000","last_undegraded":"2024-06-06T04:28:27.721685+0000","last_fullsized":"2024-06-06T04:28:27.721685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:33:46.817727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725626+0000","last_change":"2024-06-06T04:27:22.572671+0000","last_active":"2024-06-06T04:28:05.725626+0000","last_peered":"2024-06-06T04:28:05.725626+0000","last_clean":"2024-06-06T04:28:05.725626+0000","last_became_active":"2024-06-06T04:27:22.572230+0000","last_became_peered":"2024-06-06T04:27:22.572230+0000","last_unstale":"2024-06-06T04:28:05.725626+0000","last_undegraded":"2024-06-06T04:28:05.725626+0000","last_fullsized":"2024-06-06T04:28:05.725626+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:36:48.393092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725500+0000","last_change":"2024-06-06T04:27:32.620131+0000","last_active":"2024-06-06T04:28:05.725500+0000","last_peered":"2024-06-06T04:28:05.725500+0000","last_clean":"2024-06-06T04:28:05.725500+0000","last_became_active":"2024-06-06T04:27:32.619686+0000","last_became_peered":"2024-06-06T04:27:32.619686+0000","last_unstale":"2024-06-06T04:28:05.725500+0000","last_undegraded":"2024-06-06T04:28:05.725500+0000","last_fullsized":"2024-06-06T04:28:05.725500+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:19:17.130410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:29.715083+0000","last_change":"2024-06-06T04:27:19.304682+0000","last_active":"2024-06-06T04:28:29.715083+0000","last_peered":"2024-06-06T04:28:29.715083+0000","last_clean":"2024-06-06T04:28:29.715083+0000","last_became_active":"2024-06-06T04:27:19.304262+0000","last_became_peered":"2024-06-06T04:27:19.304262+0000","last_unstale":"2024-06-06T04:28:29.715083+0000","last_undegraded":"2024-06-06T04:28:29.715083+0000","last_fullsized":"2024-06-06T04:28:29.715083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:45.819520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.721153+0000","last_change":"2024-06-06T04:27:22.569544+0000","last_active":"2024-06-06T04:28:05.721153+0000","last_peered":"2024-06-06T04:28:05.721153+0000","last_clean":"2024-06-06T04:28:05.721153+0000","last_became_active":"2024-06-06T04:27:22.569072+0000","last_became_peered":"2024-06-06T04:27:22.569072+0000","last_unstale":"2024-06-06T04:28:05.721153+0000","last_undegraded":"2024-06-06T04:28:05.721153+0000","last_fullsized":"2024-06-06T04:28:05.721153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:38:56.460433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720662+0000","last_change":"2024-06-06T04:27:32.622092+0000","last_active":"2024-06-06T04:28:05.720662+0000","last_peered":"2024-06-06T04:28:05.720662+0000","last_clean":"2024-06-06T04:28:05.720662+0000","last_became_active":"2024-06-06T04:27:32.621618+0000","last_became_peered":"2024-06-06T04:27:32.621618+0000","last_unstale":"2024-06-06T04:28:05.720662+0000","last_undegraded":"2024-06-06T04:28:05.720662+0000","last_fullsized":"2024-06-06T04:28:05.720662+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:03.067292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.895082+0000","last_change":"2024-06-06T04:27:19.516803+0000","last_active":"2024-06-06T04:28:05.895082+0000","last_peered":"2024-06-06T04:28:05.895082+0000","last_clean":"2024-06-06T04:28:05.895082+0000","last_became_active":"2024-06-06T04:27:19.516458+0000","last_became_peered":"2024-06-06T04:27:19.516458+0000","last_unstale":"2024-06-06T04:28:05.895082+0000","last_undegraded":"2024-06-06T04:28:05.895082+0000","last_fullsized":"2024-06-06T04:28:05.895082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:34:53.593825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719311+0000","last_change":"2024-06-06T04:27:22.564306+0000","last_active":"2024-06-06T04:28:05.719311+0000","last_peered":"2024-06-06T04:28:05.719311+0000","last_clean":"2024-06-06T04:28:05.719311+0000","last_became_active":"2024-06-06T04:27:22.563922+0000","last_became_peered":"2024-06-06T04:27:22.563922+0000","last_unstale":"2024-06-06T04:28:05.719311+0000","last_undegraded":"2024-06-06T04:28:05.719311+0000","last_fullsized":"2024-06-06T04:28:05.719311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:42:23.189047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719883+0000","last_change":"2024-06-06T04:27:32.617014+0000","last_active":"2024-06-06T04:28:05.719883+0000","last_peered":"2024-06-06T04:28:05.719883+0000","last_clean":"2024-06-06T04:28:05.719883+0000","last_became_active":"2024-06-06T04:27:32.616639+0000","last_became_peered":"2024-06-06T04:27:32.616639+0000","last_unstale":"2024-06-06T04:28:05.719883+0000","last_undegraded":"2024-06-06T04:28:05.719883+0000","last_fullsized":"2024-06-06T04:28:05.719883+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:54:52.034711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725326+0000","last_change":"2024-06-06T04:27:19.305000+0000","last_active":"2024-06-06T04:28:05.725326+0000","last_peered":"2024-06-06T04:28:05.725326+0000","last_clean":"2024-06-06T04:28:05.725326+0000","last_became_active":"2024-06-06T04:27:19.304676+0000","last_became_peered":"2024-06-06T04:27:19.304676+0000","last_unstale":"2024-06-06T04:28:05.725326+0000","last_undegraded":"2024-06-06T04:28:05.725326+0000","last_fullsized":"2024-06-06T04:28:05.725326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:19:34.054455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719463+0000","last_change":"2024-06-06T04:27:22.572680+0000","last_active":"2024-06-06T04:28:05.719463+0000","last_peered":"2024-06-06T04:28:05.719463+0000","last_clean":"2024-06-06T04:28:05.719463+0000","last_became_active":"2024-06-06T04:27:22.572348+0000","last_became_peered":"2024-06-06T04:27:22.572348+0000","last_unstale":"2024-06-06T04:28:05.719463+0000","last_undegraded":"2024-06-06T04:28:05.719463+0000","last_fullsized":"2024-06-06T04:28:05.719463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:05:26.215771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719538+0000","last_change":"2024-06-06T04:27:32.611587+0000","last_active":"2024-06-06T04:28:05.719538+0000","last_peered":"2024-06-06T04:28:05.719538+0000","last_clean":"2024-06-06T04:28:05.719538+0000","last_became_active":"2024-06-06T04:27:32.611183+0000","last_became_peered":"2024-06-06T04:27:32.611183+0000","last_unstale":"2024-06-06T04:28:05.719538+0000","last_undegraded":"2024-06-06T04:28:05.719538+0000","last_fullsized":"2024-06-06T04:28:05.719538+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:36.773254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724730+0000","last_change":"2024-06-06T04:27:22.562590+0000","last_active":"2024-06-06T04:28:05.724730+0000","last_peered":"2024-06-06T04:28:05.724730+0000","last_clean":"2024-06-06T04:28:05.724730+0000","last_became_active":"2024-06-06T04:27:22.560542+0000","last_became_peered":"2024-06-06T04:27:22.560542+0000","last_unstale":"2024-06-06T04:28:05.724730+0000","last_undegraded":"2024-06-06T04:28:05.724730+0000","last_fullsized":"2024-06-06T04:28:05.724730+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:19:20.651098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":68,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:51.764995+0000","last_change":"2024-06-06T04:27:19.516361+0000","last_active":"2024-06-06T04:28:51.764995+0000","last_peered":"2024-06-06T04:28:51.764995+0000","last_clean":"2024-06-06T04:28:51.764995+0000","last_became_active":"2024-06-06T04:27:19.515973+0000","last_became_peered":"2024-06-06T04:27:19.515973+0000","last_unstale":"2024-06-06T04:28:51.764995+0000","last_undegraded":"2024-06-06T04:28:51.764995+0000","last_fullsized":"2024-06-06T04:28:51.764995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:37.298819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725846+0000","last_change":"2024-06-06T04:27:32.630528+0000","last_active":"2024-06-06T04:28:05.725846+0000","last_peered":"2024-06-06T04:28:05.725846+0000","last_clean":"2024-06-06T04:28:05.725846+0000","last_became_active":"2024-06-06T04:27:32.630215+0000","last_became_peered":"2024-06-06T04:27:32.630215+0000","last_unstale":"2024-06-06T04:28:05.725846+0000","last_undegraded":"2024-06-06T04:28:05.725846+0000","last_fullsized":"2024-06-06T04:28:05.725846+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:34:29.980193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717038+0000","last_change":"2024-06-06T04:27:22.567466+0000","last_active":"2024-06-06T04:28:05.717038+0000","last_peered":"2024-06-06T04:28:05.717038+0000","last_clean":"2024-06-06T04:28:05.717038+0000","last_became_active":"2024-06-06T04:27:22.567145+0000","last_became_peered":"2024-06-06T04:27:22.567145+0000","last_unstale":"2024-06-06T04:28:05.717038+0000","last_undegraded":"2024-06-06T04:28:05.717038+0000","last_fullsized":"2024-06-06T04:28:05.717038+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:56:56.203027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:29.730556+0000","last_change":"2024-06-06T04:27:19.302726+0000","last_active":"2024-06-06T04:28:29.730556+0000","last_peered":"2024-06-06T04:28:29.730556+0000","last_clean":"2024-06-06T04:28:29.730556+0000","last_became_active":"2024-06-06T04:27:19.302329+0000","last_became_peered":"2024-06-06T04:27:19.302329+0000","last_unstale":"2024-06-06T04:28:29.730556+0000","last_undegraded":"2024-06-06T04:28:29.730556+0000","last_fullsized":"2024-06-06T04:28:29.730556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:55:15.010632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":362,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724028+0000","last_change":"2024-06-06T04:25:53.854188+0000","last_active":"2024-06-06T04:28:05.724028+0000","last_peered":"2024-06-06T04:28:05.724028+0000","last_clean":"2024-06-06T04:28:05.724028+0000","last_became_active":"2024-06-06T04:25:53.853700+0000","last_became_peered":"2024-06-06T04:25:53.853700+0000","last_unstale":"2024-06-06T04:28:05.724028+0000","last_undegraded":"2024-06-06T04:28:05.724028+0000","last_fullsized":"2024-06-06T04:28:05.724028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:22:28.847333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:22:28.847333+0000","last_clean_scrub_stamp":"2024-06-06T04:22:28.847333+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:59:10.246373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.727431+0000","last_change":"2024-06-06T04:27:32.623300+0000","last_active":"2024-06-06T04:28:05.727431+0000","last_peered":"2024-06-06T04:28:05.727431+0000","last_clean":"2024-06-06T04:28:05.727431+0000","last_became_active":"2024-06-06T04:27:32.622811+0000","last_became_peered":"2024-06-06T04:27:32.622811+0000","last_unstale":"2024-06-06T04:28:05.727431+0000","last_undegraded":"2024-06-06T04:28:05.727431+0000","last_fullsized":"2024-06-06T04:28:05.727431+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:28.383079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719028+0000","last_change":"2024-06-06T04:27:22.566617+0000","last_active":"2024-06-06T04:28:05.719028+0000","last_peered":"2024-06-06T04:28:05.719028+0000","last_clean":"2024-06-06T04:28:05.719028+0000","last_became_active":"2024-06-06T04:27:22.566256+0000","last_became_peered":"2024-06-06T04:27:22.566256+0000","last_unstale":"2024-06-06T04:28:05.719028+0000","last_undegraded":"2024-06-06T04:28:05.719028+0000","last_fullsized":"2024-06-06T04:28:05.719028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:50:41.318475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:27.720308+0000","last_change":"2024-06-06T04:27:19.305941+0000","last_active":"2024-06-06T04:28:27.720308+0000","last_peered":"2024-06-06T04:28:27.720308+0000","last_clean":"2024-06-06T04:28:27.720308+0000","last_became_active":"2024-06-06T04:27:19.305037+0000","last_became_peered":"2024-06-06T04:27:19.305037+0000","last_unstale":"2024-06-06T04:28:27.720308+0000","last_undegraded":"2024-06-06T04:28:27.720308+0000","last_fullsized":"2024-06-06T04:28:27.720308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:54.084427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720749+0000","last_change":"2024-06-06T04:27:32.621182+0000","last_active":"2024-06-06T04:28:05.720749+0000","last_peered":"2024-06-06T04:28:05.720749+0000","last_clean":"2024-06-06T04:28:05.720749+0000","last_became_active":"2024-06-06T04:27:32.620710+0000","last_became_peered":"2024-06-06T04:27:32.620710+0000","last_unstale":"2024-06-06T04:28:05.720749+0000","last_undegraded":"2024-06-06T04:28:05.720749+0000","last_fullsized":"2024-06-06T04:28:05.720749+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:05.512309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:26.713063+0000","last_change":"2024-06-06T04:27:19.517033+0000","last_active":"2024-06-06T04:28:26.713063+0000","last_peered":"2024-06-06T04:28:26.713063+0000","last_clean":"2024-06-06T04:28:26.713063+0000","last_became_active":"2024-06-06T04:27:19.516278+0000","last_became_peered":"2024-06-06T04:27:19.516278+0000","last_unstale":"2024-06-06T04:28:26.713063+0000","last_undegraded":"2024-06-06T04:28:26.713063+0000","last_fullsized":"2024-06-06T04:28:26.713063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:11:20.336818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719542+0000","last_change":"2024-06-06T04:27:22.566261+0000","last_active":"2024-06-06T04:28:05.719542+0000","last_peered":"2024-06-06T04:28:05.719542+0000","last_clean":"2024-06-06T04:28:05.719542+0000","last_became_active":"2024-06-06T04:27:22.565875+0000","last_became_peered":"2024-06-06T04:27:22.565875+0000","last_unstale":"2024-06-06T04:28:05.719542+0000","last_undegraded":"2024-06-06T04:28:05.719542+0000","last_fullsized":"2024-06-06T04:28:05.719542+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:49:55.923950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.716781+0000","last_change":"2024-06-06T04:27:32.619854+0000","last_active":"2024-06-06T04:28:05.716781+0000","last_peered":"2024-06-06T04:28:05.716781+0000","last_clean":"2024-06-06T04:28:05.716781+0000","last_became_active":"2024-06-06T04:27:32.618765+0000","last_became_peered":"2024-06-06T04:27:32.618765+0000","last_unstale":"2024-06-06T04:28:05.716781+0000","last_undegraded":"2024-06-06T04:28:05.716781+0000","last_fullsized":"2024-06-06T04:28:05.716781+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:47:05.283751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'10","reported_seq":50,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:46.581679+0000","last_change":"2024-06-06T04:27:19.303057+0000","last_active":"2024-06-06T04:28:46.581679+0000","last_peered":"2024-06-06T04:28:46.581679+0000","last_clean":"2024-06-06T04:28:46.581679+0000","last_became_active":"2024-06-06T04:27:19.302716+0000","last_became_peered":"2024-06-06T04:27:19.302716+0000","last_unstale":"2024-06-06T04:28:46.581679+0000","last_undegraded":"2024-06-06T04:28:46.581679+0000","last_fullsized":"2024-06-06T04:28:46.581679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:35.273663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717447+0000","last_change":"2024-06-06T04:27:22.575274+0000","last_active":"2024-06-06T04:28:05.717447+0000","last_peered":"2024-06-06T04:28:05.717447+0000","last_clean":"2024-06-06T04:28:05.717447+0000","last_became_active":"2024-06-06T04:27:22.574979+0000","last_became_peered":"2024-06-06T04:27:22.574979+0000","last_unstale":"2024-06-06T04:28:05.717447+0000","last_undegraded":"2024-06-06T04:28:05.717447+0000","last_fullsized":"2024-06-06T04:28:05.717447+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:30.718398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717553+0000","last_change":"2024-06-06T04:27:32.619939+0000","last_active":"2024-06-06T04:28:05.717553+0000","last_peered":"2024-06-06T04:28:05.717553+0000","last_clean":"2024-06-06T04:28:05.717553+0000","last_became_active":"2024-06-06T04:27:32.619499+0000","last_became_peered":"2024-06-06T04:27:32.619499+0000","last_unstale":"2024-06-06T04:28:05.717553+0000","last_undegraded":"2024-06-06T04:28:05.717553+0000","last_fullsized":"2024-06-06T04:28:05.717553+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:41:13.933546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:26.712286+0000","last_change":"2024-06-06T04:27:19.397552+0000","last_active":"2024-06-06T04:28:26.712286+0000","last_peered":"2024-06-06T04:28:26.712286+0000","last_clean":"2024-06-06T04:28:26.712286+0000","last_became_active":"2024-06-06T04:27:19.397186+0000","last_became_peered":"2024-06-06T04:27:19.397186+0000","last_unstale":"2024-06-06T04:28:26.712286+0000","last_undegraded":"2024-06-06T04:28:26.712286+0000","last_fullsized":"2024-06-06T04:28:26.712286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:24:13.403860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.728212+0000","last_change":"2024-06-06T04:28:05.728212+0000","last_active":"2024-06-06T04:28:05.728212+0000","last_peered":"2024-06-06T04:28:05.728212+0000","last_clean":"2024-06-06T04:28:05.728212+0000","last_became_active":"2024-06-06T04:28:05.727748+0000","last_became_peered":"2024-06-06T04:28:05.727748+0000","last_unstale":"2024-06-06T04:28:05.728212+0000","last_undegraded":"2024-06-06T04:28:05.728212+0000","last_fullsized":"2024-06-06T04:28:05.728212+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:50.583716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.896480+0000","last_change":"2024-06-06T04:28:05.896480+0000","last_active":"2024-06-06T04:28:05.896480+0000","last_peered":"2024-06-06T04:28:05.896480+0000","last_clean":"2024-06-06T04:28:05.896480+0000","last_became_active":"2024-06-06T04:28:05.895967+0000","last_became_peered":"2024-06-06T04:28:05.895967+0000","last_unstale":"2024-06-06T04:28:05.896480+0000","last_undegraded":"2024-06-06T04:28:05.896480+0000","last_fullsized":"2024-06-06T04:28:05.896480+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:26:19.846884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:26.713951+0000","last_change":"2024-06-06T04:27:19.516515+0000","last_active":"2024-06-06T04:28:26.713951+0000","last_peered":"2024-06-06T04:28:26.713951+0000","last_clean":"2024-06-06T04:28:26.713951+0000","last_became_active":"2024-06-06T04:27:19.516087+0000","last_became_peered":"2024-06-06T04:27:19.516087+0000","last_unstale":"2024-06-06T04:28:26.713951+0000","last_undegraded":"2024-06-06T04:28:26.713951+0000","last_fullsized":"2024-06-06T04:28:26.713951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:58.440226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719824+0000","last_change":"2024-06-06T04:27:22.562367+0000","last_active":"2024-06-06T04:28:05.719824+0000","last_peered":"2024-06-06T04:28:05.719824+0000","last_clean":"2024-06-06T04:28:05.719824+0000","last_became_active":"2024-06-06T04:27:22.561947+0000","last_became_peered":"2024-06-06T04:27:22.561947+0000","last_unstale":"2024-06-06T04:28:05.719824+0000","last_undegraded":"2024-06-06T04:28:05.719824+0000","last_fullsized":"2024-06-06T04:28:05.719824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:28:06.200748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.895428+0000","last_change":"2024-06-06T04:27:32.631577+0000","last_active":"2024-06-06T04:28:05.895428+0000","last_peered":"2024-06-06T04:28:05.895428+0000","last_clean":"2024-06-06T04:28:05.895428+0000","last_became_active":"2024-06-06T04:27:32.629555+0000","last_became_peered":"2024-06-06T04:27:32.629555+0000","last_unstale":"2024-06-06T04:28:05.895428+0000","last_undegraded":"2024-06-06T04:28:05.895428+0000","last_fullsized":"2024-06-06T04:28:05.895428+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:52:56.197281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:26.713066+0000","last_change":"2024-06-06T04:27:19.298880+0000","last_active":"2024-06-06T04:28:26.713066+0000","last_peered":"2024-06-06T04:28:26.713066+0000","last_clean":"2024-06-06T04:28:26.713066+0000","last_became_active":"2024-06-06T04:27:19.298551+0000","last_became_peered":"2024-06-06T04:27:19.298551+0000","last_unstale":"2024-06-06T04:28:26.713066+0000","last_undegraded":"2024-06-06T04:28:26.713066+0000","last_fullsized":"2024-06-06T04:28:26.713066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:11:32.145401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720444+0000","last_change":"2024-06-06T04:27:22.575065+0000","last_active":"2024-06-06T04:28:05.720444+0000","last_peered":"2024-06-06T04:28:05.720444+0000","last_clean":"2024-06-06T04:28:05.720444+0000","last_became_active":"2024-06-06T04:27:22.574469+0000","last_became_peered":"2024-06-06T04:27:22.574469+0000","last_unstale":"2024-06-06T04:28:05.720444+0000","last_undegraded":"2024-06-06T04:28:05.720444+0000","last_fullsized":"2024-06-06T04:28:05.720444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:29:44.943833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717731+0000","last_change":"2024-06-06T04:27:32.626678+0000","last_active":"2024-06-06T04:28:05.717731+0000","last_peered":"2024-06-06T04:28:05.717731+0000","last_clean":"2024-06-06T04:28:05.717731+0000","last_became_active":"2024-06-06T04:27:32.626167+0000","last_became_peered":"2024-06-06T04:27:32.626167+0000","last_unstale":"2024-06-06T04:28:05.717731+0000","last_undegraded":"2024-06-06T04:28:05.717731+0000","last_fullsized":"2024-06-06T04:28:05.717731+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:56:26.363195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717922+0000","last_change":"2024-06-06T04:27:19.306466+0000","last_active":"2024-06-06T04:28:05.717922+0000","last_peered":"2024-06-06T04:28:05.717922+0000","last_clean":"2024-06-06T04:28:05.717922+0000","last_became_active":"2024-06-06T04:27:19.303821+0000","last_became_peered":"2024-06-06T04:27:19.303821+0000","last_unstale":"2024-06-06T04:28:05.717922+0000","last_undegraded":"2024-06-06T04:28:05.717922+0000","last_fullsized":"2024-06-06T04:28:05.717922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:23:53.593998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717221+0000","last_change":"2024-06-06T04:27:22.562662+0000","last_active":"2024-06-06T04:28:05.717221+0000","last_peered":"2024-06-06T04:28:05.717221+0000","last_clean":"2024-06-06T04:28:05.717221+0000","last_became_active":"2024-06-06T04:27:22.562318+0000","last_became_peered":"2024-06-06T04:27:22.562318+0000","last_unstale":"2024-06-06T04:28:05.717221+0000","last_undegraded":"2024-06-06T04:28:05.717221+0000","last_fullsized":"2024-06-06T04:28:05.717221+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:31:26.639275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717596+0000","last_change":"2024-06-06T04:27:32.619970+0000","last_active":"2024-06-06T04:28:05.717596+0000","last_peered":"2024-06-06T04:28:05.717596+0000","last_clean":"2024-06-06T04:28:05.717596+0000","last_became_active":"2024-06-06T04:27:32.619464+0000","last_became_peered":"2024-06-06T04:27:32.619464+0000","last_unstale":"2024-06-06T04:28:05.717596+0000","last_undegraded":"2024-06-06T04:28:05.717596+0000","last_fullsized":"2024-06-06T04:28:05.717596+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:44.844018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:26.712874+0000","last_change":"2024-06-06T04:27:19.308172+0000","last_active":"2024-06-06T04:28:26.712874+0000","last_peered":"2024-06-06T04:28:26.712874+0000","last_clean":"2024-06-06T04:28:26.712874+0000","last_became_active":"2024-06-06T04:27:19.307780+0000","last_became_peered":"2024-06-06T04:27:19.307780+0000","last_unstale":"2024-06-06T04:28:26.712874+0000","last_undegraded":"2024-06-06T04:28:26.712874+0000","last_fullsized":"2024-06-06T04:28:26.712874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:39:11.653649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725426+0000","last_change":"2024-06-06T04:27:22.568881+0000","last_active":"2024-06-06T04:28:05.725426+0000","last_peered":"2024-06-06T04:28:05.725426+0000","last_clean":"2024-06-06T04:28:05.725426+0000","last_became_active":"2024-06-06T04:27:22.568489+0000","last_became_peered":"2024-06-06T04:27:22.568489+0000","last_unstale":"2024-06-06T04:28:05.725426+0000","last_undegraded":"2024-06-06T04:28:05.725426+0000","last_fullsized":"2024-06-06T04:28:05.725426+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:00:22.339313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720343+0000","last_change":"2024-06-06T04:27:32.622357+0000","last_active":"2024-06-06T04:28:05.720343+0000","last_peered":"2024-06-06T04:28:05.720343+0000","last_clean":"2024-06-06T04:28:05.720343+0000","last_became_active":"2024-06-06T04:27:32.621680+0000","last_became_peered":"2024-06-06T04:27:32.621680+0000","last_unstale":"2024-06-06T04:28:05.720343+0000","last_undegraded":"2024-06-06T04:28:05.720343+0000","last_fullsized":"2024-06-06T04:28:05.720343+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:00.677721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:27.719936+0000","last_change":"2024-06-06T04:27:19.516890+0000","last_active":"2024-06-06T04:28:27.719936+0000","last_peered":"2024-06-06T04:28:27.719936+0000","last_clean":"2024-06-06T04:28:27.719936+0000","last_became_active":"2024-06-06T04:27:19.516490+0000","last_became_peered":"2024-06-06T04:27:19.516490+0000","last_unstale":"2024-06-06T04:28:27.719936+0000","last_undegraded":"2024-06-06T04:28:27.719936+0000","last_fullsized":"2024-06-06T04:28:27.719936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:11:59.336250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725945+0000","last_change":"2024-06-06T04:27:22.571003+0000","last_active":"2024-06-06T04:28:05.725945+0000","last_peered":"2024-06-06T04:28:05.725945+0000","last_clean":"2024-06-06T04:28:05.725945+0000","last_became_active":"2024-06-06T04:27:22.570442+0000","last_became_peered":"2024-06-06T04:27:22.570442+0000","last_unstale":"2024-06-06T04:28:05.725945+0000","last_undegraded":"2024-06-06T04:28:05.725945+0000","last_fullsized":"2024-06-06T04:28:05.725945+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:28:26.870429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.721107+0000","last_change":"2024-06-06T04:27:32.624424+0000","last_active":"2024-06-06T04:28:05.721107+0000","last_peered":"2024-06-06T04:28:05.721107+0000","last_clean":"2024-06-06T04:28:05.721107+0000","last_became_active":"2024-06-06T04:27:32.624026+0000","last_became_peered":"2024-06-06T04:27:32.624026+0000","last_unstale":"2024-06-06T04:28:05.721107+0000","last_undegraded":"2024-06-06T04:28:05.721107+0000","last_fullsized":"2024-06-06T04:28:05.721107+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:30:08.814284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.726869+0000","last_change":"2024-06-06T04:27:19.301382+0000","last_active":"2024-06-06T04:28:05.726869+0000","last_peered":"2024-06-06T04:28:05.726869+0000","last_clean":"2024-06-06T04:28:05.726869+0000","last_became_active":"2024-06-06T04:27:19.300882+0000","last_became_peered":"2024-06-06T04:27:19.300882+0000","last_unstale":"2024-06-06T04:28:05.726869+0000","last_undegraded":"2024-06-06T04:28:05.726869+0000","last_fullsized":"2024-06-06T04:28:05.726869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:50.514103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725239+0000","last_change":"2024-06-06T04:27:22.570661+0000","last_active":"2024-06-06T04:28:05.725239+0000","last_peered":"2024-06-06T04:28:05.725239+0000","last_clean":"2024-06-06T04:28:05.725239+0000","last_became_active":"2024-06-06T04:27:22.570287+0000","last_became_peered":"2024-06-06T04:27:22.570287+0000","last_unstale":"2024-06-06T04:28:05.725239+0000","last_undegraded":"2024-06-06T04:28:05.725239+0000","last_fullsized":"2024-06-06T04:28:05.725239+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:48.790492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724347+0000","last_change":"2024-06-06T04:27:32.618385+0000","last_active":"2024-06-06T04:28:05.724347+0000","last_peered":"2024-06-06T04:28:05.724347+0000","last_clean":"2024-06-06T04:28:05.724347+0000","last_became_active":"2024-06-06T04:27:32.617956+0000","last_became_peered":"2024-06-06T04:27:32.617956+0000","last_unstale":"2024-06-06T04:28:05.724347+0000","last_undegraded":"2024-06-06T04:28:05.724347+0000","last_fullsized":"2024-06-06T04:28:05.724347+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:26.308657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":52,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.728633+0000","last_change":"2024-06-06T04:28:05.728633+0000","last_active":"2024-06-06T04:28:05.728633+0000","last_peered":"2024-06-06T04:28:05.728633+0000","last_clean":"2024-06-06T04:28:05.728633+0000","last_became_active":"2024-06-06T04:28:05.722427+0000","last_became_peered":"2024-06-06T04:28:05.722427+0000","last_unstale":"2024-06-06T04:28:05.728633+0000","last_undegraded":"2024-06-06T04:28:05.728633+0000","last_fullsized":"2024-06-06T04:28:05.728633+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:13:22.061798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.726020+0000","last_change":"2024-06-06T04:27:22.574950+0000","last_active":"2024-06-06T04:28:05.726020+0000","last_peered":"2024-06-06T04:28:05.726020+0000","last_clean":"2024-06-06T04:28:05.726020+0000","last_became_active":"2024-06-06T04:27:22.574604+0000","last_became_peered":"2024-06-06T04:27:22.574604+0000","last_unstale":"2024-06-06T04:28:05.726020+0000","last_undegraded":"2024-06-06T04:28:05.726020+0000","last_fullsized":"2024-06-06T04:28:05.726020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:17:29.395137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720585+0000","last_change":"2024-06-06T04:27:22.568554+0000","last_active":"2024-06-06T04:28:05.720585+0000","last_peered":"2024-06-06T04:28:05.720585+0000","last_clean":"2024-06-06T04:28:05.720585+0000","last_became_active":"2024-06-06T04:27:22.568196+0000","last_became_peered":"2024-06-06T04:27:22.568196+0000","last_unstale":"2024-06-06T04:28:05.720585+0000","last_undegraded":"2024-06-06T04:28:05.720585+0000","last_fullsized":"2024-06-06T04:28:05.720585+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:11.564067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.718007+0000","last_change":"2024-06-06T04:27:22.565472+0000","last_active":"2024-06-06T04:28:05.718007+0000","last_peered":"2024-06-06T04:28:05.718007+0000","last_clean":"2024-06-06T04:28:05.718007+0000","last_became_active":"2024-06-06T04:27:22.565116+0000","last_became_peered":"2024-06-06T04:27:22.565116+0000","last_unstale":"2024-06-06T04:28:05.718007+0000","last_undegraded":"2024-06-06T04:28:05.718007+0000","last_fullsized":"2024-06-06T04:28:05.718007+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:03:58.232707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717634+0000","last_change":"2024-06-06T04:27:22.560210+0000","last_active":"2024-06-06T04:28:05.717634+0000","last_peered":"2024-06-06T04:28:05.717634+0000","last_clean":"2024-06-06T04:28:05.717634+0000","last_became_active":"2024-06-06T04:27:22.559853+0000","last_became_peered":"2024-06-06T04:27:22.559853+0000","last_unstale":"2024-06-06T04:28:05.717634+0000","last_undegraded":"2024-06-06T04:28:05.717634+0000","last_fullsized":"2024-06-06T04:28:05.717634+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:56:18.650427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.721312+0000","last_change":"2024-06-06T04:27:22.565071+0000","last_active":"2024-06-06T04:28:05.721312+0000","last_peered":"2024-06-06T04:28:05.721312+0000","last_clean":"2024-06-06T04:28:05.721312+0000","last_became_active":"2024-06-06T04:27:22.561411+0000","last_became_peered":"2024-06-06T04:27:22.561411+0000","last_unstale":"2024-06-06T04:28:05.721312+0000","last_undegraded":"2024-06-06T04:28:05.721312+0000","last_fullsized":"2024-06-06T04:28:05.721312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:41.948481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719436+0000","last_change":"2024-06-06T04:27:22.563280+0000","last_active":"2024-06-06T04:28:05.719436+0000","last_peered":"2024-06-06T04:28:05.719436+0000","last_clean":"2024-06-06T04:28:05.719436+0000","last_became_active":"2024-06-06T04:27:22.563049+0000","last_became_peered":"2024-06-06T04:27:22.563049+0000","last_unstale":"2024-06-06T04:28:05.719436+0000","last_undegraded":"2024-06-06T04:28:05.719436+0000","last_fullsized":"2024-06-06T04:28:05.719436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:41:48.287969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.721794+0000","last_change":"2024-06-06T04:27:22.567691+0000","last_active":"2024-06-06T04:28:05.721794+0000","last_peered":"2024-06-06T04:28:05.721794+0000","last_clean":"2024-06-06T04:28:05.721794+0000","last_became_active":"2024-06-06T04:27:22.567331+0000","last_became_peered":"2024-06-06T04:27:22.567331+0000","last_unstale":"2024-06-06T04:28:05.721794+0000","last_undegraded":"2024-06-06T04:28:05.721794+0000","last_fullsized":"2024-06-06T04:28:05.721794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:03.886084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.718283+0000","last_change":"2024-06-06T04:27:22.563853+0000","last_active":"2024-06-06T04:28:05.718283+0000","last_peered":"2024-06-06T04:28:05.718283+0000","last_clean":"2024-06-06T04:28:05.718283+0000","last_became_active":"2024-06-06T04:27:22.563436+0000","last_became_peered":"2024-06-06T04:27:22.563436+0000","last_unstale":"2024-06-06T04:28:05.718283+0000","last_undegraded":"2024-06-06T04:28:05.718283+0000","last_fullsized":"2024-06-06T04:28:05.718283+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:26.201308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717559+0000","last_change":"2024-06-06T04:27:22.568189+0000","last_active":"2024-06-06T04:28:05.717559+0000","last_peered":"2024-06-06T04:28:05.717559+0000","last_clean":"2024-06-06T04:28:05.717559+0000","last_became_active":"2024-06-06T04:27:22.567815+0000","last_became_peered":"2024-06-06T04:27:22.567815+0000","last_unstale":"2024-06-06T04:28:05.717559+0000","last_undegraded":"2024-06-06T04:28:05.717559+0000","last_fullsized":"2024-06-06T04:28:05.717559+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:13:39.259012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724663+0000","last_change":"2024-06-06T04:27:22.569698+0000","last_active":"2024-06-06T04:28:05.724663+0000","last_peered":"2024-06-06T04:28:05.724663+0000","last_clean":"2024-06-06T04:28:05.724663+0000","last_became_active":"2024-06-06T04:27:22.569347+0000","last_became_peered":"2024-06-06T04:27:22.569347+0000","last_unstale":"2024-06-06T04:28:05.724663+0000","last_undegraded":"2024-06-06T04:28:05.724663+0000","last_fullsized":"2024-06-06T04:28:05.724663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:06:34.722798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.716981+0000","last_change":"2024-06-06T04:27:22.561149+0000","last_active":"2024-06-06T04:28:05.716981+0000","last_peered":"2024-06-06T04:28:05.716981+0000","last_clean":"2024-06-06T04:28:05.716981+0000","last_became_active":"2024-06-06T04:27:22.560793+0000","last_became_peered":"2024-06-06T04:27:22.560793+0000","last_unstale":"2024-06-06T04:28:05.716981+0000","last_undegraded":"2024-06-06T04:28:05.716981+0000","last_fullsized":"2024-06-06T04:28:05.716981+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:00:41.563443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720263+0000","last_change":"2024-06-06T04:27:22.569331+0000","last_active":"2024-06-06T04:28:05.720263+0000","last_peered":"2024-06-06T04:28:05.720263+0000","last_clean":"2024-06-06T04:28:05.720263+0000","last_became_active":"2024-06-06T04:27:22.569021+0000","last_became_peered":"2024-06-06T04:27:22.569021+0000","last_unstale":"2024-06-06T04:28:05.720263+0000","last_undegraded":"2024-06-06T04:28:05.720263+0000","last_fullsized":"2024-06-06T04:28:05.720263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:52:37.064317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719495+0000","last_change":"2024-06-06T04:27:22.574558+0000","last_active":"2024-06-06T04:28:05.719495+0000","last_peered":"2024-06-06T04:28:05.719495+0000","last_clean":"2024-06-06T04:28:05.719495+0000","last_became_active":"2024-06-06T04:27:22.574215+0000","last_became_peered":"2024-06-06T04:27:22.574215+0000","last_unstale":"2024-06-06T04:28:05.719495+0000","last_undegraded":"2024-06-06T04:28:05.719495+0000","last_fullsized":"2024-06-06T04:28:05.719495+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:40:52.496322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.722077+0000","last_change":"2024-06-06T04:27:22.565320+0000","last_active":"2024-06-06T04:28:05.722077+0000","last_peered":"2024-06-06T04:28:05.722077+0000","last_clean":"2024-06-06T04:28:05.722077+0000","last_became_active":"2024-06-06T04:27:22.564786+0000","last_became_peered":"2024-06-06T04:27:22.564786+0000","last_unstale":"2024-06-06T04:28:05.722077+0000","last_undegraded":"2024-06-06T04:28:05.722077+0000","last_fullsized":"2024-06-06T04:28:05.722077+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:19:42.057666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249851,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60158,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":161,"num_read_kb":146,"num_write":162,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":180474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351010,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":553987,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612647,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28204,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716948,"statfs":{"total":95995035648,"available":95966154752,"internally_reserved":0,"allocated":1216512,"data_stored":1197803,"data_compressed":62660,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":558319,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037809,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":554143,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299446,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":1150051,"data_compressed":62660,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462972,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1073152,"data_stored":549399,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":605467,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":550787,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":606817,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":549593,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888133,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":554439,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":1145249,"data_compressed":62660,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34523,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34523,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T04:28:56.733 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T04:28:56.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:56 smithi045 ceph-mon[32299]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:28:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:28:57.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:56 smithi086 ceph-mon[36053]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:28:57.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:56 smithi154 ceph-mon[36428]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:28:58.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:57 smithi086 ceph-mon[36053]: pgmap v354: 129 pgs: 129 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-06-06T04:28:58.142 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:57 smithi045 ceph-mon[32299]: pgmap v354: 129 pgs: 129 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-06-06T04:28:58.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:57 smithi154 ceph-mon[36428]: pgmap v354: 129 pgs: 129 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-06-06T04:28:58.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.678+0000 7fa633ca0700 1 -- 172.21.15.45:0/2268996072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa62c1030f0 msgr2=0x7fa62c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:58.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.678+0000 7fa633ca0700 1 --2- 172.21.15.45:0/2268996072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa62c1030f0 0x7fa62c107110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0099c0 tx=0x7fa61c009cd0 comp rx=0 tx=0).stop 2024-06-06T04:28:58.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.678+0000 7fa633ca0700 1 -- 172.21.15.45:0/2268996072 shutdown_connections 2024-06-06T04:28:58.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.678+0000 7fa633ca0700 1 --2- 172.21.15.45:0/2268996072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa62c107840 0x7fa62c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:58.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.678+0000 7fa633ca0700 1 --2- 172.21.15.45:0/2268996072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa62c102740 0x7fa62c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:58.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.678+0000 7fa633ca0700 1 --2- 172.21.15.45:0/2268996072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa62c1030f0 0x7fa62c107110 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:58.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.678+0000 7fa633ca0700 1 -- 172.21.15.45:0/2268996072 >> 172.21.15.45:0/2268996072 conn(0x7fa62c0795f0 msgr2=0x7fa62c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:58.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.678+0000 7fa633ca0700 1 -- 172.21.15.45:0/2268996072 shutdown_connections 2024-06-06T04:28:58.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.678+0000 7fa633ca0700 1 -- 172.21.15.45:0/2268996072 wait complete. 2024-06-06T04:28:58.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa633ca0700 1 Processor -- start 2024-06-06T04:28:58.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa633ca0700 1 -- start start 2024-06-06T04:28:58.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa633ca0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa62c102740 0x7fa62c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:58.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa633ca0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa62c1030f0 0x7fa62c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:58.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa633ca0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa62c107840 0x7fa62c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:58.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa633ca0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c112600 con 0x7fa62c1030f0 2024-06-06T04:28:58.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa633ca0700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c112480 con 0x7fa62c102740 2024-06-06T04:28:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa633ca0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c112780 con 0x7fa62c107840 2024-06-06T04:28:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa63123b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa62c1030f0 0x7fa62c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa63123b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa62c1030f0 0x7fa62c074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44280/0 (socket says 172.21.15.45:44280) 2024-06-06T04:28:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa63123b700 1 -- 172.21.15.45:0/3192817676 learned_addr learned my addr 172.21.15.45:0/3192817676 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:28:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa63223d700 1 --2- 172.21.15.45:0/3192817676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa62c107840 0x7fa62c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa631a3c700 1 --2- 172.21.15.45:0/3192817676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa62c102740 0x7fa62c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa63123b700 1 -- 172.21.15.45:0/3192817676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa62c107840 msgr2=0x7fa62c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa63123b700 1 --2- 172.21.15.45:0/3192817676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa62c107840 0x7fa62c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa63123b700 1 -- 172.21.15.45:0/3192817676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa62c102740 msgr2=0x7fa62c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa63123b700 1 --2- 172.21.15.45:0/3192817676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa62c102740 0x7fa62c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa63123b700 1 -- 172.21.15.45:0/3192817676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c1a7020 con 0x7fa62c1030f0 2024-06-06T04:28:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa63223d700 1 --2- 172.21.15.45:0/3192817676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa62c107840 0x7fa62c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:28:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.679+0000 7fa631a3c700 1 --2- 172.21.15.45:0/3192817676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa62c102740 0x7fa62c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:28:58.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.680+0000 7fa63123b700 1 --2- 172.21.15.45:0/3192817676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa62c1030f0 0x7fa62c074cd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa61c006010 tx=0x7fa61c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:58.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.680+0000 7fa622ffd700 1 -- 172.21.15.45:0/3192817676 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c027070 con 0x7fa62c1030f0 2024-06-06T04:28:58.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.680+0000 7fa633ca0700 1 -- 172.21.15.45:0/3192817676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62c1a72b0 con 0x7fa62c1030f0 2024-06-06T04:28:58.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.680+0000 7fa622ffd700 1 -- 172.21.15.45:0/3192817676 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa61c003fd0 con 0x7fa62c1030f0 2024-06-06T04:28:58.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.680+0000 7fa622ffd700 1 -- 172.21.15.45:0/3192817676 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c022d70 con 0x7fa62c1030f0 2024-06-06T04:28:58.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.680+0000 7fa633ca0700 1 -- 172.21.15.45:0/3192817676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62c1a7760 con 0x7fa62c1030f0 2024-06-06T04:28:58.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.681+0000 7fa633ca0700 1 -- 172.21.15.45:0/3192817676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa62c069180 con 0x7fa62c1030f0 2024-06-06T04:28:58.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.683+0000 7fa622ffd700 1 -- 172.21.15.45:0/3192817676 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa61c02a070 con 0x7fa62c1030f0 2024-06-06T04:28:58.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.683+0000 7fa622ffd700 1 --2- 172.21.15.45:0/3192817676 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa6180a2cb0 0x7fa6180a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:28:58.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.684+0000 7fa631a3c700 1 --2- 172.21.15.45:0/3192817676 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa6180a2cb0 0x7fa6180a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:28:58.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.684+0000 7fa622ffd700 1 -- 172.21.15.45:0/3192817676 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7fa61c0d0290 con 0x7fa62c1030f0 2024-06-06T04:28:58.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.684+0000 7fa631a3c700 1 --2- 172.21.15.45:0/3192817676 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa6180a2cb0 0x7fa6180a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa628005fd0 tx=0x7fa628005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:28:58.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.687+0000 7fa622ffd700 1 -- 172.21.15.45:0/3192817676 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa61c013070 con 0x7fa62c1030f0 2024-06-06T04:28:58.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.892+0000 7fa633ca0700 1 -- 172.21.15.45:0/3192817676 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa62c0628d0 con 0x7fa6180a2cb0 2024-06-06T04:28:58.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.901+0000 7fa622ffd700 1 -- 172.21.15.45:0/3192817676 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340617 (secure 0 0 0) 0x7fa62c0628d0 con 0x7fa6180a2cb0 2024-06-06T04:28:58.906 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:28:58.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.905+0000 7fa633ca0700 1 -- 172.21.15.45:0/3192817676 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa6180a2cb0 msgr2=0x7fa6180a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:58.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.905+0000 7fa633ca0700 1 --2- 172.21.15.45:0/3192817676 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa6180a2cb0 0x7fa6180a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa628005fd0 tx=0x7fa628005ee0 comp rx=0 tx=0).stop 2024-06-06T04:28:58.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.905+0000 7fa633ca0700 1 -- 172.21.15.45:0/3192817676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa62c1030f0 msgr2=0x7fa62c074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:28:58.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.905+0000 7fa633ca0700 1 --2- 172.21.15.45:0/3192817676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa62c1030f0 0x7fa62c074cd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa61c006010 tx=0x7fa61c009f80 comp rx=0 tx=0).stop 2024-06-06T04:28:58.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.905+0000 7fa633ca0700 1 -- 172.21.15.45:0/3192817676 shutdown_connections 2024-06-06T04:28:58.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.905+0000 7fa633ca0700 1 --2- 172.21.15.45:0/3192817676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa62c107840 0x7fa62c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:58.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.905+0000 7fa633ca0700 1 --2- 172.21.15.45:0/3192817676 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa6180a2cb0 0x7fa6180a5170 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:58.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.905+0000 7fa633ca0700 1 --2- 172.21.15.45:0/3192817676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa62c102740 0x7fa62c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:58.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.905+0000 7fa633ca0700 1 --2- 172.21.15.45:0/3192817676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa62c1030f0 0x7fa62c074cd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:28:58.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.905+0000 7fa633ca0700 1 -- 172.21.15.45:0/3192817676 >> 172.21.15.45:0/3192817676 conn(0x7fa62c0795f0 msgr2=0x7fa62c1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:28:58.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.905+0000 7fa633ca0700 1 -- 172.21.15.45:0/3192817676 shutdown_connections 2024-06-06T04:28:58.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:28:58.905+0000 7fa633ca0700 1 -- 172.21.15.45:0/3192817676 wait complete. 2024-06-06T04:28:58.910 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-06-06T04:28:59.728 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-06-06T04:28:58.381865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657566,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":313,"num_read_kb":272,"num_write":351,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374748,"kb_used_data":13116,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567076,"statfs":{"total":1151940427776,"available":1151556685824,"internally_reserved":0,"allocated":13430784,"data_stored":8589666,"data_compressed":426471,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33876,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":88,"num_read_kb":75,"num_write":20,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003528"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720845+0000","last_change":"2024-06-06T04:27:22.574900+0000","last_active":"2024-06-06T04:28:05.720845+0000","last_peered":"2024-06-06T04:28:05.720845+0000","last_clean":"2024-06-06T04:28:05.720845+0000","last_became_active":"2024-06-06T04:27:22.574389+0000","last_became_peered":"2024-06-06T04:27:22.574389+0000","last_unstale":"2024-06-06T04:28:05.720845+0000","last_undegraded":"2024-06-06T04:28:05.720845+0000","last_fullsized":"2024-06-06T04:28:05.720845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:39.246463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720674+0000","last_change":"2024-06-06T04:27:22.565543+0000","last_active":"2024-06-06T04:28:05.720674+0000","last_peered":"2024-06-06T04:28:05.720674+0000","last_clean":"2024-06-06T04:28:05.720674+0000","last_became_active":"2024-06-06T04:27:22.563833+0000","last_became_peered":"2024-06-06T04:27:22.563833+0000","last_unstale":"2024-06-06T04:28:05.720674+0000","last_undegraded":"2024-06-06T04:28:05.720674+0000","last_fullsized":"2024-06-06T04:28:05.720674+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:26:17.810782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720208+0000","last_change":"2024-06-06T04:27:22.570107+0000","last_active":"2024-06-06T04:28:05.720208+0000","last_peered":"2024-06-06T04:28:05.720208+0000","last_clean":"2024-06-06T04:28:05.720208+0000","last_became_active":"2024-06-06T04:27:22.569788+0000","last_became_peered":"2024-06-06T04:27:22.569788+0000","last_unstale":"2024-06-06T04:28:05.720208+0000","last_undegraded":"2024-06-06T04:28:05.720208+0000","last_fullsized":"2024-06-06T04:28:05.720208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:48:20.454782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717897+0000","last_change":"2024-06-06T04:27:22.559431+0000","last_active":"2024-06-06T04:28:05.717897+0000","last_peered":"2024-06-06T04:28:05.717897+0000","last_clean":"2024-06-06T04:28:05.717897+0000","last_became_active":"2024-06-06T04:27:22.558961+0000","last_became_peered":"2024-06-06T04:27:22.558961+0000","last_unstale":"2024-06-06T04:28:05.717897+0000","last_undegraded":"2024-06-06T04:28:05.717897+0000","last_fullsized":"2024-06-06T04:28:05.717897+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:59.445283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719982+0000","last_change":"2024-06-06T04:27:22.561598+0000","last_active":"2024-06-06T04:28:05.719982+0000","last_peered":"2024-06-06T04:28:05.719982+0000","last_clean":"2024-06-06T04:28:05.719982+0000","last_became_active":"2024-06-06T04:27:22.561198+0000","last_became_peered":"2024-06-06T04:27:22.561198+0000","last_unstale":"2024-06-06T04:28:05.719982+0000","last_undegraded":"2024-06-06T04:28:05.719982+0000","last_fullsized":"2024-06-06T04:28:05.719982+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:20:28.314941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719457+0000","last_change":"2024-06-06T04:27:22.569395+0000","last_active":"2024-06-06T04:28:05.719457+0000","last_peered":"2024-06-06T04:28:05.719457+0000","last_clean":"2024-06-06T04:28:05.719457+0000","last_became_active":"2024-06-06T04:27:22.569054+0000","last_became_peered":"2024-06-06T04:27:22.569054+0000","last_unstale":"2024-06-06T04:28:05.719457+0000","last_undegraded":"2024-06-06T04:28:05.719457+0000","last_fullsized":"2024-06-06T04:28:05.719457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:03:56.630677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724921+0000","last_change":"2024-06-06T04:27:22.577121+0000","last_active":"2024-06-06T04:28:05.724921+0000","last_peered":"2024-06-06T04:28:05.724921+0000","last_clean":"2024-06-06T04:28:05.724921+0000","last_became_active":"2024-06-06T04:27:22.575574+0000","last_became_peered":"2024-06-06T04:27:22.575574+0000","last_unstale":"2024-06-06T04:28:05.724921+0000","last_undegraded":"2024-06-06T04:28:05.724921+0000","last_fullsized":"2024-06-06T04:28:05.724921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:05:16.799335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.716776+0000","last_change":"2024-06-06T04:27:22.570051+0000","last_active":"2024-06-06T04:28:05.716776+0000","last_peered":"2024-06-06T04:28:05.716776+0000","last_clean":"2024-06-06T04:28:05.716776+0000","last_became_active":"2024-06-06T04:27:22.569640+0000","last_became_peered":"2024-06-06T04:27:22.569640+0000","last_unstale":"2024-06-06T04:28:05.716776+0000","last_undegraded":"2024-06-06T04:28:05.716776+0000","last_fullsized":"2024-06-06T04:28:05.716776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:48:00.744167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.721268+0000","last_change":"2024-06-06T04:27:22.574900+0000","last_active":"2024-06-06T04:28:05.721268+0000","last_peered":"2024-06-06T04:28:05.721268+0000","last_clean":"2024-06-06T04:28:05.721268+0000","last_became_active":"2024-06-06T04:27:22.574570+0000","last_became_peered":"2024-06-06T04:27:22.574570+0000","last_unstale":"2024-06-06T04:28:05.721268+0000","last_undegraded":"2024-06-06T04:28:05.721268+0000","last_fullsized":"2024-06-06T04:28:05.721268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:54.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720075+0000","last_change":"2024-06-06T04:27:22.570581+0000","last_active":"2024-06-06T04:28:05.720075+0000","last_peered":"2024-06-06T04:28:05.720075+0000","last_clean":"2024-06-06T04:28:05.720075+0000","last_became_active":"2024-06-06T04:27:22.570243+0000","last_became_peered":"2024-06-06T04:27:22.570243+0000","last_unstale":"2024-06-06T04:28:05.720075+0000","last_undegraded":"2024-06-06T04:28:05.720075+0000","last_fullsized":"2024-06-06T04:28:05.720075+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:27.087527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725926+0000","last_change":"2024-06-06T04:27:22.571162+0000","last_active":"2024-06-06T04:28:05.725926+0000","last_peered":"2024-06-06T04:28:05.725926+0000","last_clean":"2024-06-06T04:28:05.725926+0000","last_became_active":"2024-06-06T04:27:22.570843+0000","last_became_peered":"2024-06-06T04:27:22.570843+0000","last_unstale":"2024-06-06T04:28:05.725926+0000","last_undegraded":"2024-06-06T04:28:05.725926+0000","last_fullsized":"2024-06-06T04:28:05.725926+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:43:05.590240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717804+0000","last_change":"2024-06-06T04:27:22.567420+0000","last_active":"2024-06-06T04:28:05.717804+0000","last_peered":"2024-06-06T04:28:05.717804+0000","last_clean":"2024-06-06T04:28:05.717804+0000","last_became_active":"2024-06-06T04:27:22.566750+0000","last_became_peered":"2024-06-06T04:27:22.566750+0000","last_unstale":"2024-06-06T04:28:05.717804+0000","last_undegraded":"2024-06-06T04:28:05.717804+0000","last_fullsized":"2024-06-06T04:28:05.717804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:21.669965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719673+0000","last_change":"2024-06-06T04:27:22.566831+0000","last_active":"2024-06-06T04:28:05.719673+0000","last_peered":"2024-06-06T04:28:05.719673+0000","last_clean":"2024-06-06T04:28:05.719673+0000","last_became_active":"2024-06-06T04:27:22.566255+0000","last_became_peered":"2024-06-06T04:27:22.566255+0000","last_unstale":"2024-06-06T04:28:05.719673+0000","last_undegraded":"2024-06-06T04:28:05.719673+0000","last_fullsized":"2024-06-06T04:28:05.719673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:40:22.486674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720828+0000","last_change":"2024-06-06T04:27:22.564596+0000","last_active":"2024-06-06T04:28:05.720828+0000","last_peered":"2024-06-06T04:28:05.720828+0000","last_clean":"2024-06-06T04:28:05.720828+0000","last_became_active":"2024-06-06T04:27:22.563694+0000","last_became_peered":"2024-06-06T04:27:22.563694+0000","last_unstale":"2024-06-06T04:28:05.720828+0000","last_undegraded":"2024-06-06T04:28:05.720828+0000","last_fullsized":"2024-06-06T04:28:05.720828+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:18.568266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.721206+0000","last_change":"2024-06-06T04:27:22.559395+0000","last_active":"2024-06-06T04:28:05.721206+0000","last_peered":"2024-06-06T04:28:05.721206+0000","last_clean":"2024-06-06T04:28:05.721206+0000","last_became_active":"2024-06-06T04:27:22.559030+0000","last_became_peered":"2024-06-06T04:27:22.559030+0000","last_unstale":"2024-06-06T04:28:05.721206+0000","last_undegraded":"2024-06-06T04:28:05.721206+0000","last_fullsized":"2024-06-06T04:28:05.721206+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:08.562184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719969+0000","last_change":"2024-06-06T04:27:22.569657+0000","last_active":"2024-06-06T04:28:05.719969+0000","last_peered":"2024-06-06T04:28:05.719969+0000","last_clean":"2024-06-06T04:28:05.719969+0000","last_became_active":"2024-06-06T04:27:22.569359+0000","last_became_peered":"2024-06-06T04:27:22.569359+0000","last_unstale":"2024-06-06T04:28:05.719969+0000","last_undegraded":"2024-06-06T04:28:05.719969+0000","last_fullsized":"2024-06-06T04:28:05.719969+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:42:20.197715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720550+0000","last_change":"2024-06-06T04:27:22.574661+0000","last_active":"2024-06-06T04:28:05.720550+0000","last_peered":"2024-06-06T04:28:05.720550+0000","last_clean":"2024-06-06T04:28:05.720550+0000","last_became_active":"2024-06-06T04:27:22.574310+0000","last_became_peered":"2024-06-06T04:27:22.574310+0000","last_unstale":"2024-06-06T04:28:05.720550+0000","last_undegraded":"2024-06-06T04:28:05.720550+0000","last_fullsized":"2024-06-06T04:28:05.720550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:45:18.772270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.895238+0000","last_change":"2024-06-06T04:27:22.565575+0000","last_active":"2024-06-06T04:28:05.895238+0000","last_peered":"2024-06-06T04:28:05.895238+0000","last_clean":"2024-06-06T04:28:05.895238+0000","last_became_active":"2024-06-06T04:27:22.565256+0000","last_became_peered":"2024-06-06T04:27:22.565256+0000","last_unstale":"2024-06-06T04:28:05.895238+0000","last_undegraded":"2024-06-06T04:28:05.895238+0000","last_fullsized":"2024-06-06T04:28:05.895238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:49.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.718703+0000","last_change":"2024-06-06T04:27:22.559606+0000","last_active":"2024-06-06T04:28:05.718703+0000","last_peered":"2024-06-06T04:28:05.718703+0000","last_clean":"2024-06-06T04:28:05.718703+0000","last_became_active":"2024-06-06T04:27:22.557881+0000","last_became_peered":"2024-06-06T04:27:22.557881+0000","last_unstale":"2024-06-06T04:28:05.718703+0000","last_undegraded":"2024-06-06T04:28:05.718703+0000","last_fullsized":"2024-06-06T04:28:05.718703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:31:06.269785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717560+0000","last_change":"2024-06-06T04:27:32.626612+0000","last_active":"2024-06-06T04:28:05.717560+0000","last_peered":"2024-06-06T04:28:05.717560+0000","last_clean":"2024-06-06T04:28:05.717560+0000","last_became_active":"2024-06-06T04:27:32.626113+0000","last_became_peered":"2024-06-06T04:27:32.626113+0000","last_unstale":"2024-06-06T04:28:05.717560+0000","last_undegraded":"2024-06-06T04:28:05.717560+0000","last_fullsized":"2024-06-06T04:28:05.717560+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.318738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":50,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:27.721774+0000","last_change":"2024-06-06T04:27:19.302744+0000","last_active":"2024-06-06T04:28:27.721774+0000","last_peered":"2024-06-06T04:28:27.721774+0000","last_clean":"2024-06-06T04:28:27.721774+0000","last_became_active":"2024-06-06T04:27:19.302389+0000","last_became_peered":"2024-06-06T04:27:19.302389+0000","last_unstale":"2024-06-06T04:28:27.721774+0000","last_undegraded":"2024-06-06T04:28:27.721774+0000","last_fullsized":"2024-06-06T04:28:27.721774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:49.872618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717319+0000","last_change":"2024-06-06T04:27:22.570220+0000","last_active":"2024-06-06T04:28:05.717319+0000","last_peered":"2024-06-06T04:28:05.717319+0000","last_clean":"2024-06-06T04:28:05.717319+0000","last_became_active":"2024-06-06T04:27:22.569835+0000","last_became_peered":"2024-06-06T04:27:22.569835+0000","last_unstale":"2024-06-06T04:28:05.717319+0000","last_undegraded":"2024-06-06T04:28:05.717319+0000","last_fullsized":"2024-06-06T04:28:05.717319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:54:57.372561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717188+0000","last_change":"2024-06-06T04:27:32.622407+0000","last_active":"2024-06-06T04:28:05.717188+0000","last_peered":"2024-06-06T04:28:05.717188+0000","last_clean":"2024-06-06T04:28:05.717188+0000","last_became_active":"2024-06-06T04:27:32.621868+0000","last_became_peered":"2024-06-06T04:27:32.621868+0000","last_unstale":"2024-06-06T04:28:05.717188+0000","last_undegraded":"2024-06-06T04:28:05.717188+0000","last_fullsized":"2024-06-06T04:28:05.717188+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:50:05.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717512+0000","last_change":"2024-06-06T04:27:19.516235+0000","last_active":"2024-06-06T04:28:05.717512+0000","last_peered":"2024-06-06T04:28:05.717512+0000","last_clean":"2024-06-06T04:28:05.717512+0000","last_became_active":"2024-06-06T04:27:19.515849+0000","last_became_peered":"2024-06-06T04:27:19.515849+0000","last_unstale":"2024-06-06T04:28:05.717512+0000","last_undegraded":"2024-06-06T04:28:05.717512+0000","last_fullsized":"2024-06-06T04:28:05.717512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:52:42.722103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720330+0000","last_change":"2024-06-06T04:27:22.570797+0000","last_active":"2024-06-06T04:28:05.720330+0000","last_peered":"2024-06-06T04:28:05.720330+0000","last_clean":"2024-06-06T04:28:05.720330+0000","last_became_active":"2024-06-06T04:27:22.570305+0000","last_became_peered":"2024-06-06T04:27:22.570305+0000","last_unstale":"2024-06-06T04:28:05.720330+0000","last_undegraded":"2024-06-06T04:28:05.720330+0000","last_fullsized":"2024-06-06T04:28:05.720330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:04:14.545306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717388+0000","last_change":"2024-06-06T04:27:32.623241+0000","last_active":"2024-06-06T04:28:05.717388+0000","last_peered":"2024-06-06T04:28:05.717388+0000","last_clean":"2024-06-06T04:28:05.717388+0000","last_became_active":"2024-06-06T04:27:32.622751+0000","last_became_peered":"2024-06-06T04:27:32.622751+0000","last_unstale":"2024-06-06T04:28:05.717388+0000","last_undegraded":"2024-06-06T04:28:05.717388+0000","last_fullsized":"2024-06-06T04:28:05.717388+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:24:53.446329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:27.719217+0000","last_change":"2024-06-06T04:28:05.733350+0000","last_active":"2024-06-06T04:28:27.719217+0000","last_peered":"2024-06-06T04:28:27.719217+0000","last_clean":"2024-06-06T04:28:27.719217+0000","last_became_active":"2024-06-06T04:28:05.726556+0000","last_became_peered":"2024-06-06T04:28:05.726556+0000","last_unstale":"2024-06-06T04:28:27.719217+0000","last_undegraded":"2024-06-06T04:28:27.719217+0000","last_fullsized":"2024-06-06T04:28:27.719217+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:26.636625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720742+0000","last_change":"2024-06-06T04:27:22.571731+0000","last_active":"2024-06-06T04:28:05.720742+0000","last_peered":"2024-06-06T04:28:05.720742+0000","last_clean":"2024-06-06T04:28:05.720742+0000","last_became_active":"2024-06-06T04:27:22.571398+0000","last_became_peered":"2024-06-06T04:27:22.571398+0000","last_unstale":"2024-06-06T04:28:05.720742+0000","last_undegraded":"2024-06-06T04:28:05.720742+0000","last_fullsized":"2024-06-06T04:28:05.720742+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:22:45.144427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717172+0000","last_change":"2024-06-06T04:27:32.618465+0000","last_active":"2024-06-06T04:28:05.717172+0000","last_peered":"2024-06-06T04:28:05.717172+0000","last_clean":"2024-06-06T04:28:05.717172+0000","last_became_active":"2024-06-06T04:27:32.618066+0000","last_became_peered":"2024-06-06T04:27:32.618066+0000","last_unstale":"2024-06-06T04:28:05.717172+0000","last_undegraded":"2024-06-06T04:28:05.717172+0000","last_fullsized":"2024-06-06T04:28:05.717172+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:54:23.988687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.730858+0000","last_change":"2024-06-06T04:28:05.730381+0000","last_active":"2024-06-06T04:28:05.730858+0000","last_peered":"2024-06-06T04:28:05.730858+0000","last_clean":"2024-06-06T04:28:05.730858+0000","last_became_active":"2024-06-06T04:28:05.726552+0000","last_became_peered":"2024-06-06T04:28:05.726552+0000","last_unstale":"2024-06-06T04:28:05.730858+0000","last_undegraded":"2024-06-06T04:28:05.730858+0000","last_fullsized":"2024-06-06T04:28:05.730858+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:52.903042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.726700+0000","last_change":"2024-06-06T04:27:22.573769+0000","last_active":"2024-06-06T04:28:05.726700+0000","last_peered":"2024-06-06T04:28:05.726700+0000","last_clean":"2024-06-06T04:28:05.726700+0000","last_became_active":"2024-06-06T04:27:22.572674+0000","last_became_peered":"2024-06-06T04:27:22.572674+0000","last_unstale":"2024-06-06T04:28:05.726700+0000","last_undegraded":"2024-06-06T04:28:05.726700+0000","last_fullsized":"2024-06-06T04:28:05.726700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:57:22.653052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.718601+0000","last_change":"2024-06-06T04:27:32.617934+0000","last_active":"2024-06-06T04:28:05.718601+0000","last_peered":"2024-06-06T04:28:05.718601+0000","last_clean":"2024-06-06T04:28:05.718601+0000","last_became_active":"2024-06-06T04:27:32.617537+0000","last_became_peered":"2024-06-06T04:27:32.617537+0000","last_unstale":"2024-06-06T04:28:05.718601+0000","last_undegraded":"2024-06-06T04:28:05.718601+0000","last_fullsized":"2024-06-06T04:28:05.718601+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:21:48.319434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'5","reported_seq":74,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:54.770183+0000","last_change":"2024-06-06T04:27:19.306236+0000","last_active":"2024-06-06T04:28:54.770183+0000","last_peered":"2024-06-06T04:28:54.770183+0000","last_clean":"2024-06-06T04:28:54.770183+0000","last_became_active":"2024-06-06T04:27:19.305850+0000","last_became_peered":"2024-06-06T04:27:19.305850+0000","last_unstale":"2024-06-06T04:28:54.770183+0000","last_undegraded":"2024-06-06T04:28:54.770183+0000","last_fullsized":"2024-06-06T04:28:54.770183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:58:47.310040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.721086+0000","last_change":"2024-06-06T04:27:22.567440+0000","last_active":"2024-06-06T04:28:05.721086+0000","last_peered":"2024-06-06T04:28:05.721086+0000","last_clean":"2024-06-06T04:28:05.721086+0000","last_became_active":"2024-06-06T04:27:22.566732+0000","last_became_peered":"2024-06-06T04:27:22.566732+0000","last_unstale":"2024-06-06T04:28:05.721086+0000","last_undegraded":"2024-06-06T04:28:05.721086+0000","last_fullsized":"2024-06-06T04:28:05.721086+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:48:46.270103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.718261+0000","last_change":"2024-06-06T04:27:32.626554+0000","last_active":"2024-06-06T04:28:05.718261+0000","last_peered":"2024-06-06T04:28:05.718261+0000","last_clean":"2024-06-06T04:28:05.718261+0000","last_became_active":"2024-06-06T04:27:32.626108+0000","last_became_peered":"2024-06-06T04:27:32.626108+0000","last_unstale":"2024-06-06T04:28:05.718261+0000","last_undegraded":"2024-06-06T04:28:05.718261+0000","last_fullsized":"2024-06-06T04:28:05.718261+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:37:43.811005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:27.718526+0000","last_change":"2024-06-06T04:27:19.306821+0000","last_active":"2024-06-06T04:28:27.718526+0000","last_peered":"2024-06-06T04:28:27.718526+0000","last_clean":"2024-06-06T04:28:27.718526+0000","last_became_active":"2024-06-06T04:27:19.306489+0000","last_became_peered":"2024-06-06T04:27:19.306489+0000","last_unstale":"2024-06-06T04:28:27.718526+0000","last_undegraded":"2024-06-06T04:28:27.718526+0000","last_fullsized":"2024-06-06T04:28:27.718526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:31:28.973112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.726240+0000","last_change":"2024-06-06T04:27:22.570617+0000","last_active":"2024-06-06T04:28:05.726240+0000","last_peered":"2024-06-06T04:28:05.726240+0000","last_clean":"2024-06-06T04:28:05.726240+0000","last_became_active":"2024-06-06T04:27:22.570271+0000","last_became_peered":"2024-06-06T04:27:22.570271+0000","last_unstale":"2024-06-06T04:28:05.726240+0000","last_undegraded":"2024-06-06T04:28:05.726240+0000","last_fullsized":"2024-06-06T04:28:05.726240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:45:33.890164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720406+0000","last_change":"2024-06-06T04:27:32.628132+0000","last_active":"2024-06-06T04:28:05.720406+0000","last_peered":"2024-06-06T04:28:05.720406+0000","last_clean":"2024-06-06T04:28:05.720406+0000","last_became_active":"2024-06-06T04:27:32.627658+0000","last_became_peered":"2024-06-06T04:27:32.627658+0000","last_unstale":"2024-06-06T04:28:05.720406+0000","last_undegraded":"2024-06-06T04:28:05.720406+0000","last_fullsized":"2024-06-06T04:28:05.720406+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:24.817079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:27.718694+0000","last_change":"2024-06-06T04:27:19.300510+0000","last_active":"2024-06-06T04:28:27.718694+0000","last_peered":"2024-06-06T04:28:27.718694+0000","last_clean":"2024-06-06T04:28:27.718694+0000","last_became_active":"2024-06-06T04:27:19.300150+0000","last_became_peered":"2024-06-06T04:27:19.300150+0000","last_unstale":"2024-06-06T04:28:27.718694+0000","last_undegraded":"2024-06-06T04:28:27.718694+0000","last_fullsized":"2024-06-06T04:28:27.718694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:38:25.084690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.716894+0000","last_change":"2024-06-06T04:27:22.563864+0000","last_active":"2024-06-06T04:28:05.716894+0000","last_peered":"2024-06-06T04:28:05.716894+0000","last_clean":"2024-06-06T04:28:05.716894+0000","last_became_active":"2024-06-06T04:27:22.563555+0000","last_became_peered":"2024-06-06T04:27:22.563555+0000","last_unstale":"2024-06-06T04:28:05.716894+0000","last_undegraded":"2024-06-06T04:28:05.716894+0000","last_fullsized":"2024-06-06T04:28:05.716894+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:39:33.707404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720390+0000","last_change":"2024-06-06T04:27:32.617627+0000","last_active":"2024-06-06T04:28:05.720390+0000","last_peered":"2024-06-06T04:28:05.720390+0000","last_clean":"2024-06-06T04:28:05.720390+0000","last_became_active":"2024-06-06T04:27:32.617187+0000","last_became_peered":"2024-06-06T04:27:32.617187+0000","last_unstale":"2024-06-06T04:28:05.720390+0000","last_undegraded":"2024-06-06T04:28:05.720390+0000","last_fullsized":"2024-06-06T04:28:05.720390+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:24:52.554176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724506+0000","last_change":"2024-06-06T04:27:19.304316+0000","last_active":"2024-06-06T04:28:05.724506+0000","last_peered":"2024-06-06T04:28:05.724506+0000","last_clean":"2024-06-06T04:28:05.724506+0000","last_became_active":"2024-06-06T04:27:19.303952+0000","last_became_peered":"2024-06-06T04:27:19.303952+0000","last_unstale":"2024-06-06T04:28:05.724506+0000","last_undegraded":"2024-06-06T04:28:05.724506+0000","last_fullsized":"2024-06-06T04:28:05.724506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:26.886916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.894742+0000","last_change":"2024-06-06T04:27:22.561824+0000","last_active":"2024-06-06T04:28:05.894742+0000","last_peered":"2024-06-06T04:28:05.894742+0000","last_clean":"2024-06-06T04:28:05.894742+0000","last_became_active":"2024-06-06T04:27:22.561495+0000","last_became_peered":"2024-06-06T04:27:22.561495+0000","last_unstale":"2024-06-06T04:28:05.894742+0000","last_undegraded":"2024-06-06T04:28:05.894742+0000","last_fullsized":"2024-06-06T04:28:05.894742+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:40:31.314627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.716867+0000","last_change":"2024-06-06T04:27:32.624540+0000","last_active":"2024-06-06T04:28:05.716867+0000","last_peered":"2024-06-06T04:28:05.716867+0000","last_clean":"2024-06-06T04:28:05.716867+0000","last_became_active":"2024-06-06T04:27:32.624036+0000","last_became_peered":"2024-06-06T04:27:32.624036+0000","last_unstale":"2024-06-06T04:28:05.716867+0000","last_undegraded":"2024-06-06T04:28:05.716867+0000","last_fullsized":"2024-06-06T04:28:05.716867+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:16.983576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":68,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:54.769736+0000","last_change":"2024-06-06T04:27:19.304614+0000","last_active":"2024-06-06T04:28:54.769736+0000","last_peered":"2024-06-06T04:28:54.769736+0000","last_clean":"2024-06-06T04:28:54.769736+0000","last_became_active":"2024-06-06T04:27:19.304126+0000","last_became_peered":"2024-06-06T04:27:19.304126+0000","last_unstale":"2024-06-06T04:28:54.769736+0000","last_undegraded":"2024-06-06T04:28:54.769736+0000","last_fullsized":"2024-06-06T04:28:54.769736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:07:39.908630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720767+0000","last_change":"2024-06-06T04:27:22.572534+0000","last_active":"2024-06-06T04:28:05.720767+0000","last_peered":"2024-06-06T04:28:05.720767+0000","last_clean":"2024-06-06T04:28:05.720767+0000","last_became_active":"2024-06-06T04:27:22.572214+0000","last_became_peered":"2024-06-06T04:27:22.572214+0000","last_unstale":"2024-06-06T04:28:05.720767+0000","last_undegraded":"2024-06-06T04:28:05.720767+0000","last_fullsized":"2024-06-06T04:28:05.720767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:13.176045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724101+0000","last_change":"2024-06-06T04:28:05.723450+0000","last_active":"2024-06-06T04:28:05.724101+0000","last_peered":"2024-06-06T04:28:05.724101+0000","last_clean":"2024-06-06T04:28:05.724101+0000","last_became_active":"2024-06-06T04:28:05.722928+0000","last_became_peered":"2024-06-06T04:28:05.722928+0000","last_unstale":"2024-06-06T04:28:05.724101+0000","last_undegraded":"2024-06-06T04:28:05.724101+0000","last_fullsized":"2024-06-06T04:28:05.724101+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:53.093276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:26.712609+0000","last_change":"2024-06-06T04:27:19.308465+0000","last_active":"2024-06-06T04:28:26.712609+0000","last_peered":"2024-06-06T04:28:26.712609+0000","last_clean":"2024-06-06T04:28:26.712609+0000","last_became_active":"2024-06-06T04:27:19.308077+0000","last_became_peered":"2024-06-06T04:27:19.308077+0000","last_unstale":"2024-06-06T04:28:26.712609+0000","last_undegraded":"2024-06-06T04:28:26.712609+0000","last_fullsized":"2024-06-06T04:28:26.712609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:37:48.282613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720627+0000","last_change":"2024-06-06T04:27:22.574141+0000","last_active":"2024-06-06T04:28:05.720627+0000","last_peered":"2024-06-06T04:28:05.720627+0000","last_clean":"2024-06-06T04:28:05.720627+0000","last_became_active":"2024-06-06T04:27:22.573679+0000","last_became_peered":"2024-06-06T04:27:22.573679+0000","last_unstale":"2024-06-06T04:28:05.720627+0000","last_undegraded":"2024-06-06T04:28:05.720627+0000","last_fullsized":"2024-06-06T04:28:05.720627+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:27:25.177990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717527+0000","last_change":"2024-06-06T04:27:32.624148+0000","last_active":"2024-06-06T04:28:05.717527+0000","last_peered":"2024-06-06T04:28:05.717527+0000","last_clean":"2024-06-06T04:28:05.717527+0000","last_became_active":"2024-06-06T04:27:32.623448+0000","last_became_peered":"2024-06-06T04:27:32.623448+0000","last_unstale":"2024-06-06T04:28:05.717527+0000","last_undegraded":"2024-06-06T04:28:05.717527+0000","last_fullsized":"2024-06-06T04:28:05.717527+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:02.037296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:26.713006+0000","last_change":"2024-06-06T04:27:19.516875+0000","last_active":"2024-06-06T04:28:26.713006+0000","last_peered":"2024-06-06T04:28:26.713006+0000","last_clean":"2024-06-06T04:28:26.713006+0000","last_became_active":"2024-06-06T04:27:19.516393+0000","last_became_peered":"2024-06-06T04:27:19.516393+0000","last_unstale":"2024-06-06T04:28:26.713006+0000","last_undegraded":"2024-06-06T04:28:26.713006+0000","last_fullsized":"2024-06-06T04:28:26.713006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:20.079218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724043+0000","last_change":"2024-06-06T04:28:05.724043+0000","last_active":"2024-06-06T04:28:05.724043+0000","last_peered":"2024-06-06T04:28:05.724043+0000","last_clean":"2024-06-06T04:28:05.724043+0000","last_became_active":"2024-06-06T04:28:05.723622+0000","last_became_peered":"2024-06-06T04:28:05.723622+0000","last_unstale":"2024-06-06T04:28:05.724043+0000","last_undegraded":"2024-06-06T04:28:05.724043+0000","last_fullsized":"2024-06-06T04:28:05.724043+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:42.162400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.894796+0000","last_change":"2024-06-06T04:27:32.626366+0000","last_active":"2024-06-06T04:28:05.894796+0000","last_peered":"2024-06-06T04:28:05.894796+0000","last_clean":"2024-06-06T04:28:05.894796+0000","last_became_active":"2024-06-06T04:27:32.625847+0000","last_became_peered":"2024-06-06T04:27:32.625847+0000","last_unstale":"2024-06-06T04:28:05.894796+0000","last_undegraded":"2024-06-06T04:28:05.894796+0000","last_fullsized":"2024-06-06T04:28:05.894796+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:47:44.167531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"98'34","reported_seq":134,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:55.771426+0000","last_change":"2024-06-06T04:27:19.303581+0000","last_active":"2024-06-06T04:28:55.771426+0000","last_peered":"2024-06-06T04:28:55.771426+0000","last_clean":"2024-06-06T04:28:55.771426+0000","last_became_active":"2024-06-06T04:27:19.303234+0000","last_became_peered":"2024-06-06T04:27:19.303234+0000","last_unstale":"2024-06-06T04:28:55.771426+0000","last_undegraded":"2024-06-06T04:28:55.771426+0000","last_fullsized":"2024-06-06T04:28:55.771426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:37.721588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62964,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":62,"num_read_kb":93,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719957+0000","last_change":"2024-06-06T04:27:22.567177+0000","last_active":"2024-06-06T04:28:05.719957+0000","last_peered":"2024-06-06T04:28:05.719957+0000","last_clean":"2024-06-06T04:28:05.719957+0000","last_became_active":"2024-06-06T04:27:22.566751+0000","last_became_peered":"2024-06-06T04:27:22.566751+0000","last_unstale":"2024-06-06T04:28:05.719957+0000","last_undegraded":"2024-06-06T04:28:05.719957+0000","last_fullsized":"2024-06-06T04:28:05.719957+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:37:15.914505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.727434+0000","last_change":"2024-06-06T04:27:32.619319+0000","last_active":"2024-06-06T04:28:05.727434+0000","last_peered":"2024-06-06T04:28:05.727434+0000","last_clean":"2024-06-06T04:28:05.727434+0000","last_became_active":"2024-06-06T04:27:32.618843+0000","last_became_peered":"2024-06-06T04:27:32.618843+0000","last_unstale":"2024-06-06T04:28:05.727434+0000","last_undegraded":"2024-06-06T04:28:05.727434+0000","last_fullsized":"2024-06-06T04:28:05.727434+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:32:20.071512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:27.720338+0000","last_change":"2024-06-06T04:27:19.304731+0000","last_active":"2024-06-06T04:28:27.720338+0000","last_peered":"2024-06-06T04:28:27.720338+0000","last_clean":"2024-06-06T04:28:27.720338+0000","last_became_active":"2024-06-06T04:27:19.304351+0000","last_became_peered":"2024-06-06T04:27:19.304351+0000","last_unstale":"2024-06-06T04:28:27.720338+0000","last_undegraded":"2024-06-06T04:28:27.720338+0000","last_fullsized":"2024-06-06T04:28:27.720338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:16:14.136633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719354+0000","last_change":"2024-06-06T04:27:22.573223+0000","last_active":"2024-06-06T04:28:05.719354+0000","last_peered":"2024-06-06T04:28:05.719354+0000","last_clean":"2024-06-06T04:28:05.719354+0000","last_became_active":"2024-06-06T04:27:22.572884+0000","last_became_peered":"2024-06-06T04:27:22.572884+0000","last_unstale":"2024-06-06T04:28:05.719354+0000","last_undegraded":"2024-06-06T04:28:05.719354+0000","last_fullsized":"2024-06-06T04:28:05.719354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:42:11.819847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.721922+0000","last_change":"2024-06-06T04:27:32.628645+0000","last_active":"2024-06-06T04:28:05.721922+0000","last_peered":"2024-06-06T04:28:05.721922+0000","last_clean":"2024-06-06T04:28:05.721922+0000","last_became_active":"2024-06-06T04:27:32.627586+0000","last_became_peered":"2024-06-06T04:27:32.627586+0000","last_unstale":"2024-06-06T04:28:05.721922+0000","last_undegraded":"2024-06-06T04:28:05.721922+0000","last_fullsized":"2024-06-06T04:28:05.721922+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:46:53.094511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:26.714189+0000","last_change":"2024-06-06T04:27:19.303805+0000","last_active":"2024-06-06T04:28:26.714189+0000","last_peered":"2024-06-06T04:28:26.714189+0000","last_clean":"2024-06-06T04:28:26.714189+0000","last_became_active":"2024-06-06T04:27:19.303391+0000","last_became_peered":"2024-06-06T04:27:19.303391+0000","last_unstale":"2024-06-06T04:28:26.714189+0000","last_undegraded":"2024-06-06T04:28:26.714189+0000","last_fullsized":"2024-06-06T04:28:26.714189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:30:27.336567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725582+0000","last_change":"2024-06-06T04:27:22.575148+0000","last_active":"2024-06-06T04:28:05.725582+0000","last_peered":"2024-06-06T04:28:05.725582+0000","last_clean":"2024-06-06T04:28:05.725582+0000","last_became_active":"2024-06-06T04:27:22.574591+0000","last_became_peered":"2024-06-06T04:27:22.574591+0000","last_unstale":"2024-06-06T04:28:05.725582+0000","last_undegraded":"2024-06-06T04:28:05.725582+0000","last_fullsized":"2024-06-06T04:28:05.725582+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:35:23.404644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.728254+0000","last_change":"2024-06-06T04:28:05.726734+0000","last_active":"2024-06-06T04:28:05.728254+0000","last_peered":"2024-06-06T04:28:05.728254+0000","last_clean":"2024-06-06T04:28:05.728254+0000","last_became_active":"2024-06-06T04:28:05.726238+0000","last_became_peered":"2024-06-06T04:28:05.726238+0000","last_unstale":"2024-06-06T04:28:05.728254+0000","last_undegraded":"2024-06-06T04:28:05.728254+0000","last_fullsized":"2024-06-06T04:28:05.728254+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:47.841475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"98'4","reported_seq":74,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:55.771500+0000","last_change":"2024-06-06T04:27:19.305571+0000","last_active":"2024-06-06T04:28:55.771500+0000","last_peered":"2024-06-06T04:28:55.771500+0000","last_clean":"2024-06-06T04:28:55.771500+0000","last_became_active":"2024-06-06T04:27:19.305183+0000","last_became_peered":"2024-06-06T04:27:19.305183+0000","last_unstale":"2024-06-06T04:28:55.771500+0000","last_undegraded":"2024-06-06T04:28:55.771500+0000","last_fullsized":"2024-06-06T04:28:55.771500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:51.205209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724269+0000","last_change":"2024-06-06T04:27:22.571135+0000","last_active":"2024-06-06T04:28:05.724269+0000","last_peered":"2024-06-06T04:28:05.724269+0000","last_clean":"2024-06-06T04:28:05.724269+0000","last_became_active":"2024-06-06T04:27:22.570352+0000","last_became_peered":"2024-06-06T04:27:22.570352+0000","last_unstale":"2024-06-06T04:28:05.724269+0000","last_undegraded":"2024-06-06T04:28:05.724269+0000","last_fullsized":"2024-06-06T04:28:05.724269+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:41:47.059281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724892+0000","last_change":"2024-06-06T04:28:05.724892+0000","last_active":"2024-06-06T04:28:05.724892+0000","last_peered":"2024-06-06T04:28:05.724892+0000","last_clean":"2024-06-06T04:28:05.724892+0000","last_became_active":"2024-06-06T04:28:05.724420+0000","last_became_peered":"2024-06-06T04:28:05.724420+0000","last_unstale":"2024-06-06T04:28:05.724892+0000","last_undegraded":"2024-06-06T04:28:05.724892+0000","last_fullsized":"2024-06-06T04:28:05.724892+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:39.879481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":55,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:27.721685+0000","last_change":"2024-06-06T04:27:19.516687+0000","last_active":"2024-06-06T04:28:27.721685+0000","last_peered":"2024-06-06T04:28:27.721685+0000","last_clean":"2024-06-06T04:28:27.721685+0000","last_became_active":"2024-06-06T04:27:19.516271+0000","last_became_peered":"2024-06-06T04:27:19.516271+0000","last_unstale":"2024-06-06T04:28:27.721685+0000","last_undegraded":"2024-06-06T04:28:27.721685+0000","last_fullsized":"2024-06-06T04:28:27.721685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:33:46.817727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725626+0000","last_change":"2024-06-06T04:27:22.572671+0000","last_active":"2024-06-06T04:28:05.725626+0000","last_peered":"2024-06-06T04:28:05.725626+0000","last_clean":"2024-06-06T04:28:05.725626+0000","last_became_active":"2024-06-06T04:27:22.572230+0000","last_became_peered":"2024-06-06T04:27:22.572230+0000","last_unstale":"2024-06-06T04:28:05.725626+0000","last_undegraded":"2024-06-06T04:28:05.725626+0000","last_fullsized":"2024-06-06T04:28:05.725626+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:36:48.393092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725500+0000","last_change":"2024-06-06T04:27:32.620131+0000","last_active":"2024-06-06T04:28:05.725500+0000","last_peered":"2024-06-06T04:28:05.725500+0000","last_clean":"2024-06-06T04:28:05.725500+0000","last_became_active":"2024-06-06T04:27:32.619686+0000","last_became_peered":"2024-06-06T04:27:32.619686+0000","last_unstale":"2024-06-06T04:28:05.725500+0000","last_undegraded":"2024-06-06T04:28:05.725500+0000","last_fullsized":"2024-06-06T04:28:05.725500+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:19:17.130410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:29.715083+0000","last_change":"2024-06-06T04:27:19.304682+0000","last_active":"2024-06-06T04:28:29.715083+0000","last_peered":"2024-06-06T04:28:29.715083+0000","last_clean":"2024-06-06T04:28:29.715083+0000","last_became_active":"2024-06-06T04:27:19.304262+0000","last_became_peered":"2024-06-06T04:27:19.304262+0000","last_unstale":"2024-06-06T04:28:29.715083+0000","last_undegraded":"2024-06-06T04:28:29.715083+0000","last_fullsized":"2024-06-06T04:28:29.715083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:45.819520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.721153+0000","last_change":"2024-06-06T04:27:22.569544+0000","last_active":"2024-06-06T04:28:05.721153+0000","last_peered":"2024-06-06T04:28:05.721153+0000","last_clean":"2024-06-06T04:28:05.721153+0000","last_became_active":"2024-06-06T04:27:22.569072+0000","last_became_peered":"2024-06-06T04:27:22.569072+0000","last_unstale":"2024-06-06T04:28:05.721153+0000","last_undegraded":"2024-06-06T04:28:05.721153+0000","last_fullsized":"2024-06-06T04:28:05.721153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:38:56.460433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720662+0000","last_change":"2024-06-06T04:27:32.622092+0000","last_active":"2024-06-06T04:28:05.720662+0000","last_peered":"2024-06-06T04:28:05.720662+0000","last_clean":"2024-06-06T04:28:05.720662+0000","last_became_active":"2024-06-06T04:27:32.621618+0000","last_became_peered":"2024-06-06T04:27:32.621618+0000","last_unstale":"2024-06-06T04:28:05.720662+0000","last_undegraded":"2024-06-06T04:28:05.720662+0000","last_fullsized":"2024-06-06T04:28:05.720662+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:03.067292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.895082+0000","last_change":"2024-06-06T04:27:19.516803+0000","last_active":"2024-06-06T04:28:05.895082+0000","last_peered":"2024-06-06T04:28:05.895082+0000","last_clean":"2024-06-06T04:28:05.895082+0000","last_became_active":"2024-06-06T04:27:19.516458+0000","last_became_peered":"2024-06-06T04:27:19.516458+0000","last_unstale":"2024-06-06T04:28:05.895082+0000","last_undegraded":"2024-06-06T04:28:05.895082+0000","last_fullsized":"2024-06-06T04:28:05.895082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:34:53.593825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719311+0000","last_change":"2024-06-06T04:27:22.564306+0000","last_active":"2024-06-06T04:28:05.719311+0000","last_peered":"2024-06-06T04:28:05.719311+0000","last_clean":"2024-06-06T04:28:05.719311+0000","last_became_active":"2024-06-06T04:27:22.563922+0000","last_became_peered":"2024-06-06T04:27:22.563922+0000","last_unstale":"2024-06-06T04:28:05.719311+0000","last_undegraded":"2024-06-06T04:28:05.719311+0000","last_fullsized":"2024-06-06T04:28:05.719311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:42:23.189047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719883+0000","last_change":"2024-06-06T04:27:32.617014+0000","last_active":"2024-06-06T04:28:05.719883+0000","last_peered":"2024-06-06T04:28:05.719883+0000","last_clean":"2024-06-06T04:28:05.719883+0000","last_became_active":"2024-06-06T04:27:32.616639+0000","last_became_peered":"2024-06-06T04:27:32.616639+0000","last_unstale":"2024-06-06T04:28:05.719883+0000","last_undegraded":"2024-06-06T04:28:05.719883+0000","last_fullsized":"2024-06-06T04:28:05.719883+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:54:52.034711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725326+0000","last_change":"2024-06-06T04:27:19.305000+0000","last_active":"2024-06-06T04:28:05.725326+0000","last_peered":"2024-06-06T04:28:05.725326+0000","last_clean":"2024-06-06T04:28:05.725326+0000","last_became_active":"2024-06-06T04:27:19.304676+0000","last_became_peered":"2024-06-06T04:27:19.304676+0000","last_unstale":"2024-06-06T04:28:05.725326+0000","last_undegraded":"2024-06-06T04:28:05.725326+0000","last_fullsized":"2024-06-06T04:28:05.725326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:19:34.054455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719463+0000","last_change":"2024-06-06T04:27:22.572680+0000","last_active":"2024-06-06T04:28:05.719463+0000","last_peered":"2024-06-06T04:28:05.719463+0000","last_clean":"2024-06-06T04:28:05.719463+0000","last_became_active":"2024-06-06T04:27:22.572348+0000","last_became_peered":"2024-06-06T04:27:22.572348+0000","last_unstale":"2024-06-06T04:28:05.719463+0000","last_undegraded":"2024-06-06T04:28:05.719463+0000","last_fullsized":"2024-06-06T04:28:05.719463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:05:26.215771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719538+0000","last_change":"2024-06-06T04:27:32.611587+0000","last_active":"2024-06-06T04:28:05.719538+0000","last_peered":"2024-06-06T04:28:05.719538+0000","last_clean":"2024-06-06T04:28:05.719538+0000","last_became_active":"2024-06-06T04:27:32.611183+0000","last_became_peered":"2024-06-06T04:27:32.611183+0000","last_unstale":"2024-06-06T04:28:05.719538+0000","last_undegraded":"2024-06-06T04:28:05.719538+0000","last_fullsized":"2024-06-06T04:28:05.719538+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:36.773254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724730+0000","last_change":"2024-06-06T04:27:22.562590+0000","last_active":"2024-06-06T04:28:05.724730+0000","last_peered":"2024-06-06T04:28:05.724730+0000","last_clean":"2024-06-06T04:28:05.724730+0000","last_became_active":"2024-06-06T04:27:22.560542+0000","last_became_peered":"2024-06-06T04:27:22.560542+0000","last_unstale":"2024-06-06T04:28:05.724730+0000","last_undegraded":"2024-06-06T04:28:05.724730+0000","last_fullsized":"2024-06-06T04:28:05.724730+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:19:20.651098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":68,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:51.764995+0000","last_change":"2024-06-06T04:27:19.516361+0000","last_active":"2024-06-06T04:28:51.764995+0000","last_peered":"2024-06-06T04:28:51.764995+0000","last_clean":"2024-06-06T04:28:51.764995+0000","last_became_active":"2024-06-06T04:27:19.515973+0000","last_became_peered":"2024-06-06T04:27:19.515973+0000","last_unstale":"2024-06-06T04:28:51.764995+0000","last_undegraded":"2024-06-06T04:28:51.764995+0000","last_fullsized":"2024-06-06T04:28:51.764995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:37.298819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725846+0000","last_change":"2024-06-06T04:27:32.630528+0000","last_active":"2024-06-06T04:28:05.725846+0000","last_peered":"2024-06-06T04:28:05.725846+0000","last_clean":"2024-06-06T04:28:05.725846+0000","last_became_active":"2024-06-06T04:27:32.630215+0000","last_became_peered":"2024-06-06T04:27:32.630215+0000","last_unstale":"2024-06-06T04:28:05.725846+0000","last_undegraded":"2024-06-06T04:28:05.725846+0000","last_fullsized":"2024-06-06T04:28:05.725846+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:34:29.980193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717038+0000","last_change":"2024-06-06T04:27:22.567466+0000","last_active":"2024-06-06T04:28:05.717038+0000","last_peered":"2024-06-06T04:28:05.717038+0000","last_clean":"2024-06-06T04:28:05.717038+0000","last_became_active":"2024-06-06T04:27:22.567145+0000","last_became_peered":"2024-06-06T04:27:22.567145+0000","last_unstale":"2024-06-06T04:28:05.717038+0000","last_undegraded":"2024-06-06T04:28:05.717038+0000","last_fullsized":"2024-06-06T04:28:05.717038+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:56:56.203027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:29.730556+0000","last_change":"2024-06-06T04:27:19.302726+0000","last_active":"2024-06-06T04:28:29.730556+0000","last_peered":"2024-06-06T04:28:29.730556+0000","last_clean":"2024-06-06T04:28:29.730556+0000","last_became_active":"2024-06-06T04:27:19.302329+0000","last_became_peered":"2024-06-06T04:27:19.302329+0000","last_unstale":"2024-06-06T04:28:29.730556+0000","last_undegraded":"2024-06-06T04:28:29.730556+0000","last_fullsized":"2024-06-06T04:28:29.730556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:55:15.010632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":362,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724028+0000","last_change":"2024-06-06T04:25:53.854188+0000","last_active":"2024-06-06T04:28:05.724028+0000","last_peered":"2024-06-06T04:28:05.724028+0000","last_clean":"2024-06-06T04:28:05.724028+0000","last_became_active":"2024-06-06T04:25:53.853700+0000","last_became_peered":"2024-06-06T04:25:53.853700+0000","last_unstale":"2024-06-06T04:28:05.724028+0000","last_undegraded":"2024-06-06T04:28:05.724028+0000","last_fullsized":"2024-06-06T04:28:05.724028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:22:28.847333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:22:28.847333+0000","last_clean_scrub_stamp":"2024-06-06T04:22:28.847333+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:59:10.246373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.727431+0000","last_change":"2024-06-06T04:27:32.623300+0000","last_active":"2024-06-06T04:28:05.727431+0000","last_peered":"2024-06-06T04:28:05.727431+0000","last_clean":"2024-06-06T04:28:05.727431+0000","last_became_active":"2024-06-06T04:27:32.622811+0000","last_became_peered":"2024-06-06T04:27:32.622811+0000","last_unstale":"2024-06-06T04:28:05.727431+0000","last_undegraded":"2024-06-06T04:28:05.727431+0000","last_fullsized":"2024-06-06T04:28:05.727431+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:28.383079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719028+0000","last_change":"2024-06-06T04:27:22.566617+0000","last_active":"2024-06-06T04:28:05.719028+0000","last_peered":"2024-06-06T04:28:05.719028+0000","last_clean":"2024-06-06T04:28:05.719028+0000","last_became_active":"2024-06-06T04:27:22.566256+0000","last_became_peered":"2024-06-06T04:27:22.566256+0000","last_unstale":"2024-06-06T04:28:05.719028+0000","last_undegraded":"2024-06-06T04:28:05.719028+0000","last_fullsized":"2024-06-06T04:28:05.719028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:50:41.318475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:27.720308+0000","last_change":"2024-06-06T04:27:19.305941+0000","last_active":"2024-06-06T04:28:27.720308+0000","last_peered":"2024-06-06T04:28:27.720308+0000","last_clean":"2024-06-06T04:28:27.720308+0000","last_became_active":"2024-06-06T04:27:19.305037+0000","last_became_peered":"2024-06-06T04:27:19.305037+0000","last_unstale":"2024-06-06T04:28:27.720308+0000","last_undegraded":"2024-06-06T04:28:27.720308+0000","last_fullsized":"2024-06-06T04:28:27.720308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:54.084427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720749+0000","last_change":"2024-06-06T04:27:32.621182+0000","last_active":"2024-06-06T04:28:05.720749+0000","last_peered":"2024-06-06T04:28:05.720749+0000","last_clean":"2024-06-06T04:28:05.720749+0000","last_became_active":"2024-06-06T04:27:32.620710+0000","last_became_peered":"2024-06-06T04:27:32.620710+0000","last_unstale":"2024-06-06T04:28:05.720749+0000","last_undegraded":"2024-06-06T04:28:05.720749+0000","last_fullsized":"2024-06-06T04:28:05.720749+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:05.512309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:26.713063+0000","last_change":"2024-06-06T04:27:19.517033+0000","last_active":"2024-06-06T04:28:26.713063+0000","last_peered":"2024-06-06T04:28:26.713063+0000","last_clean":"2024-06-06T04:28:26.713063+0000","last_became_active":"2024-06-06T04:27:19.516278+0000","last_became_peered":"2024-06-06T04:27:19.516278+0000","last_unstale":"2024-06-06T04:28:26.713063+0000","last_undegraded":"2024-06-06T04:28:26.713063+0000","last_fullsized":"2024-06-06T04:28:26.713063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:11:20.336818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719542+0000","last_change":"2024-06-06T04:27:22.566261+0000","last_active":"2024-06-06T04:28:05.719542+0000","last_peered":"2024-06-06T04:28:05.719542+0000","last_clean":"2024-06-06T04:28:05.719542+0000","last_became_active":"2024-06-06T04:27:22.565875+0000","last_became_peered":"2024-06-06T04:27:22.565875+0000","last_unstale":"2024-06-06T04:28:05.719542+0000","last_undegraded":"2024-06-06T04:28:05.719542+0000","last_fullsized":"2024-06-06T04:28:05.719542+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:49:55.923950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.716781+0000","last_change":"2024-06-06T04:27:32.619854+0000","last_active":"2024-06-06T04:28:05.716781+0000","last_peered":"2024-06-06T04:28:05.716781+0000","last_clean":"2024-06-06T04:28:05.716781+0000","last_became_active":"2024-06-06T04:27:32.618765+0000","last_became_peered":"2024-06-06T04:27:32.618765+0000","last_unstale":"2024-06-06T04:28:05.716781+0000","last_undegraded":"2024-06-06T04:28:05.716781+0000","last_fullsized":"2024-06-06T04:28:05.716781+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:47:05.283751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'11","reported_seq":51,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:49.582146+0000","last_change":"2024-06-06T04:27:19.303057+0000","last_active":"2024-06-06T04:28:49.582146+0000","last_peered":"2024-06-06T04:28:49.582146+0000","last_clean":"2024-06-06T04:28:49.582146+0000","last_became_active":"2024-06-06T04:27:19.302716+0000","last_became_peered":"2024-06-06T04:27:19.302716+0000","last_unstale":"2024-06-06T04:28:49.582146+0000","last_undegraded":"2024-06-06T04:28:49.582146+0000","last_fullsized":"2024-06-06T04:28:49.582146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:35.273663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717447+0000","last_change":"2024-06-06T04:27:22.575274+0000","last_active":"2024-06-06T04:28:05.717447+0000","last_peered":"2024-06-06T04:28:05.717447+0000","last_clean":"2024-06-06T04:28:05.717447+0000","last_became_active":"2024-06-06T04:27:22.574979+0000","last_became_peered":"2024-06-06T04:27:22.574979+0000","last_unstale":"2024-06-06T04:28:05.717447+0000","last_undegraded":"2024-06-06T04:28:05.717447+0000","last_fullsized":"2024-06-06T04:28:05.717447+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:30.718398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717553+0000","last_change":"2024-06-06T04:27:32.619939+0000","last_active":"2024-06-06T04:28:05.717553+0000","last_peered":"2024-06-06T04:28:05.717553+0000","last_clean":"2024-06-06T04:28:05.717553+0000","last_became_active":"2024-06-06T04:27:32.619499+0000","last_became_peered":"2024-06-06T04:27:32.619499+0000","last_unstale":"2024-06-06T04:28:05.717553+0000","last_undegraded":"2024-06-06T04:28:05.717553+0000","last_fullsized":"2024-06-06T04:28:05.717553+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:41:13.933546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:26.712286+0000","last_change":"2024-06-06T04:27:19.397552+0000","last_active":"2024-06-06T04:28:26.712286+0000","last_peered":"2024-06-06T04:28:26.712286+0000","last_clean":"2024-06-06T04:28:26.712286+0000","last_became_active":"2024-06-06T04:27:19.397186+0000","last_became_peered":"2024-06-06T04:27:19.397186+0000","last_unstale":"2024-06-06T04:28:26.712286+0000","last_undegraded":"2024-06-06T04:28:26.712286+0000","last_fullsized":"2024-06-06T04:28:26.712286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:24:13.403860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.728212+0000","last_change":"2024-06-06T04:28:05.728212+0000","last_active":"2024-06-06T04:28:05.728212+0000","last_peered":"2024-06-06T04:28:05.728212+0000","last_clean":"2024-06-06T04:28:05.728212+0000","last_became_active":"2024-06-06T04:28:05.727748+0000","last_became_peered":"2024-06-06T04:28:05.727748+0000","last_unstale":"2024-06-06T04:28:05.728212+0000","last_undegraded":"2024-06-06T04:28:05.728212+0000","last_fullsized":"2024-06-06T04:28:05.728212+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:50.583716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.896480+0000","last_change":"2024-06-06T04:28:05.896480+0000","last_active":"2024-06-06T04:28:05.896480+0000","last_peered":"2024-06-06T04:28:05.896480+0000","last_clean":"2024-06-06T04:28:05.896480+0000","last_became_active":"2024-06-06T04:28:05.895967+0000","last_became_peered":"2024-06-06T04:28:05.895967+0000","last_unstale":"2024-06-06T04:28:05.896480+0000","last_undegraded":"2024-06-06T04:28:05.896480+0000","last_fullsized":"2024-06-06T04:28:05.896480+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:26:19.846884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:26.713951+0000","last_change":"2024-06-06T04:27:19.516515+0000","last_active":"2024-06-06T04:28:26.713951+0000","last_peered":"2024-06-06T04:28:26.713951+0000","last_clean":"2024-06-06T04:28:26.713951+0000","last_became_active":"2024-06-06T04:27:19.516087+0000","last_became_peered":"2024-06-06T04:27:19.516087+0000","last_unstale":"2024-06-06T04:28:26.713951+0000","last_undegraded":"2024-06-06T04:28:26.713951+0000","last_fullsized":"2024-06-06T04:28:26.713951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:58.440226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719824+0000","last_change":"2024-06-06T04:27:22.562367+0000","last_active":"2024-06-06T04:28:05.719824+0000","last_peered":"2024-06-06T04:28:05.719824+0000","last_clean":"2024-06-06T04:28:05.719824+0000","last_became_active":"2024-06-06T04:27:22.561947+0000","last_became_peered":"2024-06-06T04:27:22.561947+0000","last_unstale":"2024-06-06T04:28:05.719824+0000","last_undegraded":"2024-06-06T04:28:05.719824+0000","last_fullsized":"2024-06-06T04:28:05.719824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:28:06.200748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.895428+0000","last_change":"2024-06-06T04:27:32.631577+0000","last_active":"2024-06-06T04:28:05.895428+0000","last_peered":"2024-06-06T04:28:05.895428+0000","last_clean":"2024-06-06T04:28:05.895428+0000","last_became_active":"2024-06-06T04:27:32.629555+0000","last_became_peered":"2024-06-06T04:27:32.629555+0000","last_unstale":"2024-06-06T04:28:05.895428+0000","last_undegraded":"2024-06-06T04:28:05.895428+0000","last_fullsized":"2024-06-06T04:28:05.895428+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:52:56.197281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:26.713066+0000","last_change":"2024-06-06T04:27:19.298880+0000","last_active":"2024-06-06T04:28:26.713066+0000","last_peered":"2024-06-06T04:28:26.713066+0000","last_clean":"2024-06-06T04:28:26.713066+0000","last_became_active":"2024-06-06T04:27:19.298551+0000","last_became_peered":"2024-06-06T04:27:19.298551+0000","last_unstale":"2024-06-06T04:28:26.713066+0000","last_undegraded":"2024-06-06T04:28:26.713066+0000","last_fullsized":"2024-06-06T04:28:26.713066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:11:32.145401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720444+0000","last_change":"2024-06-06T04:27:22.575065+0000","last_active":"2024-06-06T04:28:05.720444+0000","last_peered":"2024-06-06T04:28:05.720444+0000","last_clean":"2024-06-06T04:28:05.720444+0000","last_became_active":"2024-06-06T04:27:22.574469+0000","last_became_peered":"2024-06-06T04:27:22.574469+0000","last_unstale":"2024-06-06T04:28:05.720444+0000","last_undegraded":"2024-06-06T04:28:05.720444+0000","last_fullsized":"2024-06-06T04:28:05.720444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:29:44.943833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717731+0000","last_change":"2024-06-06T04:27:32.626678+0000","last_active":"2024-06-06T04:28:05.717731+0000","last_peered":"2024-06-06T04:28:05.717731+0000","last_clean":"2024-06-06T04:28:05.717731+0000","last_became_active":"2024-06-06T04:27:32.626167+0000","last_became_peered":"2024-06-06T04:27:32.626167+0000","last_unstale":"2024-06-06T04:28:05.717731+0000","last_undegraded":"2024-06-06T04:28:05.717731+0000","last_fullsized":"2024-06-06T04:28:05.717731+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:56:26.363195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717922+0000","last_change":"2024-06-06T04:27:19.306466+0000","last_active":"2024-06-06T04:28:05.717922+0000","last_peered":"2024-06-06T04:28:05.717922+0000","last_clean":"2024-06-06T04:28:05.717922+0000","last_became_active":"2024-06-06T04:27:19.303821+0000","last_became_peered":"2024-06-06T04:27:19.303821+0000","last_unstale":"2024-06-06T04:28:05.717922+0000","last_undegraded":"2024-06-06T04:28:05.717922+0000","last_fullsized":"2024-06-06T04:28:05.717922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:23:53.593998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717221+0000","last_change":"2024-06-06T04:27:22.562662+0000","last_active":"2024-06-06T04:28:05.717221+0000","last_peered":"2024-06-06T04:28:05.717221+0000","last_clean":"2024-06-06T04:28:05.717221+0000","last_became_active":"2024-06-06T04:27:22.562318+0000","last_became_peered":"2024-06-06T04:27:22.562318+0000","last_unstale":"2024-06-06T04:28:05.717221+0000","last_undegraded":"2024-06-06T04:28:05.717221+0000","last_fullsized":"2024-06-06T04:28:05.717221+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:31:26.639275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717596+0000","last_change":"2024-06-06T04:27:32.619970+0000","last_active":"2024-06-06T04:28:05.717596+0000","last_peered":"2024-06-06T04:28:05.717596+0000","last_clean":"2024-06-06T04:28:05.717596+0000","last_became_active":"2024-06-06T04:27:32.619464+0000","last_became_peered":"2024-06-06T04:27:32.619464+0000","last_unstale":"2024-06-06T04:28:05.717596+0000","last_undegraded":"2024-06-06T04:28:05.717596+0000","last_fullsized":"2024-06-06T04:28:05.717596+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:44.844018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:26.712874+0000","last_change":"2024-06-06T04:27:19.308172+0000","last_active":"2024-06-06T04:28:26.712874+0000","last_peered":"2024-06-06T04:28:26.712874+0000","last_clean":"2024-06-06T04:28:26.712874+0000","last_became_active":"2024-06-06T04:27:19.307780+0000","last_became_peered":"2024-06-06T04:27:19.307780+0000","last_unstale":"2024-06-06T04:28:26.712874+0000","last_undegraded":"2024-06-06T04:28:26.712874+0000","last_fullsized":"2024-06-06T04:28:26.712874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:39:11.653649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725426+0000","last_change":"2024-06-06T04:27:22.568881+0000","last_active":"2024-06-06T04:28:05.725426+0000","last_peered":"2024-06-06T04:28:05.725426+0000","last_clean":"2024-06-06T04:28:05.725426+0000","last_became_active":"2024-06-06T04:27:22.568489+0000","last_became_peered":"2024-06-06T04:27:22.568489+0000","last_unstale":"2024-06-06T04:28:05.725426+0000","last_undegraded":"2024-06-06T04:28:05.725426+0000","last_fullsized":"2024-06-06T04:28:05.725426+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:00:22.339313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720343+0000","last_change":"2024-06-06T04:27:32.622357+0000","last_active":"2024-06-06T04:28:05.720343+0000","last_peered":"2024-06-06T04:28:05.720343+0000","last_clean":"2024-06-06T04:28:05.720343+0000","last_became_active":"2024-06-06T04:27:32.621680+0000","last_became_peered":"2024-06-06T04:27:32.621680+0000","last_unstale":"2024-06-06T04:28:05.720343+0000","last_undegraded":"2024-06-06T04:28:05.720343+0000","last_fullsized":"2024-06-06T04:28:05.720343+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:00.677721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:27.719936+0000","last_change":"2024-06-06T04:27:19.516890+0000","last_active":"2024-06-06T04:28:27.719936+0000","last_peered":"2024-06-06T04:28:27.719936+0000","last_clean":"2024-06-06T04:28:27.719936+0000","last_became_active":"2024-06-06T04:27:19.516490+0000","last_became_peered":"2024-06-06T04:27:19.516490+0000","last_unstale":"2024-06-06T04:28:27.719936+0000","last_undegraded":"2024-06-06T04:28:27.719936+0000","last_fullsized":"2024-06-06T04:28:27.719936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:11:59.336250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725945+0000","last_change":"2024-06-06T04:27:22.571003+0000","last_active":"2024-06-06T04:28:05.725945+0000","last_peered":"2024-06-06T04:28:05.725945+0000","last_clean":"2024-06-06T04:28:05.725945+0000","last_became_active":"2024-06-06T04:27:22.570442+0000","last_became_peered":"2024-06-06T04:27:22.570442+0000","last_unstale":"2024-06-06T04:28:05.725945+0000","last_undegraded":"2024-06-06T04:28:05.725945+0000","last_fullsized":"2024-06-06T04:28:05.725945+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:28:26.870429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.721107+0000","last_change":"2024-06-06T04:27:32.624424+0000","last_active":"2024-06-06T04:28:05.721107+0000","last_peered":"2024-06-06T04:28:05.721107+0000","last_clean":"2024-06-06T04:28:05.721107+0000","last_became_active":"2024-06-06T04:27:32.624026+0000","last_became_peered":"2024-06-06T04:27:32.624026+0000","last_unstale":"2024-06-06T04:28:05.721107+0000","last_undegraded":"2024-06-06T04:28:05.721107+0000","last_fullsized":"2024-06-06T04:28:05.721107+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:30:08.814284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.726869+0000","last_change":"2024-06-06T04:27:19.301382+0000","last_active":"2024-06-06T04:28:05.726869+0000","last_peered":"2024-06-06T04:28:05.726869+0000","last_clean":"2024-06-06T04:28:05.726869+0000","last_became_active":"2024-06-06T04:27:19.300882+0000","last_became_peered":"2024-06-06T04:27:19.300882+0000","last_unstale":"2024-06-06T04:28:05.726869+0000","last_undegraded":"2024-06-06T04:28:05.726869+0000","last_fullsized":"2024-06-06T04:28:05.726869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:50.514103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.725239+0000","last_change":"2024-06-06T04:27:22.570661+0000","last_active":"2024-06-06T04:28:05.725239+0000","last_peered":"2024-06-06T04:28:05.725239+0000","last_clean":"2024-06-06T04:28:05.725239+0000","last_became_active":"2024-06-06T04:27:22.570287+0000","last_became_peered":"2024-06-06T04:27:22.570287+0000","last_unstale":"2024-06-06T04:28:05.725239+0000","last_undegraded":"2024-06-06T04:28:05.725239+0000","last_fullsized":"2024-06-06T04:28:05.725239+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:48.790492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724347+0000","last_change":"2024-06-06T04:27:32.618385+0000","last_active":"2024-06-06T04:28:05.724347+0000","last_peered":"2024-06-06T04:28:05.724347+0000","last_clean":"2024-06-06T04:28:05.724347+0000","last_became_active":"2024-06-06T04:27:32.617956+0000","last_became_peered":"2024-06-06T04:27:32.617956+0000","last_unstale":"2024-06-06T04:28:05.724347+0000","last_undegraded":"2024-06-06T04:28:05.724347+0000","last_fullsized":"2024-06-06T04:28:05.724347+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:26.308657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":52,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.728633+0000","last_change":"2024-06-06T04:28:05.728633+0000","last_active":"2024-06-06T04:28:05.728633+0000","last_peered":"2024-06-06T04:28:05.728633+0000","last_clean":"2024-06-06T04:28:05.728633+0000","last_became_active":"2024-06-06T04:28:05.722427+0000","last_became_peered":"2024-06-06T04:28:05.722427+0000","last_unstale":"2024-06-06T04:28:05.728633+0000","last_undegraded":"2024-06-06T04:28:05.728633+0000","last_fullsized":"2024-06-06T04:28:05.728633+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:13:22.061798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.726020+0000","last_change":"2024-06-06T04:27:22.574950+0000","last_active":"2024-06-06T04:28:05.726020+0000","last_peered":"2024-06-06T04:28:05.726020+0000","last_clean":"2024-06-06T04:28:05.726020+0000","last_became_active":"2024-06-06T04:27:22.574604+0000","last_became_peered":"2024-06-06T04:27:22.574604+0000","last_unstale":"2024-06-06T04:28:05.726020+0000","last_undegraded":"2024-06-06T04:28:05.726020+0000","last_fullsized":"2024-06-06T04:28:05.726020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:17:29.395137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720585+0000","last_change":"2024-06-06T04:27:22.568554+0000","last_active":"2024-06-06T04:28:05.720585+0000","last_peered":"2024-06-06T04:28:05.720585+0000","last_clean":"2024-06-06T04:28:05.720585+0000","last_became_active":"2024-06-06T04:27:22.568196+0000","last_became_peered":"2024-06-06T04:27:22.568196+0000","last_unstale":"2024-06-06T04:28:05.720585+0000","last_undegraded":"2024-06-06T04:28:05.720585+0000","last_fullsized":"2024-06-06T04:28:05.720585+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:11.564067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.718007+0000","last_change":"2024-06-06T04:27:22.565472+0000","last_active":"2024-06-06T04:28:05.718007+0000","last_peered":"2024-06-06T04:28:05.718007+0000","last_clean":"2024-06-06T04:28:05.718007+0000","last_became_active":"2024-06-06T04:27:22.565116+0000","last_became_peered":"2024-06-06T04:27:22.565116+0000","last_unstale":"2024-06-06T04:28:05.718007+0000","last_undegraded":"2024-06-06T04:28:05.718007+0000","last_fullsized":"2024-06-06T04:28:05.718007+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:03:58.232707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717634+0000","last_change":"2024-06-06T04:27:22.560210+0000","last_active":"2024-06-06T04:28:05.717634+0000","last_peered":"2024-06-06T04:28:05.717634+0000","last_clean":"2024-06-06T04:28:05.717634+0000","last_became_active":"2024-06-06T04:27:22.559853+0000","last_became_peered":"2024-06-06T04:27:22.559853+0000","last_unstale":"2024-06-06T04:28:05.717634+0000","last_undegraded":"2024-06-06T04:28:05.717634+0000","last_fullsized":"2024-06-06T04:28:05.717634+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:56:18.650427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.721312+0000","last_change":"2024-06-06T04:27:22.565071+0000","last_active":"2024-06-06T04:28:05.721312+0000","last_peered":"2024-06-06T04:28:05.721312+0000","last_clean":"2024-06-06T04:28:05.721312+0000","last_became_active":"2024-06-06T04:27:22.561411+0000","last_became_peered":"2024-06-06T04:27:22.561411+0000","last_unstale":"2024-06-06T04:28:05.721312+0000","last_undegraded":"2024-06-06T04:28:05.721312+0000","last_fullsized":"2024-06-06T04:28:05.721312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:41.948481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719436+0000","last_change":"2024-06-06T04:27:22.563280+0000","last_active":"2024-06-06T04:28:05.719436+0000","last_peered":"2024-06-06T04:28:05.719436+0000","last_clean":"2024-06-06T04:28:05.719436+0000","last_became_active":"2024-06-06T04:27:22.563049+0000","last_became_peered":"2024-06-06T04:27:22.563049+0000","last_unstale":"2024-06-06T04:28:05.719436+0000","last_undegraded":"2024-06-06T04:28:05.719436+0000","last_fullsized":"2024-06-06T04:28:05.719436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:41:48.287969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.721794+0000","last_change":"2024-06-06T04:27:22.567691+0000","last_active":"2024-06-06T04:28:05.721794+0000","last_peered":"2024-06-06T04:28:05.721794+0000","last_clean":"2024-06-06T04:28:05.721794+0000","last_became_active":"2024-06-06T04:27:22.567331+0000","last_became_peered":"2024-06-06T04:27:22.567331+0000","last_unstale":"2024-06-06T04:28:05.721794+0000","last_undegraded":"2024-06-06T04:28:05.721794+0000","last_fullsized":"2024-06-06T04:28:05.721794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:03.886084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.718283+0000","last_change":"2024-06-06T04:27:22.563853+0000","last_active":"2024-06-06T04:28:05.718283+0000","last_peered":"2024-06-06T04:28:05.718283+0000","last_clean":"2024-06-06T04:28:05.718283+0000","last_became_active":"2024-06-06T04:27:22.563436+0000","last_became_peered":"2024-06-06T04:27:22.563436+0000","last_unstale":"2024-06-06T04:28:05.718283+0000","last_undegraded":"2024-06-06T04:28:05.718283+0000","last_fullsized":"2024-06-06T04:28:05.718283+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:26.201308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.717559+0000","last_change":"2024-06-06T04:27:22.568189+0000","last_active":"2024-06-06T04:28:05.717559+0000","last_peered":"2024-06-06T04:28:05.717559+0000","last_clean":"2024-06-06T04:28:05.717559+0000","last_became_active":"2024-06-06T04:27:22.567815+0000","last_became_peered":"2024-06-06T04:27:22.567815+0000","last_unstale":"2024-06-06T04:28:05.717559+0000","last_undegraded":"2024-06-06T04:28:05.717559+0000","last_fullsized":"2024-06-06T04:28:05.717559+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:13:39.259012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.724663+0000","last_change":"2024-06-06T04:27:22.569698+0000","last_active":"2024-06-06T04:28:05.724663+0000","last_peered":"2024-06-06T04:28:05.724663+0000","last_clean":"2024-06-06T04:28:05.724663+0000","last_became_active":"2024-06-06T04:27:22.569347+0000","last_became_peered":"2024-06-06T04:27:22.569347+0000","last_unstale":"2024-06-06T04:28:05.724663+0000","last_undegraded":"2024-06-06T04:28:05.724663+0000","last_fullsized":"2024-06-06T04:28:05.724663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:06:34.722798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.716981+0000","last_change":"2024-06-06T04:27:22.561149+0000","last_active":"2024-06-06T04:28:05.716981+0000","last_peered":"2024-06-06T04:28:05.716981+0000","last_clean":"2024-06-06T04:28:05.716981+0000","last_became_active":"2024-06-06T04:27:22.560793+0000","last_became_peered":"2024-06-06T04:27:22.560793+0000","last_unstale":"2024-06-06T04:28:05.716981+0000","last_undegraded":"2024-06-06T04:28:05.716981+0000","last_fullsized":"2024-06-06T04:28:05.716981+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:00:41.563443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.720263+0000","last_change":"2024-06-06T04:27:22.569331+0000","last_active":"2024-06-06T04:28:05.720263+0000","last_peered":"2024-06-06T04:28:05.720263+0000","last_clean":"2024-06-06T04:28:05.720263+0000","last_became_active":"2024-06-06T04:27:22.569021+0000","last_became_peered":"2024-06-06T04:27:22.569021+0000","last_unstale":"2024-06-06T04:28:05.720263+0000","last_undegraded":"2024-06-06T04:28:05.720263+0000","last_fullsized":"2024-06-06T04:28:05.720263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:52:37.064317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.719495+0000","last_change":"2024-06-06T04:27:22.574558+0000","last_active":"2024-06-06T04:28:05.719495+0000","last_peered":"2024-06-06T04:28:05.719495+0000","last_clean":"2024-06-06T04:28:05.719495+0000","last_became_active":"2024-06-06T04:27:22.574215+0000","last_became_peered":"2024-06-06T04:27:22.574215+0000","last_unstale":"2024-06-06T04:28:05.719495+0000","last_undegraded":"2024-06-06T04:28:05.719495+0000","last_fullsized":"2024-06-06T04:28:05.719495+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:40:52.496322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T04:28:05.722077+0000","last_change":"2024-06-06T04:27:22.565320+0000","last_active":"2024-06-06T04:28:05.722077+0000","last_peered":"2024-06-06T04:28:05.722077+0000","last_clean":"2024-06-06T04:28:05.722077+0000","last_became_active":"2024-06-06T04:27:22.564786+0000","last_became_peered":"2024-06-06T04:27:22.564786+0000","last_unstale":"2024-06-06T04:28:05.722077+0000","last_undegraded":"2024-06-06T04:28:05.722077+0000","last_fullsized":"2024-06-06T04:28:05.722077+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:19:42.057666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249851,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66964,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":187,"num_read_kb":163,"num_write":168,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":194086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351011,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":553987,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612648,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28212,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716940,"statfs":{"total":95995035648,"available":95966146560,"internally_reserved":0,"allocated":1224704,"data_stored":1204609,"data_compressed":62660,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776173,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":558319,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":554143,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299446,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":1150051,"data_compressed":62660,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462972,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1073152,"data_stored":549399,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":605467,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":550787,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":613623,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":549593,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":554439,"data_compressed":26499,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724609,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":1145249,"data_compressed":62660,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34523,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34523,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T04:28:59.735 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-06T04:28:59.736 INFO:tasks.cephadm:Setting up client nodes... 2024-06-06T04:28:59.736 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-06T04:28:59.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:59 smithi045 ceph-mon[32299]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-06-06T04:28:59.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:28:59 smithi045 ceph-mon[32299]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:28:59.994 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:00.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:59 smithi154 ceph-mon[36428]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-06-06T04:29:00.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:28:59 smithi154 ceph-mon[36428]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:29:00.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:59 smithi086 ceph-mon[36053]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-06-06T04:29:00.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:28:59 smithi086 ceph-mon[36053]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:29:01.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.519+0000 7f22a11be700 1 -- 172.21.15.45:0/2823846563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f229c102fb0 msgr2=0x7f229c109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:01.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.519+0000 7f22a11be700 1 --2- 172.21.15.45:0/2823846563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f229c102fb0 0x7f229c109860 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f229400b0d0 tx=0x7f229400b3e0 comp rx=0 tx=0).stop 2024-06-06T04:29:01.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.519+0000 7f22a11be700 1 -- 172.21.15.45:0/2823846563 shutdown_connections 2024-06-06T04:29:01.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.519+0000 7f22a11be700 1 --2- 172.21.15.45:0/2823846563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f229c102fb0 0x7f229c109860 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:01.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.519+0000 7f22a11be700 1 --2- 172.21.15.45:0/2823846563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f229c101410 0x7f229c101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:01.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.519+0000 7f22a11be700 1 --2- 172.21.15.45:0/2823846563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f229c102610 0x7f229c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:01.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.519+0000 7f22a11be700 1 -- 172.21.15.45:0/2823846563 >> 172.21.15.45:0/2823846563 conn(0x7f229c0fc9b0 msgr2=0x7f229c0fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:01.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.519+0000 7f22a11be700 1 -- 172.21.15.45:0/2823846563 shutdown_connections 2024-06-06T04:29:01.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.519+0000 7f22a11be700 1 -- 172.21.15.45:0/2823846563 wait complete. 2024-06-06T04:29:01.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.520+0000 7f22a11be700 1 Processor -- start 2024-06-06T04:29:01.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.520+0000 7f22a11be700 1 -- start start 2024-06-06T04:29:01.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.520+0000 7f22a11be700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f229c101410 0x7f229c06acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:01.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.520+0000 7f22a11be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f229c102610 0x7f229c19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:01.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.520+0000 7f22a11be700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f229c102fb0 0x7f229c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:01.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.520+0000 7f22a11be700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f229c10ce80 con 0x7f229c102610 2024-06-06T04:29:01.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.520+0000 7f22a11be700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f229c10cd00 con 0x7f229c101410 2024-06-06T04:29:01.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.520+0000 7f22a11be700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f229c10d000 con 0x7f229c102fb0 2024-06-06T04:29:01.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.520+0000 7f229a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f229c102610 0x7f229c19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:01.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.520+0000 7f229a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f229c102610 0x7f229c19d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44286/0 (socket says 172.21.15.45:44286) 2024-06-06T04:29:01.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.520+0000 7f229a59c700 1 -- 172.21.15.45:0/1617685805 learned_addr learned my addr 172.21.15.45:0/1617685805 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:01.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.520+0000 7f229ad9d700 1 --2- 172.21.15.45:0/1617685805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f229c101410 0x7f229c06acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:01.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.521+0000 7f229b59e700 1 --2- 172.21.15.45:0/1617685805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f229c102fb0 0x7f229c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:01.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.521+0000 7f229a59c700 1 -- 172.21.15.45:0/1617685805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f229c102fb0 msgr2=0x7f229c19f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:01.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.521+0000 7f229a59c700 1 --2- 172.21.15.45:0/1617685805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f229c102fb0 0x7f229c19f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:01.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.521+0000 7f229a59c700 1 -- 172.21.15.45:0/1617685805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f229c101410 msgr2=0x7f229c06acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:01.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.521+0000 7f229a59c700 1 --2- 172.21.15.45:0/1617685805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f229c101410 0x7f229c06acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:01.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.521+0000 7f229a59c700 1 -- 172.21.15.45:0/1617685805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c1a0110 con 0x7f229c102610 2024-06-06T04:29:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.521+0000 7f229a59c700 1 --2- 172.21.15.45:0/1617685805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f229c102610 0x7f229c19d1f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f229000c920 tx=0x7f229000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.521+0000 7f2283fff700 1 -- 172.21.15.45:0/1617685805 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290007b00 con 0x7f229c102610 2024-06-06T04:29:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.521+0000 7f2283fff700 1 -- 172.21.15.45:0/1617685805 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2290004ce0 con 0x7f229c102610 2024-06-06T04:29:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.521+0000 7f2283fff700 1 -- 172.21.15.45:0/1617685805 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290005550 con 0x7f229c102610 2024-06-06T04:29:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.521+0000 7f22a11be700 1 -- 172.21.15.45:0/1617685805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229c1a0400 con 0x7f229c102610 2024-06-06T04:29:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.521+0000 7f22a11be700 1 -- 172.21.15.45:0/1617685805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229c1a08f0 con 0x7f229c102610 2024-06-06T04:29:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.522+0000 7f22a11be700 1 -- 172.21.15.45:0/1617685805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229c050a00 con 0x7f229c102610 2024-06-06T04:29:01.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.523+0000 7f2283fff700 1 -- 172.21.15.45:0/1617685805 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2290004750 con 0x7f229c102610 2024-06-06T04:29:01.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.524+0000 7f2283fff700 1 --2- 172.21.15.45:0/1617685805 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f22840a2de0 0x7f22840a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:01.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.524+0000 7f2283fff700 1 -- 172.21.15.45:0/1617685805 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7f22900cf320 con 0x7f229c102610 2024-06-06T04:29:01.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.524+0000 7f229ad9d700 1 --2- 172.21.15.45:0/1617685805 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f22840a2de0 0x7f22840a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:01.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.525+0000 7f229ad9d700 1 --2- 172.21.15.45:0/1617685805 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f22840a2de0 0x7f22840a52a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f229c102470 tx=0x7f228c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:01.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.528+0000 7f2283fff700 1 -- 172.21.15.45:0/1617685805 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2290014030 con 0x7f229c102610 2024-06-06T04:29:01.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.819+0000 7f22a11be700 1 -- 172.21.15.45:0/1617685805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f229c02dd70 con 0x7f229c102610 2024-06-06T04:29:01.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.822+0000 7f2283fff700 1 -- 172.21.15.45:0/1617685805 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f229009bf00 con 0x7f229c102610 2024-06-06T04:29:01.822 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-06-06T04:29:01.822 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQANO2FmvevzMBAAwOpu0YKwmX+EbtuhZpmjww== 2024-06-06T04:29:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.824+0000 7f22a11be700 1 -- 172.21.15.45:0/1617685805 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f22840a2de0 msgr2=0x7f22840a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.824+0000 7f22a11be700 1 --2- 172.21.15.45:0/1617685805 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f22840a2de0 0x7f22840a52a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f229c102470 tx=0x7f228c00a380 comp rx=0 tx=0).stop 2024-06-06T04:29:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.824+0000 7f22a11be700 1 -- 172.21.15.45:0/1617685805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f229c102610 msgr2=0x7f229c19d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.824+0000 7f22a11be700 1 --2- 172.21.15.45:0/1617685805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f229c102610 0x7f229c19d1f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f229000c920 tx=0x7f229000cc30 comp rx=0 tx=0).stop 2024-06-06T04:29:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.824+0000 7f22a11be700 1 -- 172.21.15.45:0/1617685805 shutdown_connections 2024-06-06T04:29:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.824+0000 7f22a11be700 1 --2- 172.21.15.45:0/1617685805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f229c102fb0 0x7f229c19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.824+0000 7f22a11be700 1 --2- 172.21.15.45:0/1617685805 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f22840a2de0 0x7f22840a52a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.824+0000 7f22a11be700 1 --2- 172.21.15.45:0/1617685805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f229c101410 0x7f229c06acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.824+0000 7f22a11be700 1 --2- 172.21.15.45:0/1617685805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f229c102610 0x7f229c19d1f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.824+0000 7f22a11be700 1 -- 172.21.15.45:0/1617685805 >> 172.21.15.45:0/1617685805 conn(0x7f229c0fc9b0 msgr2=0x7f229c1058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.825+0000 7f22a11be700 1 -- 172.21.15.45:0/1617685805 shutdown_connections 2024-06-06T04:29:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:01.825+0000 7f22a11be700 1 -- 172.21.15.45:0/1617685805 wait complete. 2024-06-06T04:29:02.100 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:01 smithi045 ceph-mon[32299]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T04:29:02.100 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:01 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1617685805' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-06T04:29:02.100 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:01 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1617685805' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-06T04:29:02.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:01 smithi154 ceph-mon[36428]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T04:29:02.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:01 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1617685805' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-06T04:29:02.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:01 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1617685805' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-06T04:29:02.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:01 smithi086 ceph-mon[36053]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-06T04:29:02.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:01 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1617685805' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-06T04:29:02.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:01 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1617685805' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-06T04:29:02.523 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:29:02.524 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-06T04:29:02.524 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-06T04:29:02.564 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-06T04:29:02.564 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-06T04:29:02.564 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-06T04:29:02.851 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:04.078 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:03 smithi045 ceph-mon[32299]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T04:29:04.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:03 smithi154 ceph-mon[36428]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T04:29:04.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:03 smithi086 ceph-mon[36053]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T04:29:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.628+0000 7fb45494b700 1 -- 172.21.15.45:0/3577740433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb450101380 msgr2=0x7fb450101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.628+0000 7fb45494b700 1 --2- 172.21.15.45:0/3577740433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb450101380 0x7fb450101800 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb444009990 tx=0x7fb444009ca0 comp rx=0 tx=0).stop 2024-06-06T04:29:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.628+0000 7fb45494b700 1 -- 172.21.15.45:0/3577740433 shutdown_connections 2024-06-06T04:29:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.628+0000 7fb45494b700 1 --2- 172.21.15.45:0/3577740433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb450101d40 0x7fb45010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.628+0000 7fb45494b700 1 --2- 172.21.15.45:0/3577740433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4501073f0 0x7fb4501077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.628+0000 7fb45494b700 1 --2- 172.21.15.45:0/3577740433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb450101380 0x7fb450101800 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:04.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.628+0000 7fb45494b700 1 -- 172.21.15.45:0/3577740433 >> 172.21.15.45:0/3577740433 conn(0x7fb450078300 msgr2=0x7fb450078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:04.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.628+0000 7fb45494b700 1 -- 172.21.15.45:0/3577740433 shutdown_connections 2024-06-06T04:29:04.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.628+0000 7fb45494b700 1 -- 172.21.15.45:0/3577740433 wait complete. 2024-06-06T04:29:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.629+0000 7fb45494b700 1 Processor -- start 2024-06-06T04:29:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.629+0000 7fb45494b700 1 -- start start 2024-06-06T04:29:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.629+0000 7fb45494b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb450101380 0x7fb45010e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.629+0000 7fb45494b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb450101d40 0x7fb45010ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.629+0000 7fb45494b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4501073f0 0x7fb45010f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.629+0000 7fb45494b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb450112680 con 0x7fb4501073f0 2024-06-06T04:29:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.629+0000 7fb45494b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb450112500 con 0x7fb450101d40 2024-06-06T04:29:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.629+0000 7fb45494b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb450112800 con 0x7fb450101380 2024-06-06T04:29:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb44ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4501073f0 0x7fb45010f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb44e59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb450101380 0x7fb45010e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb44ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4501073f0 0x7fb45010f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44300/0 (socket says 172.21.15.45:44300) 2024-06-06T04:29:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb44ed9d700 1 -- 172.21.15.45:0/2209091055 learned_addr learned my addr 172.21.15.45:0/2209091055 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb44dd9b700 1 --2- 172.21.15.45:0/2209091055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb450101d40 0x7fb45010ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:04.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb44ed9d700 1 -- 172.21.15.45:0/2209091055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb450101380 msgr2=0x7fb45010e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb44ed9d700 1 --2- 172.21.15.45:0/2209091055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb450101380 0x7fb45010e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb44ed9d700 1 -- 172.21.15.45:0/2209091055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb450101d40 msgr2=0x7fb45010ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb44ed9d700 1 --2- 172.21.15.45:0/2209091055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb450101d40 0x7fb45010ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb44ed9d700 1 -- 172.21.15.45:0/2209091055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4501a7060 con 0x7fb4501073f0 2024-06-06T04:29:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb44dd9b700 1 --2- 172.21.15.45:0/2209091055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb450101d40 0x7fb45010ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:29:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb44ed9d700 1 --2- 172.21.15.45:0/2209091055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4501073f0 0x7fb45010f440 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb448007ee0 tx=0x7fb44800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb43f7fe700 1 -- 172.21.15.45:0/2209091055 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44800dbd0 con 0x7fb4501073f0 2024-06-06T04:29:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb43f7fe700 1 -- 172.21.15.45:0/2209091055 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb44800dd70 con 0x7fb4501073f0 2024-06-06T04:29:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb43f7fe700 1 -- 172.21.15.45:0/2209091055 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb448005990 con 0x7fb4501073f0 2024-06-06T04:29:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb45494b700 1 -- 172.21.15.45:0/2209091055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4501a7350 con 0x7fb4501073f0 2024-06-06T04:29:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.630+0000 7fb45494b700 1 -- 172.21.15.45:0/2209091055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4501a7890 con 0x7fb4501073f0 2024-06-06T04:29:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.631+0000 7fb45494b700 1 -- 172.21.15.45:0/2209091055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb450069180 con 0x7fb4501073f0 2024-06-06T04:29:04.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.633+0000 7fb43f7fe700 1 -- 172.21.15.45:0/2209091055 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb44801a020 con 0x7fb4501073f0 2024-06-06T04:29:04.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.634+0000 7fb43f7fe700 1 --2- 172.21.15.45:0/2209091055 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4380a2d30 0x7fb4380a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:04.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.634+0000 7fb44e59c700 1 --2- 172.21.15.45:0/2209091055 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4380a2d30 0x7fb4380a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:04.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.634+0000 7fb43f7fe700 1 -- 172.21.15.45:0/2209091055 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9071+0+0 (secure 0 0 0) 0x7fb4480cf2a0 con 0x7fb4501073f0 2024-06-06T04:29:04.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.634+0000 7fb44e59c700 1 --2- 172.21.15.45:0/2209091055 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4380a2d30 0x7fb4380a51f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb440005950 tx=0x7fb44000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:04.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.638+0000 7fb43f7fe700 1 -- 172.21.15.45:0/2209091055 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb448010070 con 0x7fb4501073f0 2024-06-06T04:29:04.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.909+0000 7fb45494b700 1 -- 172.21.15.45:0/2209091055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb45002de60 con 0x7fb4501073f0 2024-06-06T04:29:04.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.914+0000 7fb43f7fe700 1 -- 172.21.15.45:0/2209091055 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263201 (secure 0 0 0) 0x7fb44809bd00 con 0x7fb4501073f0 2024-06-06T04:29:04.918 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:29:04.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.919+0000 7fb45494b700 1 -- 172.21.15.45:0/2209091055 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4380a2d30 msgr2=0x7fb4380a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:04.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.919+0000 7fb45494b700 1 --2- 172.21.15.45:0/2209091055 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4380a2d30 0x7fb4380a51f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb440005950 tx=0x7fb44000b410 comp rx=0 tx=0).stop 2024-06-06T04:29:04.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.919+0000 7fb45494b700 1 -- 172.21.15.45:0/2209091055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4501073f0 msgr2=0x7fb45010f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:04.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.919+0000 7fb45494b700 1 --2- 172.21.15.45:0/2209091055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4501073f0 0x7fb45010f440 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb448007ee0 tx=0x7fb44800d3b0 comp rx=0 tx=0).stop 2024-06-06T04:29:04.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.919+0000 7fb45494b700 1 -- 172.21.15.45:0/2209091055 shutdown_connections 2024-06-06T04:29:04.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.919+0000 7fb45494b700 1 --2- 172.21.15.45:0/2209091055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4501073f0 0x7fb45010f440 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:04.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.919+0000 7fb45494b700 1 --2- 172.21.15.45:0/2209091055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb450101380 0x7fb45010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:04.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.919+0000 7fb45494b700 1 --2- 172.21.15.45:0/2209091055 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4380a2d30 0x7fb4380a51f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:04.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.919+0000 7fb45494b700 1 --2- 172.21.15.45:0/2209091055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb450101d40 0x7fb45010ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:04.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.919+0000 7fb45494b700 1 -- 172.21.15.45:0/2209091055 >> 172.21.15.45:0/2209091055 conn(0x7fb450078300 msgr2=0x7fb4500fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:04.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.919+0000 7fb45494b700 1 -- 172.21.15.45:0/2209091055 shutdown_connections 2024-06-06T04:29:04.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:04.919+0000 7fb45494b700 1 -- 172.21.15.45:0/2209091055 wait complete. 2024-06-06T04:29:05.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-06-06T04:29:05.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-06-06T04:29:05.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-06-06T04:29:05.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-06-06T04:29:05.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-06-06T04:29:05.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-06-06T04:29:05.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-06-06T04:29:05.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-06-06T04:29:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-06-06T04:29:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-06-06T04:29:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-06-06T04:29:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-06-06T04:29:05.694 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1718508550},{"type":"v1","addr":"172.21.15.45:6801","nonce":1718508550}]},"active_addr":"172.21.15.45:6801/1718508550","active_change":"2024-06-06T04:20:04.267379+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.45:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":2850286095}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":1119866721}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":3473588129}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":1520152515}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":3413171054}]}]} 2024-06-06T04:29:05.699 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-06T04:29:05.699 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-06T04:29:05.699 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T04:29:05.956 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:05.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:05 smithi045 ceph-mon[32299]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 853 B/s wr, 6 op/s 2024-06-06T04:29:05.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-06-06T04:29:05.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-06-06T04:29:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-06-06T04:29:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-06-06T04:29:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:05 smithi045 ceph-mon[32299]: osdmap e99: 12 total, 12 up, 12 in 2024-06-06T04:29:05.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:05 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2209091055' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-06T04:29:06.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:05 smithi154 ceph-mon[36428]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 853 B/s wr, 6 op/s 2024-06-06T04:29:06.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-06-06T04:29:06.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-06-06T04:29:06.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-06-06T04:29:06.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-06-06T04:29:06.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:05 smithi154 ceph-mon[36428]: osdmap e99: 12 total, 12 up, 12 in 2024-06-06T04:29:06.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:05 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2209091055' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-06T04:29:06.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:05 smithi086 ceph-mon[36053]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 853 B/s wr, 6 op/s 2024-06-06T04:29:06.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-06-06T04:29:06.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-06-06T04:29:06.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-06-06T04:29:06.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-06-06T04:29:06.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:05 smithi086 ceph-mon[36053]: osdmap e99: 12 total, 12 up, 12 in 2024-06-06T04:29:06.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:05 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2209091055' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-06T04:29:07.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:06 smithi154 ceph-mon[36428]: osdmap e100: 12 total, 12 up, 12 in 2024-06-06T04:29:07.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:06 smithi045 ceph-mon[32299]: osdmap e100: 12 total, 12 up, 12 in 2024-06-06T04:29:07.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:06 smithi086 ceph-mon[36053]: osdmap e100: 12 total, 12 up, 12 in 2024-06-06T04:29:07.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.575+0000 7fe533691700 1 -- 172.21.15.45:0/903274676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe52c102740 msgr2=0x7fe52c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:07.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.575+0000 7fe533691700 1 --2- 172.21.15.45:0/903274676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe52c102740 0x7fe52c102b20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe5280099c0 tx=0x7fe528009cd0 comp rx=0 tx=0).stop 2024-06-06T04:29:07.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.575+0000 7fe533691700 1 -- 172.21.15.45:0/903274676 shutdown_connections 2024-06-06T04:29:07.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.575+0000 7fe533691700 1 --2- 172.21.15.45:0/903274676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe52c107840 0x7fe52c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:07.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.575+0000 7fe533691700 1 --2- 172.21.15.45:0/903274676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe52c102740 0x7fe52c102b20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:07.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.575+0000 7fe533691700 1 --2- 172.21.15.45:0/903274676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe52c1030f0 0x7fe52c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:07.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.575+0000 7fe533691700 1 -- 172.21.15.45:0/903274676 >> 172.21.15.45:0/903274676 conn(0x7fe52c0795f0 msgr2=0x7fe52c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:07.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.575+0000 7fe533691700 1 -- 172.21.15.45:0/903274676 shutdown_connections 2024-06-06T04:29:07.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.575+0000 7fe533691700 1 -- 172.21.15.45:0/903274676 wait complete. 2024-06-06T04:29:07.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe533691700 1 Processor -- start 2024-06-06T04:29:07.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe533691700 1 -- start start 2024-06-06T04:29:07.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe533691700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe52c102740 0x7fe52c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:07.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe533691700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe52c1030f0 0x7fe52c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:07.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe533691700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe52c107840 0x7fe52c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:07.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe533691700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c112630 con 0x7fe52c1030f0 2024-06-06T04:29:07.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe533691700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c1124b0 con 0x7fe52c102740 2024-06-06T04:29:07.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe533691700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c1127b0 con 0x7fe52c107840 2024-06-06T04:29:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe530c2c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe52c1030f0 0x7fe52c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe530c2c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe52c1030f0 0x7fe52c074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44322/0 (socket says 172.21.15.45:44322) 2024-06-06T04:29:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe530c2c700 1 -- 172.21.15.45:0/640274212 learned_addr learned my addr 172.21.15.45:0/640274212 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe531c2e700 1 --2- 172.21.15.45:0/640274212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe52c107840 0x7fe52c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe53142d700 1 --2- 172.21.15.45:0/640274212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe52c102740 0x7fe52c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe530c2c700 1 -- 172.21.15.45:0/640274212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe52c107840 msgr2=0x7fe52c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe530c2c700 1 --2- 172.21.15.45:0/640274212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe52c107840 0x7fe52c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe530c2c700 1 -- 172.21.15.45:0/640274212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe52c102740 msgr2=0x7fe52c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe530c2c700 1 --2- 172.21.15.45:0/640274212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe52c102740 0x7fe52c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.576+0000 7fe530c2c700 1 -- 172.21.15.45:0/640274212 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52c1a7060 con 0x7fe52c1030f0 2024-06-06T04:29:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.577+0000 7fe53142d700 1 --2- 172.21.15.45:0/640274212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe52c102740 0x7fe52c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:29:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.577+0000 7fe530c2c700 1 --2- 172.21.15.45:0/640274212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe52c1030f0 0x7fe52c074cd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00d980 tx=0x7fe51c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.577+0000 7fe5227fc700 1 -- 172.21.15.45:0/640274212 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c017070 con 0x7fe52c1030f0 2024-06-06T04:29:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.577+0000 7fe5227fc700 1 -- 172.21.15.45:0/640274212 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe51c005020 con 0x7fe52c1030f0 2024-06-06T04:29:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.577+0000 7fe533691700 1 -- 172.21.15.45:0/640274212 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52c1a7350 con 0x7fe52c1030f0 2024-06-06T04:29:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.577+0000 7fe5227fc700 1 -- 172.21.15.45:0/640274212 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c009b50 con 0x7fe52c1030f0 2024-06-06T04:29:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.577+0000 7fe533691700 1 -- 172.21.15.45:0/640274212 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52c1a7890 con 0x7fe52c1030f0 2024-06-06T04:29:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.578+0000 7fe533691700 1 -- 172.21.15.45:0/640274212 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe52c10aa80 con 0x7fe52c1030f0 2024-06-06T04:29:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.579+0000 7fe5227fc700 1 -- 172.21.15.45:0/640274212 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe51c009cf0 con 0x7fe52c1030f0 2024-06-06T04:29:07.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.580+0000 7fe5227fc700 1 --2- 172.21.15.45:0/640274212 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5180a2d30 0x7fe5180a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:07.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.580+0000 7fe53142d700 1 --2- 172.21.15.45:0/640274212 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5180a2d30 0x7fe5180a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:07.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.580+0000 7fe5227fc700 1 -- 172.21.15.45:0/640274212 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fe51c0cf830 con 0x7fe52c1030f0 2024-06-06T04:29:07.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.581+0000 7fe53142d700 1 --2- 172.21.15.45:0/640274212 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5180a2d30 0x7fe5180a51f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe528000c00 tx=0x7fe528005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:07.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.584+0000 7fe5227fc700 1 -- 172.21.15.45:0/640274212 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe51c020020 con 0x7fe52c1030f0 2024-06-06T04:29:07.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.791+0000 7fe533691700 1 -- 172.21.15.45:0/640274212 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe52c075c70 con 0x7fe52c1030f0 2024-06-06T04:29:07.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.793+0000 7fe5227fc700 1 -- 172.21.15.45:0/640274212 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24789 (secure 0 0 0) 0x7fe51c09c290 con 0x7fe52c1030f0 2024-06-06T04:29:07.794 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:29:07.794 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":100,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","created":"2024-06-06T04:19:02.626916+0000","modified":"2024-06-06T04:29:05.901370+0000","last_up_change":"2024-06-06T04:26:17.390429+0000","last_in_change":"2024-06-06T04:25:56.938388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T04:22:28.797137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T04:27:17.750625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T04:27:21.486917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T04:27:31.029818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d3cee4a-5025-45b3-b217-082af37e79db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6803","nonce":2111759250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6805","nonce":2111759250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6809","nonce":2111759250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6807","nonce":2111759250}]},"public_addr":"172.21.15.45:6803/2111759250","cluster_addr":"172.21.15.45:6805/2111759250","heartbeat_back_addr":"172.21.15.45:6809/2111759250","heartbeat_front_addr":"172.21.15.45:6807/2111759250","state":["exists","up"]},{"osd":1,"uuid":"75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6801","nonce":3732225287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6803","nonce":3732225287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6807","nonce":3732225287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6805","nonce":3732225287}]},"public_addr":"172.21.15.86:6801/3732225287","cluster_addr":"172.21.15.86:6803/3732225287","heartbeat_back_addr":"172.21.15.86:6807/3732225287","heartbeat_front_addr":"172.21.15.86:6805/3732225287","state":["exists","up"]},{"osd":2,"uuid":"bb20565f-f9ae-47e3-9604-c69f737cff4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6801","nonce":2898783039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6803","nonce":2898783039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6807","nonce":2898783039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6805","nonce":2898783039}]},"public_addr":"172.21.15.154:6801/2898783039","cluster_addr":"172.21.15.154:6803/2898783039","heartbeat_back_addr":"172.21.15.154:6807/2898783039","heartbeat_front_addr":"172.21.15.154:6805/2898783039","state":["exists","up"]},{"osd":3,"uuid":"e1160799-7811-4326-9792-7e6c7d189c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6811","nonce":3436753135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6813","nonce":3436753135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6817","nonce":3436753135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6815","nonce":3436753135}]},"public_addr":"172.21.15.45:6811/3436753135","cluster_addr":"172.21.15.45:6813/3436753135","heartbeat_back_addr":"172.21.15.45:6817/3436753135","heartbeat_front_addr":"172.21.15.45:6815/3436753135","state":["exists","up"]},{"osd":4,"uuid":"6b389bc8-e19a-47f3-83ff-7b782d146d0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6809","nonce":3030235796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6811","nonce":3030235796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6815","nonce":3030235796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6813","nonce":3030235796}]},"public_addr":"172.21.15.86:6809/3030235796","cluster_addr":"172.21.15.86:6811/3030235796","heartbeat_back_addr":"172.21.15.86:6815/3030235796","heartbeat_front_addr":"172.21.15.86:6813/3030235796","state":["exists","up"]},{"osd":5,"uuid":"069de7b2-89bf-4150-bed7-7b8832573dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6809","nonce":2222268646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6811","nonce":2222268646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6815","nonce":2222268646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6813","nonce":2222268646}]},"public_addr":"172.21.15.154:6809/2222268646","cluster_addr":"172.21.15.154:6811/2222268646","heartbeat_back_addr":"172.21.15.154:6815/2222268646","heartbeat_front_addr":"172.21.15.154:6813/2222268646","state":["exists","up"]},{"osd":6,"uuid":"4c70b627-eafa-4e27-ba2a-4d8e1e335962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6819","nonce":384734939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6821","nonce":384734939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6825","nonce":384734939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6823","nonce":384734939}]},"public_addr":"172.21.15.45:6819/384734939","cluster_addr":"172.21.15.45:6821/384734939","heartbeat_back_addr":"172.21.15.45:6825/384734939","heartbeat_front_addr":"172.21.15.45:6823/384734939","state":["exists","up"]},{"osd":7,"uuid":"3b09b875-be52-4be0-b116-7fd9b0c26705","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6817","nonce":1119791005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6819","nonce":1119791005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6823","nonce":1119791005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6821","nonce":1119791005}]},"public_addr":"172.21.15.86:6817/1119791005","cluster_addr":"172.21.15.86:6819/1119791005","heartbeat_back_addr":"172.21.15.86:6823/1119791005","heartbeat_front_addr":"172.21.15.86:6821/1119791005","state":["exists","up"]},{"osd":8,"uuid":"cdf1dd1b-5392-4836-8e65-4c7c5b688d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6817","nonce":3054812496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6819","nonce":3054812496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6823","nonce":3054812496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6821","nonce":3054812496}]},"public_addr":"172.21.15.154:6817/3054812496","cluster_addr":"172.21.15.154:6819/3054812496","heartbeat_back_addr":"172.21.15.154:6823/3054812496","heartbeat_front_addr":"172.21.15.154:6821/3054812496","state":["exists","up"]},{"osd":9,"uuid":"acede92d-2cc1-4967-a357-7e5e201e99d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6827","nonce":1360655510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6829","nonce":1360655510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6833","nonce":1360655510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6831","nonce":1360655510}]},"public_addr":"172.21.15.45:6827/1360655510","cluster_addr":"172.21.15.45:6829/1360655510","heartbeat_back_addr":"172.21.15.45:6833/1360655510","heartbeat_front_addr":"172.21.15.45:6831/1360655510","state":["exists","up"]},{"osd":10,"uuid":"a2d0c488-1892-4da4-adbf-952c22df0ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6825","nonce":1701691877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6827","nonce":1701691877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6831","nonce":1701691877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6829","nonce":1701691877}]},"public_addr":"172.21.15.86:6825/1701691877","cluster_addr":"172.21.15.86:6827/1701691877","heartbeat_back_addr":"172.21.15.86:6831/1701691877","heartbeat_front_addr":"172.21.15.86:6829/1701691877","state":["exists","up"]},{"osd":11,"uuid":"6b349c4c-b672-480b-ac6a-8594dab0eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6825","nonce":2065648007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6827","nonce":2065648007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6831","nonce":2065648007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6829","nonce":2065648007}]},"public_addr":"172.21.15.154:6825/2065648007","cluster_addr":"172.21.15.154:6827/2065648007","heartbeat_back_addr":"172.21.15.154:6831/2065648007","heartbeat_front_addr":"172.21.15.154:6829/2065648007","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:21:37.147394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:00.776617+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:24.786849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:50.487455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:14.869272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:39.815158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:05.972619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:31.410063+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:56.795119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:23.114884+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:49.256504+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:26:15.251967+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/36732849":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/1500318775":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6801/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6800/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/2041380391":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/2504003341":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/3798552599":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/498952822":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6800/1562511056":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6801/1562511056":"2024-06-07T04:19:37.026199+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T04:29:07.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.795+0000 7fe533691700 1 -- 172.21.15.45:0/640274212 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5180a2d30 msgr2=0x7fe5180a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.795+0000 7fe533691700 1 --2- 172.21.15.45:0/640274212 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5180a2d30 0x7fe5180a51f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe528000c00 tx=0x7fe528005fd0 comp rx=0 tx=0).stop 2024-06-06T04:29:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.795+0000 7fe533691700 1 -- 172.21.15.45:0/640274212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe52c1030f0 msgr2=0x7fe52c074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.795+0000 7fe533691700 1 --2- 172.21.15.45:0/640274212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe52c1030f0 0x7fe52c074cd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00d980 tx=0x7fe51c00dc90 comp rx=0 tx=0).stop 2024-06-06T04:29:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.795+0000 7fe533691700 1 -- 172.21.15.45:0/640274212 shutdown_connections 2024-06-06T04:29:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.795+0000 7fe533691700 1 --2- 172.21.15.45:0/640274212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe52c107840 0x7fe52c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.795+0000 7fe533691700 1 --2- 172.21.15.45:0/640274212 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5180a2d30 0x7fe5180a51f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.795+0000 7fe533691700 1 --2- 172.21.15.45:0/640274212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe52c102740 0x7fe52c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.795+0000 7fe533691700 1 --2- 172.21.15.45:0/640274212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe52c1030f0 0x7fe52c074cd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.795+0000 7fe533691700 1 -- 172.21.15.45:0/640274212 >> 172.21.15.45:0/640274212 conn(0x7fe52c0795f0 msgr2=0x7fe52c108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.795+0000 7fe533691700 1 -- 172.21.15.45:0/640274212 shutdown_connections 2024-06-06T04:29:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:07.795+0000 7fe533691700 1 -- 172.21.15.45:0/640274212 wait complete. 2024-06-06T04:29:08.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:07 smithi154 ceph-mon[36428]: pgmap v361: 129 pgs: 1 activating, 128 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:29:08.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:07 smithi154 ceph-mon[36428]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-06-06T04:29:08.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:07 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/640274212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:29:08.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:07 smithi045 ceph-mon[32299]: pgmap v361: 129 pgs: 1 activating, 128 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:29:08.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:07 smithi045 ceph-mon[32299]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-06-06T04:29:08.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:07 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/640274212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:29:08.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:07 smithi086 ceph-mon[36053]: pgmap v361: 129 pgs: 1 activating, 128 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:29:08.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:07 smithi086 ceph-mon[36053]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-06-06T04:29:08.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:07 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/640274212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:29:08.610 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-06T04:29:08.610 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T04:29:08.869 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:09.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:09 smithi045 ceph-mon[32299]: pgmap v362: 129 pgs: 1 activating, 128 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:29:10.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:09 smithi154 ceph-mon[36428]: pgmap v362: 129 pgs: 1 activating, 128 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:29:10.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:09 smithi086 ceph-mon[36053]: pgmap v362: 129 pgs: 1 activating, 128 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:29:10.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.557+0000 7fa8c8dd3700 1 -- 172.21.15.45:0/1959775373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8c406b580 msgr2=0x7fa8c410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:10.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.557+0000 7fa8c8dd3700 1 --2- 172.21.15.45:0/1959775373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8c406b580 0x7fa8c410f320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00b0a0 tx=0x7fa8bc00b3b0 comp rx=0 tx=0).stop 2024-06-06T04:29:10.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.558+0000 7fa8c8dd3700 1 -- 172.21.15.45:0/1959775373 shutdown_connections 2024-06-06T04:29:10.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.558+0000 7fa8c8dd3700 1 --2- 172.21.15.45:0/1959775373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8c406b580 0x7fa8c410f320 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:10.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.558+0000 7fa8c8dd3700 1 --2- 172.21.15.45:0/1959775373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c4108500 0x7fa8c41088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:10.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.558+0000 7fa8c8dd3700 1 --2- 172.21.15.45:0/1959775373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8c406abc0 0x7fa8c406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:10.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.558+0000 7fa8c8dd3700 1 -- 172.21.15.45:0/1959775373 >> 172.21.15.45:0/1959775373 conn(0x7fa8c4078340 msgr2=0x7fa8c4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:10.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.558+0000 7fa8c8dd3700 1 -- 172.21.15.45:0/1959775373 shutdown_connections 2024-06-06T04:29:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.558+0000 7fa8c8dd3700 1 -- 172.21.15.45:0/1959775373 wait complete. 2024-06-06T04:29:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.558+0000 7fa8c8dd3700 1 Processor -- start 2024-06-06T04:29:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c8dd3700 1 -- start start 2024-06-06T04:29:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c8dd3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c406abc0 0x7fa8c4101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c8dd3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8c406b580 0x7fa8c41019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c8dd3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8c4108500 0x7fa8c4101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c8dd3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c4112660 con 0x7fa8c406b580 2024-06-06T04:29:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c8dd3700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41124e0 con 0x7fa8c406abc0 2024-06-06T04:29:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c8dd3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41127e0 con 0x7fa8c4108500 2024-06-06T04:29:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8c406b580 0x7fa8c41019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8c406b580 0x7fa8c41019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54054/0 (socket says 172.21.15.45:54054) 2024-06-06T04:29:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c1d9b700 1 -- 172.21.15.45:0/3890116353 learned_addr learned my addr 172.21.15.45:0/3890116353 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c259c700 1 --2- 172.21.15.45:0/3890116353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c406abc0 0x7fa8c4101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c2d9d700 1 --2- 172.21.15.45:0/3890116353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8c4108500 0x7fa8c4101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c1d9b700 1 -- 172.21.15.45:0/3890116353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8c4108500 msgr2=0x7fa8c4101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c1d9b700 1 --2- 172.21.15.45:0/3890116353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8c4108500 0x7fa8c4101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c1d9b700 1 -- 172.21.15.45:0/3890116353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c406abc0 msgr2=0x7fa8c4101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c1d9b700 1 --2- 172.21.15.45:0/3890116353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c406abc0 0x7fa8c4101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.559+0000 7fa8c1d9b700 1 -- 172.21.15.45:0/3890116353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c41a6f30 con 0x7fa8c406b580 2024-06-06T04:29:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.560+0000 7fa8c1d9b700 1 --2- 172.21.15.45:0/3890116353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8c406b580 0x7fa8c41019d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800c9b0 tx=0x7fa8b800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.560+0000 7fa8b37fe700 1 -- 172.21.15.45:0/3890116353 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8007b00 con 0x7fa8c406b580 2024-06-06T04:29:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.560+0000 7fa8b37fe700 1 -- 172.21.15.45:0/3890116353 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8b800ce50 con 0x7fa8c406b580 2024-06-06T04:29:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.560+0000 7fa8c8dd3700 1 -- 172.21.15.45:0/3890116353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c41a7220 con 0x7fa8c406b580 2024-06-06T04:29:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.560+0000 7fa8b37fe700 1 -- 172.21.15.45:0/3890116353 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8018660 con 0x7fa8c406b580 2024-06-06T04:29:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.560+0000 7fa8c8dd3700 1 -- 172.21.15.45:0/3890116353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c41a7760 con 0x7fa8c406b580 2024-06-06T04:29:10.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.561+0000 7fa8c8dd3700 1 -- 172.21.15.45:0/3890116353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c4069180 con 0x7fa8c406b580 2024-06-06T04:29:10.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.562+0000 7fa8b37fe700 1 -- 172.21.15.45:0/3890116353 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa8b800f450 con 0x7fa8c406b580 2024-06-06T04:29:10.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.563+0000 7fa8b37fe700 1 --2- 172.21.15.45:0/3890116353 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa8ac0a2cb0 0x7fa8ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:10.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.563+0000 7fa8b37fe700 1 -- 172.21.15.45:0/3890116353 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fa8b80cf750 con 0x7fa8c406b580 2024-06-06T04:29:10.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.563+0000 7fa8c259c700 1 --2- 172.21.15.45:0/3890116353 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa8ac0a2cb0 0x7fa8ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:10.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.564+0000 7fa8c259c700 1 --2- 172.21.15.45:0/3890116353 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa8ac0a2cb0 0x7fa8ac0a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400abd0 tx=0x7fa8b400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:10.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.567+0000 7fa8b37fe700 1 -- 172.21.15.45:0/3890116353 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa8b8014030 con 0x7fa8c406b580 2024-06-06T04:29:10.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.777+0000 7fa8c8dd3700 1 -- 172.21.15.45:0/3890116353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8c402dca0 con 0x7fa8c406b580 2024-06-06T04:29:10.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.779+0000 7fa8b37fe700 1 -- 172.21.15.45:0/3890116353 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24789 (secure 0 0 0) 0x7fa8b809c220 con 0x7fa8c406b580 2024-06-06T04:29:10.781 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:29:10.781 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":100,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","created":"2024-06-06T04:19:02.626916+0000","modified":"2024-06-06T04:29:05.901370+0000","last_up_change":"2024-06-06T04:26:17.390429+0000","last_in_change":"2024-06-06T04:25:56.938388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T04:22:28.797137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T04:27:17.750625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T04:27:21.486917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T04:27:31.029818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d3cee4a-5025-45b3-b217-082af37e79db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6803","nonce":2111759250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6805","nonce":2111759250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6809","nonce":2111759250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6807","nonce":2111759250}]},"public_addr":"172.21.15.45:6803/2111759250","cluster_addr":"172.21.15.45:6805/2111759250","heartbeat_back_addr":"172.21.15.45:6809/2111759250","heartbeat_front_addr":"172.21.15.45:6807/2111759250","state":["exists","up"]},{"osd":1,"uuid":"75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6801","nonce":3732225287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6803","nonce":3732225287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6807","nonce":3732225287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6805","nonce":3732225287}]},"public_addr":"172.21.15.86:6801/3732225287","cluster_addr":"172.21.15.86:6803/3732225287","heartbeat_back_addr":"172.21.15.86:6807/3732225287","heartbeat_front_addr":"172.21.15.86:6805/3732225287","state":["exists","up"]},{"osd":2,"uuid":"bb20565f-f9ae-47e3-9604-c69f737cff4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6801","nonce":2898783039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6803","nonce":2898783039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6807","nonce":2898783039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6805","nonce":2898783039}]},"public_addr":"172.21.15.154:6801/2898783039","cluster_addr":"172.21.15.154:6803/2898783039","heartbeat_back_addr":"172.21.15.154:6807/2898783039","heartbeat_front_addr":"172.21.15.154:6805/2898783039","state":["exists","up"]},{"osd":3,"uuid":"e1160799-7811-4326-9792-7e6c7d189c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6811","nonce":3436753135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6813","nonce":3436753135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6817","nonce":3436753135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6815","nonce":3436753135}]},"public_addr":"172.21.15.45:6811/3436753135","cluster_addr":"172.21.15.45:6813/3436753135","heartbeat_back_addr":"172.21.15.45:6817/3436753135","heartbeat_front_addr":"172.21.15.45:6815/3436753135","state":["exists","up"]},{"osd":4,"uuid":"6b389bc8-e19a-47f3-83ff-7b782d146d0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6809","nonce":3030235796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6811","nonce":3030235796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6815","nonce":3030235796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6813","nonce":3030235796}]},"public_addr":"172.21.15.86:6809/3030235796","cluster_addr":"172.21.15.86:6811/3030235796","heartbeat_back_addr":"172.21.15.86:6815/3030235796","heartbeat_front_addr":"172.21.15.86:6813/3030235796","state":["exists","up"]},{"osd":5,"uuid":"069de7b2-89bf-4150-bed7-7b8832573dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6809","nonce":2222268646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6811","nonce":2222268646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6815","nonce":2222268646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6813","nonce":2222268646}]},"public_addr":"172.21.15.154:6809/2222268646","cluster_addr":"172.21.15.154:6811/2222268646","heartbeat_back_addr":"172.21.15.154:6815/2222268646","heartbeat_front_addr":"172.21.15.154:6813/2222268646","state":["exists","up"]},{"osd":6,"uuid":"4c70b627-eafa-4e27-ba2a-4d8e1e335962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6819","nonce":384734939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6821","nonce":384734939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6825","nonce":384734939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6823","nonce":384734939}]},"public_addr":"172.21.15.45:6819/384734939","cluster_addr":"172.21.15.45:6821/384734939","heartbeat_back_addr":"172.21.15.45:6825/384734939","heartbeat_front_addr":"172.21.15.45:6823/384734939","state":["exists","up"]},{"osd":7,"uuid":"3b09b875-be52-4be0-b116-7fd9b0c26705","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6817","nonce":1119791005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6819","nonce":1119791005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6823","nonce":1119791005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6821","nonce":1119791005}]},"public_addr":"172.21.15.86:6817/1119791005","cluster_addr":"172.21.15.86:6819/1119791005","heartbeat_back_addr":"172.21.15.86:6823/1119791005","heartbeat_front_addr":"172.21.15.86:6821/1119791005","state":["exists","up"]},{"osd":8,"uuid":"cdf1dd1b-5392-4836-8e65-4c7c5b688d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6817","nonce":3054812496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6819","nonce":3054812496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6823","nonce":3054812496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6821","nonce":3054812496}]},"public_addr":"172.21.15.154:6817/3054812496","cluster_addr":"172.21.15.154:6819/3054812496","heartbeat_back_addr":"172.21.15.154:6823/3054812496","heartbeat_front_addr":"172.21.15.154:6821/3054812496","state":["exists","up"]},{"osd":9,"uuid":"acede92d-2cc1-4967-a357-7e5e201e99d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6827","nonce":1360655510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6829","nonce":1360655510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6833","nonce":1360655510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6831","nonce":1360655510}]},"public_addr":"172.21.15.45:6827/1360655510","cluster_addr":"172.21.15.45:6829/1360655510","heartbeat_back_addr":"172.21.15.45:6833/1360655510","heartbeat_front_addr":"172.21.15.45:6831/1360655510","state":["exists","up"]},{"osd":10,"uuid":"a2d0c488-1892-4da4-adbf-952c22df0ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6825","nonce":1701691877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6827","nonce":1701691877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6831","nonce":1701691877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6829","nonce":1701691877}]},"public_addr":"172.21.15.86:6825/1701691877","cluster_addr":"172.21.15.86:6827/1701691877","heartbeat_back_addr":"172.21.15.86:6831/1701691877","heartbeat_front_addr":"172.21.15.86:6829/1701691877","state":["exists","up"]},{"osd":11,"uuid":"6b349c4c-b672-480b-ac6a-8594dab0eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6825","nonce":2065648007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6827","nonce":2065648007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6831","nonce":2065648007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6829","nonce":2065648007}]},"public_addr":"172.21.15.154:6825/2065648007","cluster_addr":"172.21.15.154:6827/2065648007","heartbeat_back_addr":"172.21.15.154:6831/2065648007","heartbeat_front_addr":"172.21.15.154:6829/2065648007","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:21:37.147394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:00.776617+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:24.786849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:50.487455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:14.869272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:39.815158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:05.972619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:31.410063+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:56.795119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:23.114884+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:49.256504+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:26:15.251967+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/36732849":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/1500318775":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6801/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6800/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/2041380391":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/2504003341":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/3798552599":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/498952822":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6800/1562511056":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6801/1562511056":"2024-06-07T04:19:37.026199+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T04:29:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.781+0000 7fa8c8dd3700 1 -- 172.21.15.45:0/3890116353 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa8ac0a2cb0 msgr2=0x7fa8ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.781+0000 7fa8c8dd3700 1 --2- 172.21.15.45:0/3890116353 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa8ac0a2cb0 0x7fa8ac0a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400abd0 tx=0x7fa8b400a380 comp rx=0 tx=0).stop 2024-06-06T04:29:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.781+0000 7fa8c8dd3700 1 -- 172.21.15.45:0/3890116353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8c406b580 msgr2=0x7fa8c41019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.781+0000 7fa8c8dd3700 1 --2- 172.21.15.45:0/3890116353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8c406b580 0x7fa8c41019d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800c9b0 tx=0x7fa8b800ccc0 comp rx=0 tx=0).stop 2024-06-06T04:29:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.781+0000 7fa8c8dd3700 1 -- 172.21.15.45:0/3890116353 shutdown_connections 2024-06-06T04:29:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.781+0000 7fa8c8dd3700 1 --2- 172.21.15.45:0/3890116353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8c4108500 0x7fa8c4101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.781+0000 7fa8c8dd3700 1 --2- 172.21.15.45:0/3890116353 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa8ac0a2cb0 0x7fa8ac0a5170 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.781+0000 7fa8c8dd3700 1 --2- 172.21.15.45:0/3890116353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c406abc0 0x7fa8c4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.781+0000 7fa8c8dd3700 1 --2- 172.21.15.45:0/3890116353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8c406b580 0x7fa8c41019d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.781+0000 7fa8c8dd3700 1 -- 172.21.15.45:0/3890116353 >> 172.21.15.45:0/3890116353 conn(0x7fa8c4078340 msgr2=0x7fa8c40fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.782+0000 7fa8c8dd3700 1 -- 172.21.15.45:0/3890116353 shutdown_connections 2024-06-06T04:29:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:10.782+0000 7fa8c8dd3700 1 -- 172.21.15.45:0/3890116353 wait complete. 2024-06-06T04:29:11.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:10 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3890116353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:29:11.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:10 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3890116353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:29:11.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:10 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3890116353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:29:11.426 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-06T04:29:11.427 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-06T04:29:11.427 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-06T04:29:11.428 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-06T04:29:11.428 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-06T04:29:11.428 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-06T04:29:11.429 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-06T04:29:11.429 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-06T04:29:11.430 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-06-06T04:29:11.430 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-06-06T04:29:11.431 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-06-06T04:29:11.431 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-06-06T04:29:12.161 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:12.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:11 smithi154 ceph-mon[36428]: pgmap v363: 129 pgs: 1 activating, 128 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:29:12.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:11 smithi045 ceph-mon[32299]: pgmap v363: 129 pgs: 1 activating, 128 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:29:12.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:11 smithi086 ceph-mon[36053]: pgmap v363: 129 pgs: 1 activating, 128 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:29:12.978 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:12.984 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:12.986 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:12.991 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:12.994 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:12.998 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:12.998 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:12.998 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:13.003 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:13.004 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:13.004 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:13.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:12 smithi045 ceph-mon[32299]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-06-06T04:29:13.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:12 smithi045 ceph-mon[32299]: Cluster is now healthy 2024-06-06T04:29:13.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:12 smithi086 ceph-mon[36053]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-06-06T04:29:13.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:12 smithi086 ceph-mon[36053]: Cluster is now healthy 2024-06-06T04:29:13.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:12 smithi154 ceph-mon[36428]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-06-06T04:29:13.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:12 smithi154 ceph-mon[36428]: Cluster is now healthy 2024-06-06T04:29:14.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:13 smithi045 ceph-mon[32299]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T04:29:14.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:13 smithi086 ceph-mon[36053]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T04:29:14.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:13 smithi154 ceph-mon[36428]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T04:29:16.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:15 smithi086 ceph-mon[36053]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-06-06T04:29:16.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:15 smithi154 ceph-mon[36428]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-06-06T04:29:16.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:15 smithi045 ceph-mon[32299]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-06-06T04:29:18.111 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:17 smithi045 ceph-mon[32299]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 195 B/s wr, 5 op/s 2024-06-06T04:29:18.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:17 smithi086 ceph-mon[36053]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 195 B/s wr, 5 op/s 2024-06-06T04:29:18.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:17 smithi154 ceph-mon[36428]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 195 B/s wr, 5 op/s 2024-06-06T04:29:20.296 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:19 smithi045 ceph-mon[32299]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-06-06T04:29:20.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:19 smithi086 ceph-mon[36053]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-06-06T04:29:20.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:19 smithi154 ceph-mon[36428]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-06-06T04:29:20.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.978+0000 7f3ad16bb700 1 -- 172.21.15.45:0/3534392164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc1073f0 msgr2=0x7f3acc1077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.978+0000 7f3ad16bb700 1 --2- 172.21.15.45:0/3534392164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc1073f0 0x7f3acc1077d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009990 tx=0x7f3abc009ca0 comp rx=0 tx=0).stop 2024-06-06T04:29:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.978+0000 7f3ad16bb700 1 -- 172.21.15.45:0/3534392164 shutdown_connections 2024-06-06T04:29:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.978+0000 7f3ad16bb700 1 --2- 172.21.15.45:0/3534392164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3acc101d40 0x7f3acc10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.978+0000 7f3ad16bb700 1 --2- 172.21.15.45:0/3534392164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc1073f0 0x7f3acc1077d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.978+0000 7f3ad16bb700 1 --2- 172.21.15.45:0/3534392164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3acc101380 0x7f3acc101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.978+0000 7f3ad16bb700 1 -- 172.21.15.45:0/3534392164 >> 172.21.15.45:0/3534392164 conn(0x7f3acc078300 msgr2=0x7f3acc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.978+0000 7f3ad16bb700 1 -- 172.21.15.45:0/3534392164 shutdown_connections 2024-06-06T04:29:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.978+0000 7f3ad16bb700 1 -- 172.21.15.45:0/3534392164 wait complete. 2024-06-06T04:29:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.979+0000 7f3ad16bb700 1 Processor -- start 2024-06-06T04:29:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.979+0000 7f3ad16bb700 1 -- start start 2024-06-06T04:29:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.979+0000 7f3ad16bb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3acc101380 0x7f3acc19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.979+0000 7f3ad16bb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3acc101d40 0x7f3acc19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.979+0000 7f3ad16bb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc1073f0 0x7f3acc198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.979+0000 7f3ad16bb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc112640 con 0x7f3acc1073f0 2024-06-06T04:29:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3ad16bb700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc1124c0 con 0x7f3acc101d40 2024-06-06T04:29:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3ad16bb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc1127c0 con 0x7f3acc101380 2024-06-06T04:29:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3acb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc1073f0 0x7f3acc198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3acb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc1073f0 0x7f3acc198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39414/0 (socket says 172.21.15.45:39414) 2024-06-06T04:29:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3acb7fe700 1 -- 172.21.15.45:0/3764829964 learned_addr learned my addr 172.21.15.45:0/3764829964 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3acaffd700 1 --2- 172.21.15.45:0/3764829964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3acc101380 0x7f3acc19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3ac27fc700 1 --2- 172.21.15.45:0/3764829964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3acc101d40 0x7f3acc19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3acb7fe700 1 -- 172.21.15.45:0/3764829964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3acc101380 msgr2=0x7f3acc19e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3acb7fe700 1 --2- 172.21.15.45:0/3764829964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3acc101380 0x7f3acc19e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3acb7fe700 1 -- 172.21.15.45:0/3764829964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3acc101d40 msgr2=0x7f3acc19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:20.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3acb7fe700 1 --2- 172.21.15.45:0/3764829964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3acc101d40 0x7f3acc19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3acb7fe700 1 -- 172.21.15.45:0/3764829964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3acc199080 con 0x7f3acc1073f0 2024-06-06T04:29:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3acaffd700 1 --2- 172.21.15.45:0/3764829964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3acc101380 0x7f3acc19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:29:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3acb7fe700 1 --2- 172.21.15.45:0/3764829964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc1073f0 0x7f3acc198920 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4007ee0 tx=0x7f3ac400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3ac8ff9700 1 -- 172.21.15.45:0/3764829964 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac400dd20 con 0x7f3acc1073f0 2024-06-06T04:29:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3ac8ff9700 1 -- 172.21.15.45:0/3764829964 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ac4004020 con 0x7f3acc1073f0 2024-06-06T04:29:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3ad16bb700 1 -- 172.21.15.45:0/3764829964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3acc199370 con 0x7f3acc1073f0 2024-06-06T04:29:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.980+0000 7f3ac8ff9700 1 -- 172.21.15.45:0/3764829964 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4015660 con 0x7f3acc1073f0 2024-06-06T04:29:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.981+0000 7f3ad16bb700 1 -- 172.21.15.45:0/3764829964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3acc1a5630 con 0x7f3acc1073f0 2024-06-06T04:29:20.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.981+0000 7f3ad16bb700 1 -- 172.21.15.45:0/3764829964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3acc062c20 con 0x7f3acc1073f0 2024-06-06T04:29:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.983+0000 7f3ac8ff9700 1 -- 172.21.15.45:0/3764829964 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3ac40041d0 con 0x7f3acc1073f0 2024-06-06T04:29:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.984+0000 7f3ac8ff9700 1 --2- 172.21.15.45:0/3764829964 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3ab80a2d30 0x7f3ab80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.984+0000 7f3acaffd700 1 --2- 172.21.15.45:0/3764829964 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3ab80a2d30 0x7f3ab80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.984+0000 7f3ac8ff9700 1 -- 172.21.15.45:0/3764829964 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f3ac40cfa50 con 0x7f3acc1073f0 2024-06-06T04:29:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.985+0000 7f3ac8ff9700 1 --2- 172.21.15.45:0/3764829964 >> [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287] conn(0x7f3ab80ae010 0x7f3ab80b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.985+0000 7f3ac8ff9700 1 -- 172.21.15.45:0/3764829964 --> [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ac4020e50 con 0x7f3ab80ae010 2024-06-06T04:29:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.985+0000 7f3ac8ff9700 1 -- 172.21.15.45:0/3764829964 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f3ac40cfe00 con 0x7f3acc1073f0 2024-06-06T04:29:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.985+0000 7f3acaffd700 1 --2- 172.21.15.45:0/3764829964 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3ab80a2d30 0x7f3ab80a51f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00b2e0 tx=0x7f3abc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.985+0000 7f3ac27fc700 1 --2- 172.21.15.45:0/3764829964 >> [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287] conn(0x7f3ab80ae010 0x7f3ab80b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.985+0000 7f3ac27fc700 1 --2- 172.21.15.45:0/3764829964 >> [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287] conn(0x7f3ab80ae010 0x7f3ab80b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:20.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:18.986+0000 7f3ac8ff9700 1 -- 172.21.15.45:0/3764829964 <== osd.1 v2:172.21.15.86:6800/3732225287 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3ac4020e50 con 0x7f3ab80ae010 2024-06-06T04:29:20.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:19.003+0000 7f3ad16bb700 1 -- 172.21.15.45:0/3764829964 --> [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3acc0630a0 con 0x7f3ab80ae010 2024-06-06T04:29:20.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:19.004+0000 7f3ac8ff9700 1 -- 172.21.15.45:0/3764829964 <== osd.1 v2:172.21.15.86:6800/3732225287 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3acc0630a0 con 0x7f3ab80ae010 2024-06-06T04:29:20.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:19.004+0000 7f3ad16bb700 1 -- 172.21.15.45:0/3764829964 >> [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287] conn(0x7f3ab80ae010 msgr2=0x7f3ab80b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:20.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:19.004+0000 7f3ad16bb700 1 --2- 172.21.15.45:0/3764829964 >> [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287] conn(0x7f3ab80ae010 0x7f3ab80b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:20.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:19.004+0000 7f3ad16bb700 1 -- 172.21.15.45:0/3764829964 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3ab80a2d30 msgr2=0x7f3ab80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:20.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:19.004+0000 7f3ad16bb700 1 --2- 172.21.15.45:0/3764829964 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3ab80a2d30 0x7f3ab80a51f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00b2e0 tx=0x7f3abc01a040 comp rx=0 tx=0).stop 2024-06-06T04:29:20.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:19.004+0000 7f3ad16bb700 1 -- 172.21.15.45:0/3764829964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc1073f0 msgr2=0x7f3acc198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:20.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:19.004+0000 7f3ad16bb700 1 --2- 172.21.15.45:0/3764829964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3acc1073f0 0x7f3acc198920 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4007ee0 tx=0x7f3ac400d3b0 comp rx=0 tx=0).stop 2024-06-06T04:29:20.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:19.004+0000 7f3acb7fe700 1 -- 172.21.15.45:0/3764829964 reap_dead start 2024-06-06T04:29:20.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:19.004+0000 7f3ad16bb700 1 -- 172.21.15.45:0/3764829964 shutdown_connections 2024-06-06T04:29:20.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:19.004+0000 7f3ad16bb700 1 -- 172.21.15.45:0/3764829964 >> 172.21.15.45:0/3764829964 conn(0x7f3acc078300 msgr2=0x7f3acc1002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:20.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:19.004+0000 7f3ad16bb700 1 -- 172.21.15.45:0/3764829964 shutdown_connections 2024-06-06T04:29:20.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:19.005+0000 7f3ad16bb700 1 -- 172.21.15.45:0/3764829964 wait complete. 2024-06-06T04:29:22.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:21 smithi045 ceph-mon[32299]: pgmap v368: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:22.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:21 smithi086 ceph-mon[36053]: pgmap v368: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:22.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:21 smithi154 ceph-mon[36428]: pgmap v368: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.325+0000 7f0cf14d3700 1 -- 172.21.15.45:0/3337206761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec06a170 msgr2=0x7f0cec06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.325+0000 7f0cf14d3700 1 --2- 172.21.15.45:0/3337206761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec06a170 0x7f0cec06a5f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009990 tx=0x7f0cd4009ca0 comp rx=0 tx=0).stop 2024-06-06T04:29:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.325+0000 7f0cf14d3700 1 -- 172.21.15.45:0/3337206761 shutdown_connections 2024-06-06T04:29:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.325+0000 7f0cf14d3700 1 --2- 172.21.15.45:0/3337206761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0cec06ab30 0x7f0cec10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.325+0000 7f0cf14d3700 1 --2- 172.21.15.45:0/3337206761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cec108500 0x7f0cec1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.325+0000 7f0cf14d3700 1 --2- 172.21.15.45:0/3337206761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec06a170 0x7f0cec06a5f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.325+0000 7f0cf14d3700 1 -- 172.21.15.45:0/3337206761 >> 172.21.15.45:0/3337206761 conn(0x7f0cec078340 msgr2=0x7f0cec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.325+0000 7f0cf14d3700 1 -- 172.21.15.45:0/3337206761 shutdown_connections 2024-06-06T04:29:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.325+0000 7f0cf14d3700 1 -- 172.21.15.45:0/3337206761 wait complete. 2024-06-06T04:29:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.326+0000 7f0cf14d3700 1 Processor -- start 2024-06-06T04:29:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.326+0000 7f0cf14d3700 1 -- start start 2024-06-06T04:29:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.326+0000 7f0cf14d3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cec06a170 0x7f0cec19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.326+0000 7f0cf14d3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec06ab30 0x7f0cec19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.326+0000 7f0cf14d3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0cec108500 0x7f0cec198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.326+0000 7f0cf14d3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec112630 con 0x7f0cec06ab30 2024-06-06T04:29:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.326+0000 7f0cf14d3700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec1124b0 con 0x7f0cec06a170 2024-06-06T04:29:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.326+0000 7f0cf14d3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec1127b0 con 0x7f0cec108500 2024-06-06T04:29:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.326+0000 7f0ce27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec06ab30 0x7f0cec19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.326+0000 7f0ce27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec06ab30 0x7f0cec19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39420/0 (socket says 172.21.15.45:39420) 2024-06-06T04:29:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.326+0000 7f0ce27fc700 1 -- 172.21.15.45:0/1148001471 learned_addr learned my addr 172.21.15.45:0/1148001471 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.327+0000 7f0ceaffd700 1 --2- 172.21.15.45:0/1148001471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cec06a170 0x7f0cec19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.327+0000 7f0ceb7fe700 1 --2- 172.21.15.45:0/1148001471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0cec108500 0x7f0cec198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.327+0000 7f0ce27fc700 1 -- 172.21.15.45:0/1148001471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0cec108500 msgr2=0x7f0cec198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.327+0000 7f0ce27fc700 1 --2- 172.21.15.45:0/1148001471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0cec108500 0x7f0cec198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.327+0000 7f0ce27fc700 1 -- 172.21.15.45:0/1148001471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cec06a170 msgr2=0x7f0cec19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.327+0000 7f0ce27fc700 1 --2- 172.21.15.45:0/1148001471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cec06a170 0x7f0cec19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.327+0000 7f0ce27fc700 1 -- 172.21.15.45:0/1148001471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec1990f0 con 0x7f0cec06ab30 2024-06-06T04:29:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.327+0000 7f0ce27fc700 1 --2- 172.21.15.45:0/1148001471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec06ab30 0x7f0cec19ecc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40057c0 tx=0x7f0cd40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.327+0000 7f0ce8ff9700 1 -- 172.21.15.45:0/1148001471 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4026070 con 0x7f0cec06ab30 2024-06-06T04:29:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.327+0000 7f0cf14d3700 1 -- 172.21.15.45:0/1148001471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cec199380 con 0x7f0cec06ab30 2024-06-06T04:29:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.327+0000 7f0ce8ff9700 1 -- 172.21.15.45:0/1148001471 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cd4021410 con 0x7f0cec06ab30 2024-06-06T04:29:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.327+0000 7f0ce8ff9700 1 -- 172.21.15.45:0/1148001471 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4017650 con 0x7f0cec06ab30 2024-06-06T04:29:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.327+0000 7f0cf14d3700 1 -- 172.21.15.45:0/1148001471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cec1a5650 con 0x7f0cec06ab30 2024-06-06T04:29:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.328+0000 7f0cf14d3700 1 -- 172.21.15.45:0/1148001471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0cec062c20 con 0x7f0cec06ab30 2024-06-06T04:29:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.330+0000 7f0ce8ff9700 1 -- 172.21.15.45:0/1148001471 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0cd4029070 con 0x7f0cec06ab30 2024-06-06T04:29:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.330+0000 7f0ce8ff9700 1 --2- 172.21.15.45:0/1148001471 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0cd80a2cb0 0x7f0cd80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.330+0000 7f0ce8ff9700 1 -- 172.21.15.45:0/1148001471 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f0cd40d07e0 con 0x7f0cec06ab30 2024-06-06T04:29:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.330+0000 7f0ceaffd700 1 --2- 172.21.15.45:0/1148001471 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0cd80a2cb0 0x7f0cd80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.331+0000 7f0ce8ff9700 1 --2- 172.21.15.45:0/1148001471 >> [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007] conn(0x7f0cd80ade60 0x7f0cd80b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.331+0000 7f0ce8ff9700 1 -- 172.21.15.45:0/1148001471 --> [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0cd80b09d0 con 0x7f0cd80ade60 2024-06-06T04:29:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.331+0000 7f0ce8ff9700 1 -- 172.21.15.45:0/1148001471 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f0cd40d0c30 con 0x7f0cec06ab30 2024-06-06T04:29:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.331+0000 7f0ceaffd700 1 --2- 172.21.15.45:0/1148001471 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0cd80a2cb0 0x7f0cd80a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0ffa60 tx=0x7f0cdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.331+0000 7f0ceb7fe700 1 --2- 172.21.15.45:0/1148001471 >> [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007] conn(0x7f0cd80ade60 0x7f0cd80b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.331+0000 7f0ceb7fe700 1 --2- 172.21.15.45:0/1148001471 >> [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007] conn(0x7f0cd80ade60 0x7f0cd80b0300 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.332+0000 7f0ce8ff9700 1 -- 172.21.15.45:0/1148001471 <== osd.11 v2:172.21.15.154:6824/2065648007 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0cd80b09d0 con 0x7f0cd80ade60 2024-06-06T04:29:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.355+0000 7f0cf14d3700 1 -- 172.21.15.45:0/1148001471 --> [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0cec063020 con 0x7f0cd80ade60 2024-06-06T04:29:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.356+0000 7f0ce8ff9700 1 -- 172.21.15.45:0/1148001471 <== osd.11 v2:172.21.15.154:6824/2065648007 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0cec063020 con 0x7f0cd80ade60 2024-06-06T04:29:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.357+0000 7f0cf14d3700 1 -- 172.21.15.45:0/1148001471 >> [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007] conn(0x7f0cd80ade60 msgr2=0x7f0cd80b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.357+0000 7f0cf14d3700 1 --2- 172.21.15.45:0/1148001471 >> [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007] conn(0x7f0cd80ade60 0x7f0cd80b0300 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.357+0000 7f0cf14d3700 1 -- 172.21.15.45:0/1148001471 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0cd80a2cb0 msgr2=0x7f0cd80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.357+0000 7f0cf14d3700 1 --2- 172.21.15.45:0/1148001471 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0cd80a2cb0 0x7f0cd80a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0ffa60 tx=0x7f0cdc009260 comp rx=0 tx=0).stop 2024-06-06T04:29:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.357+0000 7f0cf14d3700 1 -- 172.21.15.45:0/1148001471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec06ab30 msgr2=0x7f0cec19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.357+0000 7f0cf14d3700 1 --2- 172.21.15.45:0/1148001471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cec06ab30 0x7f0cec19ecc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40057c0 tx=0x7f0cd40057f0 comp rx=0 tx=0).stop 2024-06-06T04:29:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.357+0000 7f0ceb7fe700 1 -- 172.21.15.45:0/1148001471 reap_dead start 2024-06-06T04:29:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.357+0000 7f0cf14d3700 1 -- 172.21.15.45:0/1148001471 shutdown_connections 2024-06-06T04:29:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.357+0000 7f0cf14d3700 1 -- 172.21.15.45:0/1148001471 >> 172.21.15.45:0/1148001471 conn(0x7f0cec078340 msgr2=0x7f0cec100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.357+0000 7f0cf14d3700 1 -- 172.21.15.45:0/1148001471 shutdown_connections 2024-06-06T04:29:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:22.357+0000 7f0cf14d3700 1 -- 172.21.15.45:0/1148001471 wait complete. 2024-06-06T04:29:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.781+0000 7fa9904f4700 1 -- 172.21.15.45:0/4227834540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa98806aa00 msgr2=0x7fa98806ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.781+0000 7fa9904f4700 1 --2- 172.21.15.45:0/4227834540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa98806aa00 0x7fa98806ae80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009990 tx=0x7fa97c009ca0 comp rx=0 tx=0).stop 2024-06-06T04:29:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.781+0000 7fa9904f4700 1 -- 172.21.15.45:0/4227834540 shutdown_connections 2024-06-06T04:29:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.781+0000 7fa9904f4700 1 --2- 172.21.15.45:0/4227834540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa98806b3c0 0x7fa98810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.781+0000 7fa9904f4700 1 --2- 172.21.15.45:0/4227834540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9881084f0 0x7fa9881088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.781+0000 7fa9904f4700 1 --2- 172.21.15.45:0/4227834540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa98806aa00 0x7fa98806ae80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.781+0000 7fa9904f4700 1 -- 172.21.15.45:0/4227834540 >> 172.21.15.45:0/4227834540 conn(0x7fa988078300 msgr2=0x7fa988078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.781+0000 7fa9904f4700 1 -- 172.21.15.45:0/4227834540 shutdown_connections 2024-06-06T04:29:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.781+0000 7fa9904f4700 1 -- 172.21.15.45:0/4227834540 wait complete. 2024-06-06T04:29:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.782+0000 7fa9904f4700 1 Processor -- start 2024-06-06T04:29:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.782+0000 7fa9904f4700 1 -- start start 2024-06-06T04:29:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.782+0000 7fa9904f4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa98806aa00 0x7fa98819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.782+0000 7fa9904f4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98806b3c0 0x7fa98819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.782+0000 7fa9904f4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9881084f0 0x7fa98819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.782+0000 7fa9904f4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa988112600 con 0x7fa98806aa00 2024-06-06T04:29:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.782+0000 7fa9904f4700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa988112480 con 0x7fa98806b3c0 2024-06-06T04:29:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.782+0000 7fa9904f4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa988112780 con 0x7fa9881084f0 2024-06-06T04:29:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.782+0000 7fa98e290700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa98806aa00 0x7fa98819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.782+0000 7fa98e290700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa98806aa00 0x7fa98819ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39444/0 (socket says 172.21.15.45:39444) 2024-06-06T04:29:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.782+0000 7fa98e290700 1 -- 172.21.15.45:0/3638492152 learned_addr learned my addr 172.21.15.45:0/3638492152 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.782+0000 7fa98da8f700 1 --2- 172.21.15.45:0/3638492152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98806b3c0 0x7fa98819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.783+0000 7fa98ea91700 1 --2- 172.21.15.45:0/3638492152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9881084f0 0x7fa98819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.783+0000 7fa98da8f700 1 -- 172.21.15.45:0/3638492152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9881084f0 msgr2=0x7fa98819f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.783+0000 7fa98da8f700 1 --2- 172.21.15.45:0/3638492152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9881084f0 0x7fa98819f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.783+0000 7fa98da8f700 1 -- 172.21.15.45:0/3638492152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa98806aa00 msgr2=0x7fa98819ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.783+0000 7fa98da8f700 1 --2- 172.21.15.45:0/3638492152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa98806aa00 0x7fa98819ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.783+0000 7fa98da8f700 1 -- 172.21.15.45:0/3638492152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98819fd40 con 0x7fa98806b3c0 2024-06-06T04:29:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.783+0000 7fa98e290700 1 --2- 172.21.15.45:0/3638492152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa98806aa00 0x7fa98819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:29:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.783+0000 7fa98da8f700 1 --2- 172.21.15.45:0/3638492152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98806b3c0 0x7fa98819b270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0059d0 tx=0x7fa97c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.783+0000 7fa97b7fe700 1 -- 172.21.15.45:0/3638492152 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c027070 con 0x7fa98806b3c0 2024-06-06T04:29:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.783+0000 7fa97b7fe700 1 -- 172.21.15.45:0/3638492152 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa97c022470 con 0x7fa98806b3c0 2024-06-06T04:29:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.783+0000 7fa9904f4700 1 -- 172.21.15.45:0/3638492152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98819ffd0 con 0x7fa98806b3c0 2024-06-06T04:29:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.783+0000 7fa97b7fe700 1 -- 172.21.15.45:0/3638492152 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c017630 con 0x7fa98806b3c0 2024-06-06T04:29:23.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.783+0000 7fa9904f4700 1 -- 172.21.15.45:0/3638492152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9881a78b0 con 0x7fa98806b3c0 2024-06-06T04:29:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.784+0000 7fa9904f4700 1 -- 172.21.15.45:0/3638492152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa98810e0a0 con 0x7fa98806b3c0 2024-06-06T04:29:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.785+0000 7fa97b7fe700 1 -- 172.21.15.45:0/3638492152 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa97c02a070 con 0x7fa98806b3c0 2024-06-06T04:29:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.786+0000 7fa97b7fe700 1 --2- 172.21.15.45:0/3638492152 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa9740a2d30 0x7fa9740a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.786+0000 7fa97b7fe700 1 -- 172.21.15.45:0/3638492152 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fa97c0d0740 con 0x7fa98806b3c0 2024-06-06T04:29:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.786+0000 7fa98e290700 1 --2- 172.21.15.45:0/3638492152 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa9740a2d30 0x7fa9740a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.786+0000 7fa97b7fe700 1 --2- 172.21.15.45:0/3638492152 >> [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646] conn(0x7fa9740ae010 0x7fa9740b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.786+0000 7fa97b7fe700 1 -- 172.21.15.45:0/3638492152 --> [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa97c019e80 con 0x7fa9740ae010 2024-06-06T04:29:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.786+0000 7fa97b7fe700 1 -- 172.21.15.45:0/3638492152 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fa97c0d0af0 con 0x7fa98806b3c0 2024-06-06T04:29:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.787+0000 7fa98ea91700 1 --2- 172.21.15.45:0/3638492152 >> [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646] conn(0x7fa9740ae010 0x7fa9740b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.787+0000 7fa98e290700 1 --2- 172.21.15.45:0/3638492152 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa9740a2d30 0x7fa9740a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa984011550 tx=0x7fa984009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.787+0000 7fa98ea91700 1 --2- 172.21.15.45:0/3638492152 >> [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646] conn(0x7fa9740ae010 0x7fa9740b04b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:23.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.788+0000 7fa97b7fe700 1 -- 172.21.15.45:0/3638492152 <== osd.5 v2:172.21.15.154:6808/2222268646 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa97c019e80 con 0x7fa9740ae010 2024-06-06T04:29:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.805+0000 7fa9904f4700 1 -- 172.21.15.45:0/3638492152 --> [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa988103bb0 con 0x7fa9740ae010 2024-06-06T04:29:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.806+0000 7fa97b7fe700 1 -- 172.21.15.45:0/3638492152 <== osd.5 v2:172.21.15.154:6808/2222268646 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa988103bb0 con 0x7fa9740ae010 2024-06-06T04:29:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.806+0000 7fa9904f4700 1 -- 172.21.15.45:0/3638492152 >> [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646] conn(0x7fa9740ae010 msgr2=0x7fa9740b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.806+0000 7fa9904f4700 1 --2- 172.21.15.45:0/3638492152 >> [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646] conn(0x7fa9740ae010 0x7fa9740b04b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.806+0000 7fa9904f4700 1 -- 172.21.15.45:0/3638492152 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa9740a2d30 msgr2=0x7fa9740a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.806+0000 7fa9904f4700 1 --2- 172.21.15.45:0/3638492152 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa9740a2d30 0x7fa9740a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa984011550 tx=0x7fa984009260 comp rx=0 tx=0).stop 2024-06-06T04:29:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.806+0000 7fa9904f4700 1 -- 172.21.15.45:0/3638492152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98806b3c0 msgr2=0x7fa98819b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.806+0000 7fa9904f4700 1 --2- 172.21.15.45:0/3638492152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98806b3c0 0x7fa98819b270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0059d0 tx=0x7fa97c0047b0 comp rx=0 tx=0).stop 2024-06-06T04:29:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.806+0000 7fa98ea91700 1 -- 172.21.15.45:0/3638492152 reap_dead start 2024-06-06T04:29:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.806+0000 7fa9904f4700 1 -- 172.21.15.45:0/3638492152 shutdown_connections 2024-06-06T04:29:23.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.806+0000 7fa9904f4700 1 -- 172.21.15.45:0/3638492152 >> 172.21.15.45:0/3638492152 conn(0x7fa988078300 msgr2=0x7fa98810d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.806+0000 7fa9904f4700 1 -- 172.21.15.45:0/3638492152 shutdown_connections 2024-06-06T04:29:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:23.806+0000 7fa9904f4700 1 -- 172.21.15.45:0/3638492152 wait complete. 2024-06-06T04:29:24.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:23 smithi045 ceph-mon[32299]: pgmap v369: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:29:24.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:23 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:29:24.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:23 smithi086 ceph-mon[36053]: pgmap v369: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:29:24.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:23 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:29:24.362 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:23 smithi154 ceph-mon[36428]: pgmap v369: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:29:24.362 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:23 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:29:24.508 INFO:teuthology.orchestra.run.smithi045.stdout:347892351016 2024-06-06T04:29:24.509 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-06-06T04:29:26.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.008+0000 7ff25eb69700 1 -- 172.21.15.45:0/1469376629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff258102ef0 msgr2=0x7ff25810f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:26.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.008+0000 7ff25eb69700 1 --2- 172.21.15.45:0/1469376629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff258102ef0 0x7ff25810f3a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff25000b0d0 tx=0x7ff25000b3e0 comp rx=0 tx=0).stop 2024-06-06T04:29:26.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.008+0000 7ff25eb69700 1 -- 172.21.15.45:0/1469376629 shutdown_connections 2024-06-06T04:29:26.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.008+0000 7ff25eb69700 1 --2- 172.21.15.45:0/1469376629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff258102ef0 0x7ff25810f3a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:26.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.008+0000 7ff25eb69700 1 --2- 172.21.15.45:0/1469376629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2581085d0 0x7ff2581089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:26.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.008+0000 7ff25eb69700 1 --2- 172.21.15.45:0/1469376629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff258102530 0x7ff2581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:26.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.008+0000 7ff25eb69700 1 -- 172.21.15.45:0/1469376629 >> 172.21.15.45:0/1469376629 conn(0x7ff258079640 msgr2=0x7ff258079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:26.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.008+0000 7ff25eb69700 1 -- 172.21.15.45:0/1469376629 shutdown_connections 2024-06-06T04:29:26.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.008+0000 7ff25eb69700 1 -- 172.21.15.45:0/1469376629 wait complete. 2024-06-06T04:29:26.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.009+0000 7ff25eb69700 1 Processor -- start 2024-06-06T04:29:26.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.009+0000 7ff25eb69700 1 -- start start 2024-06-06T04:29:26.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.009+0000 7ff25eb69700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff258102530 0x7ff25819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:26.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.009+0000 7ff25eb69700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff258102ef0 0x7ff25819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:26.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.009+0000 7ff25eb69700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2581085d0 0x7ff258198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:26.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.009+0000 7ff25eb69700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff258112710 con 0x7ff258102530 2024-06-06T04:29:26.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.009+0000 7ff25eb69700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff258112590 con 0x7ff2581085d0 2024-06-06T04:29:26.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.009+0000 7ff25eb69700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff258112890 con 0x7ff258102ef0 2024-06-06T04:29:26.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.009+0000 7ff25c905700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff258102530 0x7ff25819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:26.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.009+0000 7ff25d106700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2581085d0 0x7ff258198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:26.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.009+0000 7ff25c905700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff258102530 0x7ff25819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39458/0 (socket says 172.21.15.45:39458) 2024-06-06T04:29:26.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.009+0000 7ff25c905700 1 -- 172.21.15.45:0/3538548116 learned_addr learned my addr 172.21.15.45:0/3538548116 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:26.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.009+0000 7ff257fff700 1 --2- 172.21.15.45:0/3538548116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff258102ef0 0x7ff25819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:26.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.010+0000 7ff25c905700 1 -- 172.21.15.45:0/3538548116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff258102ef0 msgr2=0x7ff25819ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:26.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.010+0000 7ff25c905700 1 --2- 172.21.15.45:0/3538548116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff258102ef0 0x7ff25819ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:26.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.010+0000 7ff25c905700 1 -- 172.21.15.45:0/3538548116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2581085d0 msgr2=0x7ff258198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:26.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.010+0000 7ff25c905700 1 --2- 172.21.15.45:0/3538548116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2581085d0 0x7ff258198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:26.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.010+0000 7ff25c905700 1 -- 172.21.15.45:0/3538548116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2581990b0 con 0x7ff258102530 2024-06-06T04:29:26.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.010+0000 7ff25c905700 1 --2- 172.21.15.45:0/3538548116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff258102530 0x7ff25819e740 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00e9a0 tx=0x7ff24c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:26.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.010+0000 7ff255ffb700 1 -- 172.21.15.45:0/3538548116 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00ca10 con 0x7ff258102530 2024-06-06T04:29:26.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.010+0000 7ff25eb69700 1 -- 172.21.15.45:0/3538548116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2581993a0 con 0x7ff258102530 2024-06-06T04:29:26.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.010+0000 7ff255ffb700 1 -- 172.21.15.45:0/3538548116 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff24c00cbb0 con 0x7ff258102530 2024-06-06T04:29:26.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.010+0000 7ff255ffb700 1 -- 172.21.15.45:0/3538548116 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c018680 con 0x7ff258102530 2024-06-06T04:29:26.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.010+0000 7ff25eb69700 1 -- 172.21.15.45:0/3538548116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2581a5610 con 0x7ff258102530 2024-06-06T04:29:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.011+0000 7ff25eb69700 1 -- 172.21.15.45:0/3538548116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff258062ba0 con 0x7ff258102530 2024-06-06T04:29:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.013+0000 7ff255ffb700 1 -- 172.21.15.45:0/3538548116 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff24c010430 con 0x7ff258102530 2024-06-06T04:29:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.013+0000 7ff255ffb700 1 --2- 172.21.15.45:0/3538548116 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff2480a2d30 0x7ff2480a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.014+0000 7ff257fff700 1 --2- 172.21.15.45:0/3538548116 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff2480a2d30 0x7ff2480a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.014+0000 7ff255ffb700 1 -- 172.21.15.45:0/3538548116 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7ff24c0d0250 con 0x7ff258102530 2024-06-06T04:29:26.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.014+0000 7ff255ffb700 1 --2- 172.21.15.45:0/3538548116 >> [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496] conn(0x7ff2480ae010 0x7ff2480b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:26.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.014+0000 7ff255ffb700 1 -- 172.21.15.45:0/3538548116 --> [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff24c01bec0 con 0x7ff2480ae010 2024-06-06T04:29:26.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.014+0000 7ff255ffb700 1 -- 172.21.15.45:0/3538548116 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7ff24c0d0690 con 0x7ff258102530 2024-06-06T04:29:26.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.014+0000 7ff25d106700 1 --2- 172.21.15.45:0/3538548116 >> [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496] conn(0x7ff2480ae010 0x7ff2480b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:26.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.014+0000 7ff257fff700 1 --2- 172.21.15.45:0/3538548116 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff2480a2d30 0x7ff2480a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff258079c40 tx=0x7ff244009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:26.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.014+0000 7ff25d106700 1 --2- 172.21.15.45:0/3538548116 >> [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496] conn(0x7ff2480ae010 0x7ff2480b04b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:26.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.015+0000 7ff255ffb700 1 -- 172.21.15.45:0/3538548116 <== osd.8 v2:172.21.15.154:6816/3054812496 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff24c01bec0 con 0x7ff2480ae010 2024-06-06T04:29:26.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.032+0000 7ff25eb69700 1 -- 172.21.15.45:0/3538548116 --> [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff258062fa0 con 0x7ff2480ae010 2024-06-06T04:29:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.033+0000 7ff255ffb700 1 -- 172.21.15.45:0/3538548116 <== osd.8 v2:172.21.15.154:6816/3054812496 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff258062fa0 con 0x7ff2480ae010 2024-06-06T04:29:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.033+0000 7ff25eb69700 1 -- 172.21.15.45:0/3538548116 >> [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496] conn(0x7ff2480ae010 msgr2=0x7ff2480b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.033+0000 7ff25eb69700 1 --2- 172.21.15.45:0/3538548116 >> [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496] conn(0x7ff2480ae010 0x7ff2480b04b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.033+0000 7ff25eb69700 1 -- 172.21.15.45:0/3538548116 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff2480a2d30 msgr2=0x7ff2480a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.033+0000 7ff25eb69700 1 --2- 172.21.15.45:0/3538548116 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff2480a2d30 0x7ff2480a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff258079c40 tx=0x7ff244009260 comp rx=0 tx=0).stop 2024-06-06T04:29:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.033+0000 7ff25eb69700 1 -- 172.21.15.45:0/3538548116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff258102530 msgr2=0x7ff25819e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.033+0000 7ff25eb69700 1 --2- 172.21.15.45:0/3538548116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff258102530 0x7ff25819e740 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00e9a0 tx=0x7ff24c00ecb0 comp rx=0 tx=0).stop 2024-06-06T04:29:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.033+0000 7ff25d106700 1 -- 172.21.15.45:0/3538548116 reap_dead start 2024-06-06T04:29:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.033+0000 7ff25eb69700 1 -- 172.21.15.45:0/3538548116 shutdown_connections 2024-06-06T04:29:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.033+0000 7ff25eb69700 1 -- 172.21.15.45:0/3538548116 >> 172.21.15.45:0/3538548116 conn(0x7ff258079640 msgr2=0x7ff258101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:26.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.034+0000 7ff25eb69700 1 -- 172.21.15.45:0/3538548116 shutdown_connections 2024-06-06T04:29:26.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:26.034+0000 7ff25eb69700 1 -- 172.21.15.45:0/3538548116 wait complete. 2024-06-06T04:29:26.458 INFO:teuthology.orchestra.run.smithi045.stdout:55834574939 2024-06-06T04:29:26.458 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-06T04:29:26.485 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:26 smithi045 ceph-mon[32299]: pgmap v370: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:26.485 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:29:26.485 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:29:26.485 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:29:26.485 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:26 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:29:26.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:26 smithi086 ceph-mon[36053]: pgmap v370: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:26.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:29:26.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:29:26.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:29:26.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:26 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:29:26.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:26 smithi154 ceph-mon[36428]: pgmap v370: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:26.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:29:26.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:29:26.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:29:26.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:26 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:29:27.105 INFO:teuthology.orchestra.run.smithi045.stdout:167503724615 2024-06-06T04:29:27.105 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-06T04:29:27.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.625+0000 7fbe8aba1700 1 -- 172.21.15.45:0/2055641076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe84103f10 msgr2=0x7fbe8410c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:27.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.625+0000 7fbe8aba1700 1 --2- 172.21.15.45:0/2055641076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe84103f10 0x7fbe8410c400 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009990 tx=0x7fbe74009ca0 comp rx=0 tx=0).stop 2024-06-06T04:29:27.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.625+0000 7fbe8aba1700 1 -- 172.21.15.45:0/2055641076 shutdown_connections 2024-06-06T04:29:27.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.625+0000 7fbe8aba1700 1 --2- 172.21.15.45:0/2055641076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8410ca20 0x7fbe8410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:27.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.625+0000 7fbe8aba1700 1 --2- 172.21.15.45:0/2055641076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe841035f0 0x7fbe841039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:27.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.625+0000 7fbe8aba1700 1 --2- 172.21.15.45:0/2055641076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe84103f10 0x7fbe8410c400 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:27.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.625+0000 7fbe8aba1700 1 -- 172.21.15.45:0/2055641076 >> 172.21.15.45:0/2055641076 conn(0x7fbe840fce40 msgr2=0x7fbe840ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:27.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.625+0000 7fbe8aba1700 1 -- 172.21.15.45:0/2055641076 shutdown_connections 2024-06-06T04:29:27.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.625+0000 7fbe8aba1700 1 -- 172.21.15.45:0/2055641076 wait complete. 2024-06-06T04:29:27.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.626+0000 7fbe8aba1700 1 Processor -- start 2024-06-06T04:29:27.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.626+0000 7fbe8aba1700 1 -- start start 2024-06-06T04:29:27.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.626+0000 7fbe8aba1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe841035f0 0x7fbe8419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:27.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.626+0000 7fbe8aba1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe84103f10 0x7fbe8419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:27.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.626+0000 7fbe8aba1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8410ca20 0x7fbe8419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:27.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.626+0000 7fbe8aba1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbe841126f0 con 0x7fbe841035f0 2024-06-06T04:29:27.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.626+0000 7fbe8aba1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbe84112570 con 0x7fbe84103f10 2024-06-06T04:29:27.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.626+0000 7fbe8aba1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbe84112870 con 0x7fbe8410ca20 2024-06-06T04:29:27.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.626+0000 7fbe89b9f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe841035f0 0x7fbe8419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:27.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.627+0000 7fbe89b9f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe841035f0 0x7fbe8419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39474/0 (socket says 172.21.15.45:39474) 2024-06-06T04:29:27.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.627+0000 7fbe89b9f700 1 -- 172.21.15.45:0/4230221255 learned_addr learned my addr 172.21.15.45:0/4230221255 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:27.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.627+0000 7fbe8939e700 1 --2- 172.21.15.45:0/4230221255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe84103f10 0x7fbe8419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:27.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.627+0000 7fbe8a3a0700 1 --2- 172.21.15.45:0/4230221255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8410ca20 0x7fbe8419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:27.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.627+0000 7fbe89b9f700 1 -- 172.21.15.45:0/4230221255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8410ca20 msgr2=0x7fbe8419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:27.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.627+0000 7fbe89b9f700 1 --2- 172.21.15.45:0/4230221255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8410ca20 0x7fbe8419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:27.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.627+0000 7fbe89b9f700 1 -- 172.21.15.45:0/4230221255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe84103f10 msgr2=0x7fbe8419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:27.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.627+0000 7fbe89b9f700 1 --2- 172.21.15.45:0/4230221255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe84103f10 0x7fbe8419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:27.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.627+0000 7fbe89b9f700 1 -- 172.21.15.45:0/4230221255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8419fe30 con 0x7fbe841035f0 2024-06-06T04:29:27.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.627+0000 7fbe8a3a0700 1 --2- 172.21.15.45:0/4230221255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8410ca20 0x7fbe8419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:29:27.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.627+0000 7fbe89b9f700 1 --2- 172.21.15.45:0/4230221255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe841035f0 0x7fbe8419ae20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbe8000b530 tx=0x7fbe8000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:27.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.627+0000 7fbe7affd700 1 -- 172.21.15.45:0/4230221255 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80015020 con 0x7fbe841035f0 2024-06-06T04:29:27.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.627+0000 7fbe7affd700 1 -- 172.21.15.45:0/4230221255 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe80011ce0 con 0x7fbe841035f0 2024-06-06T04:29:27.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.627+0000 7fbe8aba1700 1 -- 172.21.15.45:0/4230221255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe841a0120 con 0x7fbe841035f0 2024-06-06T04:29:27.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.627+0000 7fbe7affd700 1 -- 172.21.15.45:0/4230221255 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80010e40 con 0x7fbe841035f0 2024-06-06T04:29:27.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.627+0000 7fbe8aba1700 1 -- 172.21.15.45:0/4230221255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe841a79a0 con 0x7fbe841035f0 2024-06-06T04:29:27.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.628+0000 7fbe8aba1700 1 -- 172.21.15.45:0/4230221255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe84102cd0 con 0x7fbe841035f0 2024-06-06T04:29:27.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.630+0000 7fbe7affd700 1 -- 172.21.15.45:0/4230221255 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbe80018020 con 0x7fbe841035f0 2024-06-06T04:29:27.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.630+0000 7fbe7affd700 1 --2- 172.21.15.45:0/4230221255 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbe700a2d30 0x7fbe700a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:27.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.630+0000 7fbe8939e700 1 --2- 172.21.15.45:0/4230221255 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbe700a2d30 0x7fbe700a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:27.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.630+0000 7fbe7affd700 1 -- 172.21.15.45:0/4230221255 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fbe800cf330 con 0x7fbe841035f0 2024-06-06T04:29:27.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.631+0000 7fbe7affd700 1 --2- 172.21.15.45:0/4230221255 >> [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135] conn(0x7fbe700ae010 0x7fbe700b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:27.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.631+0000 7fbe7affd700 1 -- 172.21.15.45:0/4230221255 --> [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe700b0b80 con 0x7fbe700ae010 2024-06-06T04:29:27.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.631+0000 7fbe7affd700 1 -- 172.21.15.45:0/4230221255 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fbe800cf780 con 0x7fbe841035f0 2024-06-06T04:29:27.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.631+0000 7fbe8a3a0700 1 --2- 172.21.15.45:0/4230221255 >> [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135] conn(0x7fbe700ae010 0x7fbe700b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:27.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.631+0000 7fbe8939e700 1 --2- 172.21.15.45:0/4230221255 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbe700a2d30 0x7fbe700a51f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbe74006010 tx=0x7fbe74005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:27.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.631+0000 7fbe8a3a0700 1 --2- 172.21.15.45:0/4230221255 >> [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135] conn(0x7fbe700ae010 0x7fbe700b04b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:27.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.632+0000 7fbe7affd700 1 -- 172.21.15.45:0/4230221255 <== osd.3 v2:172.21.15.45:6810/3436753135 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe700b0b80 con 0x7fbe700ae010 2024-06-06T04:29:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.648+0000 7fbe8aba1700 1 -- 172.21.15.45:0/4230221255 --> [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbe8406b2e0 con 0x7fbe700ae010 2024-06-06T04:29:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.649+0000 7fbe7affd700 1 -- 172.21.15.45:0/4230221255 <== osd.3 v2:172.21.15.45:6810/3436753135 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbe8406b2e0 con 0x7fbe700ae010 2024-06-06T04:29:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.649+0000 7fbe8aba1700 1 -- 172.21.15.45:0/4230221255 >> [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135] conn(0x7fbe700ae010 msgr2=0x7fbe700b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.649+0000 7fbe8aba1700 1 --2- 172.21.15.45:0/4230221255 >> [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135] conn(0x7fbe700ae010 0x7fbe700b04b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.649+0000 7fbe8aba1700 1 -- 172.21.15.45:0/4230221255 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbe700a2d30 msgr2=0x7fbe700a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.649+0000 7fbe8aba1700 1 --2- 172.21.15.45:0/4230221255 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbe700a2d30 0x7fbe700a51f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbe74006010 tx=0x7fbe74005960 comp rx=0 tx=0).stop 2024-06-06T04:29:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.649+0000 7fbe8aba1700 1 -- 172.21.15.45:0/4230221255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe841035f0 msgr2=0x7fbe8419ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.649+0000 7fbe8aba1700 1 --2- 172.21.15.45:0/4230221255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe841035f0 0x7fbe8419ae20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbe8000b530 tx=0x7fbe8000b840 comp rx=0 tx=0).stop 2024-06-06T04:29:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.649+0000 7fbe8a3a0700 1 -- 172.21.15.45:0/4230221255 reap_dead start 2024-06-06T04:29:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.649+0000 7fbe8aba1700 1 -- 172.21.15.45:0/4230221255 shutdown_connections 2024-06-06T04:29:27.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.649+0000 7fbe8aba1700 1 -- 172.21.15.45:0/4230221255 >> 172.21.15.45:0/4230221255 conn(0x7fbe840fce40 msgr2=0x7fbe8410dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:27.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.649+0000 7fbe8aba1700 1 -- 172.21.15.45:0/4230221255 shutdown_connections 2024-06-06T04:29:27.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:27.649+0000 7fbe8aba1700 1 -- 172.21.15.45:0/4230221255 wait complete. 2024-06-06T04:29:28.005 INFO:teuthology.orchestra.run.smithi045.stdout:257698037816 2024-06-06T04:29:28.006 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-06-06T04:29:28.564 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:28 smithi045 ceph-mon[32299]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T04:29:28.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:28 smithi086 ceph-mon[36053]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T04:29:28.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:28 smithi154 ceph-mon[36428]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-06T04:29:30.553 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:30 smithi045 ceph-mon[32299]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:29:30.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.923+0000 7fb136af1700 1 -- 172.21.15.45:0/3774750179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1301030f0 msgr2=0x7fb130107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:30.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.923+0000 7fb136af1700 1 --2- 172.21.15.45:0/3774750179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1301030f0 0x7fb130107110 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0099c0 tx=0x7fb11c009cd0 comp rx=0 tx=0).stop 2024-06-06T04:29:30.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.923+0000 7fb136af1700 1 -- 172.21.15.45:0/3774750179 shutdown_connections 2024-06-06T04:29:30.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.923+0000 7fb136af1700 1 --2- 172.21.15.45:0/3774750179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb130107840 0x7fb13010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:30.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.923+0000 7fb136af1700 1 --2- 172.21.15.45:0/3774750179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb130102740 0x7fb130102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:30.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.923+0000 7fb136af1700 1 --2- 172.21.15.45:0/3774750179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1301030f0 0x7fb130107110 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:30.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.923+0000 7fb136af1700 1 -- 172.21.15.45:0/3774750179 >> 172.21.15.45:0/3774750179 conn(0x7fb1300795f0 msgr2=0x7fb130079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:30.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.923+0000 7fb136af1700 1 -- 172.21.15.45:0/3774750179 shutdown_connections 2024-06-06T04:29:30.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.923+0000 7fb136af1700 1 -- 172.21.15.45:0/3774750179 wait complete. 2024-06-06T04:29:30.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.924+0000 7fb136af1700 1 Processor -- start 2024-06-06T04:29:30.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.924+0000 7fb136af1700 1 -- start start 2024-06-06T04:29:30.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb136af1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb130102740 0x7fb130074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:30.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb136af1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1301030f0 0x7fb130074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:30.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb136af1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb130107840 0x7fb130075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:30.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb136af1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb130112600 con 0x7fb130102740 2024-06-06T04:29:30.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb136af1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb130112480 con 0x7fb1301030f0 2024-06-06T04:29:30.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb136af1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb130112780 con 0x7fb130107840 2024-06-06T04:29:30.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb13488d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb130102740 0x7fb130074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:30.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb13488d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb130102740 0x7fb130074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33804/0 (socket says 172.21.15.45:33804) 2024-06-06T04:29:30.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb13488d700 1 -- 172.21.15.45:0/4170925164 learned_addr learned my addr 172.21.15.45:0/4170925164 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:30.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb12ffff700 1 --2- 172.21.15.45:0/4170925164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1301030f0 0x7fb130074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:30.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb13508e700 1 --2- 172.21.15.45:0/4170925164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb130107840 0x7fb130075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:30.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb13488d700 1 -- 172.21.15.45:0/4170925164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb130107840 msgr2=0x7fb130075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:30.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb13488d700 1 --2- 172.21.15.45:0/4170925164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb130107840 0x7fb130075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:30.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb13488d700 1 -- 172.21.15.45:0/4170925164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1301030f0 msgr2=0x7fb130074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:30.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb13488d700 1 --2- 172.21.15.45:0/4170925164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1301030f0 0x7fb130074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:30.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb13488d700 1 -- 172.21.15.45:0/4170925164 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1301a7020 con 0x7fb130102740 2024-06-06T04:29:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.925+0000 7fb13488d700 1 --2- 172.21.15.45:0/4170925164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb130102740 0x7fb130074790 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb12400d950 tx=0x7fb12400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.926+0000 7fb12dffb700 1 -- 172.21.15.45:0/4170925164 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124016070 con 0x7fb130102740 2024-06-06T04:29:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.926+0000 7fb12dffb700 1 -- 172.21.15.45:0/4170925164 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb12400fd40 con 0x7fb130102740 2024-06-06T04:29:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.926+0000 7fb136af1700 1 -- 172.21.15.45:0/4170925164 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1301a7310 con 0x7fb130102740 2024-06-06T04:29:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.926+0000 7fb12dffb700 1 -- 172.21.15.45:0/4170925164 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12400e3b0 con 0x7fb130102740 2024-06-06T04:29:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.926+0000 7fb136af1700 1 -- 172.21.15.45:0/4170925164 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1301a7870 con 0x7fb130102740 2024-06-06T04:29:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.927+0000 7fb136af1700 1 -- 172.21.15.45:0/4170925164 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb130062c20 con 0x7fb130102740 2024-06-06T04:29:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.928+0000 7fb12dffb700 1 -- 172.21.15.45:0/4170925164 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb124019070 con 0x7fb130102740 2024-06-06T04:29:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.929+0000 7fb12dffb700 1 --2- 172.21.15.45:0/4170925164 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb1200a2d30 0x7fb1200a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.929+0000 7fb12ffff700 1 --2- 172.21.15.45:0/4170925164 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb1200a2d30 0x7fb1200a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.929+0000 7fb12dffb700 1 -- 172.21.15.45:0/4170925164 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fb1240cf510 con 0x7fb130102740 2024-06-06T04:29:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.929+0000 7fb12dffb700 1 --2- 172.21.15.45:0/4170925164 >> [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005] conn(0x7fb1200ae010 0x7fb1200b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.929+0000 7fb12dffb700 1 -- 172.21.15.45:0/4170925164 --> [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1200b0b80 con 0x7fb1200ae010 2024-06-06T04:29:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.929+0000 7fb12dffb700 1 -- 172.21.15.45:0/4170925164 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fb1240cf8c0 con 0x7fb130102740 2024-06-06T04:29:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.929+0000 7fb12ffff700 1 --2- 172.21.15.45:0/4170925164 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb1200a2d30 0x7fb1200a51f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb130076220 tx=0x7fb11c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.929+0000 7fb13508e700 1 --2- 172.21.15.45:0/4170925164 >> [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005] conn(0x7fb1200ae010 0x7fb1200b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.930+0000 7fb13508e700 1 --2- 172.21.15.45:0/4170925164 >> [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005] conn(0x7fb1200ae010 0x7fb1200b04b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.931+0000 7fb12dffb700 1 -- 172.21.15.45:0/4170925164 <== osd.7 v2:172.21.15.86:6816/1119791005 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1200b0b80 con 0x7fb1200ae010 2024-06-06T04:29:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.947+0000 7fb136af1700 1 -- 172.21.15.45:0/4170925164 --> [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb130063020 con 0x7fb1200ae010 2024-06-06T04:29:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.948+0000 7fb12dffb700 1 -- 172.21.15.45:0/4170925164 <== osd.7 v2:172.21.15.86:6816/1119791005 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb130063020 con 0x7fb1200ae010 2024-06-06T04:29:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.948+0000 7fb136af1700 1 -- 172.21.15.45:0/4170925164 >> [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005] conn(0x7fb1200ae010 msgr2=0x7fb1200b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.948+0000 7fb136af1700 1 --2- 172.21.15.45:0/4170925164 >> [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005] conn(0x7fb1200ae010 0x7fb1200b04b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.948+0000 7fb136af1700 1 -- 172.21.15.45:0/4170925164 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb1200a2d30 msgr2=0x7fb1200a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.948+0000 7fb136af1700 1 --2- 172.21.15.45:0/4170925164 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb1200a2d30 0x7fb1200a51f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb130076220 tx=0x7fb11c005fb0 comp rx=0 tx=0).stop 2024-06-06T04:29:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.948+0000 7fb136af1700 1 -- 172.21.15.45:0/4170925164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb130102740 msgr2=0x7fb130074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.948+0000 7fb136af1700 1 --2- 172.21.15.45:0/4170925164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb130102740 0x7fb130074790 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb12400d950 tx=0x7fb12400dc60 comp rx=0 tx=0).stop 2024-06-06T04:29:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.948+0000 7fb13508e700 1 -- 172.21.15.45:0/4170925164 reap_dead start 2024-06-06T04:29:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.948+0000 7fb136af1700 1 -- 172.21.15.45:0/4170925164 shutdown_connections 2024-06-06T04:29:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.948+0000 7fb136af1700 1 -- 172.21.15.45:0/4170925164 >> 172.21.15.45:0/4170925164 conn(0x7fb1300795f0 msgr2=0x7fb1301007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.948+0000 7fb136af1700 1 -- 172.21.15.45:0/4170925164 shutdown_connections 2024-06-06T04:29:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:29.948+0000 7fb136af1700 1 -- 172.21.15.45:0/4170925164 wait complete. 2024-06-06T04:29:30.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:30 smithi086 ceph-mon[36053]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:29:30.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:30 smithi154 ceph-mon[36428]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:29:30.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.553+0000 7ff43b44f700 1 -- 172.21.15.45:0/2764083854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4341084e0 msgr2=0x7ff4341088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:30.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.553+0000 7ff43b44f700 1 --2- 172.21.15.45:0/2764083854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4341084e0 0x7ff4341088c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff43000b0d0 tx=0x7ff43000b3e0 comp rx=0 tx=0).stop 2024-06-06T04:29:30.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.553+0000 7ff43b44f700 1 -- 172.21.15.45:0/2764083854 shutdown_connections 2024-06-06T04:29:30.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.553+0000 7ff43b44f700 1 --2- 172.21.15.45:0/2764083854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff43406ab10 0x7ff43410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:30.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.553+0000 7ff43b44f700 1 --2- 172.21.15.45:0/2764083854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4341084e0 0x7ff4341088c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:30.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.553+0000 7ff43b44f700 1 --2- 172.21.15.45:0/2764083854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff43406a170 0x7ff43406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:30.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.553+0000 7ff43b44f700 1 -- 172.21.15.45:0/2764083854 >> 172.21.15.45:0/2764083854 conn(0x7ff4340782c0 msgr2=0x7ff4340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:30.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.553+0000 7ff43b44f700 1 -- 172.21.15.45:0/2764083854 shutdown_connections 2024-06-06T04:29:30.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.553+0000 7ff43b44f700 1 -- 172.21.15.45:0/2764083854 wait complete. 2024-06-06T04:29:30.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.554+0000 7ff43b44f700 1 Processor -- start 2024-06-06T04:29:30.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.554+0000 7ff43b44f700 1 -- start start 2024-06-06T04:29:30.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.554+0000 7ff43b44f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff43406a170 0x7ff434101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:30.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.554+0000 7ff43b44f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff43406ab10 0x7ff4341019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:30.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.554+0000 7ff43b44f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4341084e0 0x7ff434101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:30.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.554+0000 7ff43b44f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff4341125d0 con 0x7ff4341084e0 2024-06-06T04:29:30.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.554+0000 7ff43b44f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff434112450 con 0x7ff43406a170 2024-06-06T04:29:30.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.554+0000 7ff43b44f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff434112750 con 0x7ff43406ab10 2024-06-06T04:29:30.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.554+0000 7ff4399ec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4341084e0 0x7ff434101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:30.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.554+0000 7ff4399ec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4341084e0 0x7ff434101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33824/0 (socket says 172.21.15.45:33824) 2024-06-06T04:29:30.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.554+0000 7ff4399ec700 1 -- 172.21.15.45:0/1116060052 learned_addr learned my addr 172.21.15.45:0/1116060052 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:30.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.554+0000 7ff4391eb700 1 --2- 172.21.15.45:0/1116060052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff43406a170 0x7ff434101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:30.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.554+0000 7ff4389ea700 1 --2- 172.21.15.45:0/1116060052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff43406ab10 0x7ff4341019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:30.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.555+0000 7ff4399ec700 1 -- 172.21.15.45:0/1116060052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff43406ab10 msgr2=0x7ff4341019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:30.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.555+0000 7ff4399ec700 1 --2- 172.21.15.45:0/1116060052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff43406ab10 0x7ff4341019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:30.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.555+0000 7ff4399ec700 1 -- 172.21.15.45:0/1116060052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff43406a170 msgr2=0x7ff434101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:30.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.555+0000 7ff4399ec700 1 --2- 172.21.15.45:0/1116060052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff43406a170 0x7ff434101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:30.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.555+0000 7ff4399ec700 1 -- 172.21.15.45:0/1116060052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4341a70b0 con 0x7ff4341084e0 2024-06-06T04:29:30.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.555+0000 7ff4389ea700 1 --2- 172.21.15.45:0/1116060052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff43406ab10 0x7ff4341019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:29:30.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.555+0000 7ff4399ec700 1 --2- 172.21.15.45:0/1116060052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4341084e0 0x7ff434101ee0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00d980 tx=0x7ff42c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:30.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.555+0000 7ff42a7fc700 1 -- 172.21.15.45:0/1116060052 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c017070 con 0x7ff4341084e0 2024-06-06T04:29:30.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.555+0000 7ff42a7fc700 1 -- 172.21.15.45:0/1116060052 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff42c005020 con 0x7ff4341084e0 2024-06-06T04:29:30.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.555+0000 7ff43b44f700 1 -- 172.21.15.45:0/1116060052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4341a73a0 con 0x7ff4341084e0 2024-06-06T04:29:30.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.555+0000 7ff42a7fc700 1 -- 172.21.15.45:0/1116060052 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c0099b0 con 0x7ff4341084e0 2024-06-06T04:29:30.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.555+0000 7ff43b44f700 1 -- 172.21.15.45:0/1116060052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4341a78e0 con 0x7ff4341084e0 2024-06-06T04:29:30.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.556+0000 7ff43b44f700 1 -- 172.21.15.45:0/1116060052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff434062ba0 con 0x7ff4341084e0 2024-06-06T04:29:30.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.558+0000 7ff42a7fc700 1 -- 172.21.15.45:0/1116060052 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff42c0051d0 con 0x7ff4341084e0 2024-06-06T04:29:30.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.559+0000 7ff42a7fc700 1 --2- 172.21.15.45:0/1116060052 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff4200a2cb0 0x7ff4200a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:30.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.559+0000 7ff4391eb700 1 --2- 172.21.15.45:0/1116060052 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff4200a2cb0 0x7ff4200a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:30.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.559+0000 7ff42a7fc700 1 -- 172.21.15.45:0/1116060052 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7ff42c0cf900 con 0x7ff4341084e0 2024-06-06T04:29:30.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.559+0000 7ff42a7fc700 1 --2- 172.21.15.45:0/1116060052 >> [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250] conn(0x7ff4200adf90 0x7ff4200b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:30.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.559+0000 7ff42a7fc700 1 -- 172.21.15.45:0/1116060052 --> [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff4200b0b00 con 0x7ff4200adf90 2024-06-06T04:29:30.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.559+0000 7ff42a7fc700 1 -- 172.21.15.45:0/1116060052 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7ff42c0cfcb0 con 0x7ff4341084e0 2024-06-06T04:29:30.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.559+0000 7ff4389ea700 1 --2- 172.21.15.45:0/1116060052 >> [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250] conn(0x7ff4200adf90 0x7ff4200b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:30.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.559+0000 7ff4391eb700 1 --2- 172.21.15.45:0/1116060052 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff4200a2cb0 0x7ff4200a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff430009fd0 tx=0x7ff43000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:30.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.560+0000 7ff4389ea700 1 --2- 172.21.15.45:0/1116060052 >> [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250] conn(0x7ff4200adf90 0x7ff4200b0430 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:30.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.560+0000 7ff42a7fc700 1 -- 172.21.15.45:0/1116060052 <== osd.0 v2:172.21.15.45:6802/2111759250 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff4200b0b00 con 0x7ff4200adf90 2024-06-06T04:29:30.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.579+0000 7ff43b44f700 1 -- 172.21.15.45:0/1116060052 --> [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff434062fa0 con 0x7ff4200adf90 2024-06-06T04:29:30.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.580+0000 7ff42a7fc700 1 -- 172.21.15.45:0/1116060052 <== osd.0 v2:172.21.15.45:6802/2111759250 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff434062fa0 con 0x7ff4200adf90 2024-06-06T04:29:30.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.580+0000 7ff43b44f700 1 -- 172.21.15.45:0/1116060052 >> [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250] conn(0x7ff4200adf90 msgr2=0x7ff4200b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:30.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.580+0000 7ff43b44f700 1 --2- 172.21.15.45:0/1116060052 >> [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250] conn(0x7ff4200adf90 0x7ff4200b0430 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:30.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.580+0000 7ff43b44f700 1 -- 172.21.15.45:0/1116060052 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff4200a2cb0 msgr2=0x7ff4200a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.580+0000 7ff43b44f700 1 --2- 172.21.15.45:0/1116060052 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff4200a2cb0 0x7ff4200a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff430009fd0 tx=0x7ff43000b040 comp rx=0 tx=0).stop 2024-06-06T04:29:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.580+0000 7ff43b44f700 1 -- 172.21.15.45:0/1116060052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4341084e0 msgr2=0x7ff434101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.580+0000 7ff43b44f700 1 --2- 172.21.15.45:0/1116060052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4341084e0 0x7ff434101ee0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00d980 tx=0x7ff42c00dc90 comp rx=0 tx=0).stop 2024-06-06T04:29:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.580+0000 7ff4399ec700 1 -- 172.21.15.45:0/1116060052 reap_dead start 2024-06-06T04:29:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.580+0000 7ff43b44f700 1 -- 172.21.15.45:0/1116060052 shutdown_connections 2024-06-06T04:29:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.580+0000 7ff43b44f700 1 -- 172.21.15.45:0/1116060052 >> 172.21.15.45:0/1116060052 conn(0x7ff4340782c0 msgr2=0x7ff4340ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.580+0000 7ff43b44f700 1 -- 172.21.15.45:0/1116060052 shutdown_connections 2024-06-06T04:29:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:30.580+0000 7ff43b44f700 1 -- 172.21.15.45:0/1116060052 wait complete. 2024-06-06T04:29:32.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:32 smithi045 ceph-mon[32299]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:32.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:32 smithi086 ceph-mon[36053]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:32.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:32 smithi154 ceph-mon[36428]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:32.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.748+0000 7f0b1727f700 1 -- 172.21.15.45:0/1755425075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b10101390 msgr2=0x7f0b10101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:32.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.748+0000 7f0b1727f700 1 --2- 172.21.15.45:0/1755425075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b10101390 0x7f0b10101810 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009990 tx=0x7f0b0c009ca0 comp rx=0 tx=0).stop 2024-06-06T04:29:32.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.748+0000 7f0b1727f700 1 -- 172.21.15.45:0/1755425075 shutdown_connections 2024-06-06T04:29:32.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.748+0000 7f0b1727f700 1 --2- 172.21.15.45:0/1755425075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b10101d50 0x7f0b1010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:32.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.748+0000 7f0b1727f700 1 --2- 172.21.15.45:0/1755425075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b10107400 0x7f0b101077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:32.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.748+0000 7f0b1727f700 1 --2- 172.21.15.45:0/1755425075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b10101390 0x7f0b10101810 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.748+0000 7f0b1727f700 1 -- 172.21.15.45:0/1755425075 >> 172.21.15.45:0/1755425075 conn(0x7f0b10078340 msgr2=0x7f0b10078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.748+0000 7f0b1727f700 1 -- 172.21.15.45:0/1755425075 shutdown_connections 2024-06-06T04:29:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.748+0000 7f0b1727f700 1 -- 172.21.15.45:0/1755425075 wait complete. 2024-06-06T04:29:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.749+0000 7f0b1727f700 1 Processor -- start 2024-06-06T04:29:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.749+0000 7f0b1727f700 1 -- start start 2024-06-06T04:29:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.749+0000 7f0b1727f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b10101390 0x7f0b1019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.749+0000 7f0b1727f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b10101d50 0x7f0b1019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.749+0000 7f0b1727f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b10107400 0x7f0b10198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:32.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.749+0000 7f0b1727f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0b10111540 con 0x7f0b10107400 2024-06-06T04:29:32.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.749+0000 7f0b1727f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101113c0 con 0x7f0b10101390 2024-06-06T04:29:32.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.749+0000 7f0b1727f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101116c0 con 0x7f0b10101d50 2024-06-06T04:29:32.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.749+0000 7f0b1581c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b10107400 0x7f0b10198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:32.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.749+0000 7f0b1581c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b10107400 0x7f0b10198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33832/0 (socket says 172.21.15.45:33832) 2024-06-06T04:29:32.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.749+0000 7f0b1581c700 1 -- 172.21.15.45:0/3974956668 learned_addr learned my addr 172.21.15.45:0/3974956668 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:32.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.750+0000 7f0b1481a700 1 --2- 172.21.15.45:0/3974956668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b10101d50 0x7f0b1019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:32.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.750+0000 7f0b1501b700 1 --2- 172.21.15.45:0/3974956668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b10101390 0x7f0b1019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:32.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.750+0000 7f0b1581c700 1 -- 172.21.15.45:0/3974956668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b10101d50 msgr2=0x7f0b1019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:32.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.750+0000 7f0b1581c700 1 --2- 172.21.15.45:0/3974956668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0b10101d50 0x7f0b1019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:32.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.750+0000 7f0b1581c700 1 -- 172.21.15.45:0/3974956668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b10101390 msgr2=0x7f0b1019e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:32.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.750+0000 7f0b1581c700 1 --2- 172.21.15.45:0/3974956668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b10101390 0x7f0b1019e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:32.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.750+0000 7f0b1581c700 1 -- 172.21.15.45:0/3974956668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b10199060 con 0x7f0b10107400 2024-06-06T04:29:32.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.750+0000 7f0b1501b700 1 --2- 172.21.15.45:0/3974956668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b10101390 0x7f0b1019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:29:32.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.750+0000 7f0b1581c700 1 --2- 172.21.15.45:0/3974956668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b10107400 0x7f0b10198900 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0b08007d70 tx=0x7f0b0800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:32.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.750+0000 7f0b027fc700 1 -- 172.21.15.45:0/3974956668 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08016020 con 0x7f0b10107400 2024-06-06T04:29:32.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.750+0000 7f0b027fc700 1 -- 172.21.15.45:0/3974956668 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b0800ad40 con 0x7f0b10107400 2024-06-06T04:29:32.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.750+0000 7f0b1727f700 1 -- 172.21.15.45:0/3974956668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b10199350 con 0x7f0b10107400 2024-06-06T04:29:32.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.750+0000 7f0b027fc700 1 -- 172.21.15.45:0/3974956668 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b080115e0 con 0x7f0b10107400 2024-06-06T04:29:32.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.750+0000 7f0b1727f700 1 -- 172.21.15.45:0/3974956668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b101a5610 con 0x7f0b10107400 2024-06-06T04:29:32.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.751+0000 7f0b1727f700 1 -- 172.21.15.45:0/3974956668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b10105500 con 0x7f0b10107400 2024-06-06T04:29:32.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.752+0000 7f0b027fc700 1 -- 172.21.15.45:0/3974956668 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0b08019020 con 0x7f0b10107400 2024-06-06T04:29:32.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.753+0000 7f0b027fc700 1 --2- 172.21.15.45:0/3974956668 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0afc0a2d30 0x7f0afc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:32.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.753+0000 7f0b1501b700 1 --2- 172.21.15.45:0/3974956668 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0afc0a2d30 0x7f0afc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:32.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.753+0000 7f0b027fc700 1 -- 172.21.15.45:0/3974956668 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f0b080cf3e0 con 0x7f0b10107400 2024-06-06T04:29:32.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.753+0000 7f0b027fc700 1 --2- 172.21.15.45:0/3974956668 >> [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877] conn(0x7f0afc0ae010 0x7f0afc0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:32.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.753+0000 7f0b027fc700 1 -- 172.21.15.45:0/3974956668 --> [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0afc0b0b80 con 0x7f0afc0ae010 2024-06-06T04:29:32.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.753+0000 7f0b027fc700 1 -- 172.21.15.45:0/3974956668 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f0b080d6050 con 0x7f0b10107400 2024-06-06T04:29:32.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.754+0000 7f0b1501b700 1 --2- 172.21.15.45:0/3974956668 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0afc0a2d30 0x7f0afc0a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0b04005950 tx=0x7f0b0400a5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:32.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.754+0000 7f0b1481a700 1 --2- 172.21.15.45:0/3974956668 >> [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877] conn(0x7f0afc0ae010 0x7f0afc0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:32.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.754+0000 7f0b1481a700 1 --2- 172.21.15.45:0/3974956668 >> [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877] conn(0x7f0afc0ae010 0x7f0afc0b04b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:32.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.755+0000 7f0b027fc700 1 -- 172.21.15.45:0/3974956668 <== osd.10 v2:172.21.15.86:6824/1701691877 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0afc0b0b80 con 0x7f0afc0ae010 2024-06-06T04:29:32.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.771+0000 7f0b1727f700 1 -- 172.21.15.45:0/3974956668 --> [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b10102ab0 con 0x7f0afc0ae010 2024-06-06T04:29:32.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.772+0000 7f0b027fc700 1 -- 172.21.15.45:0/3974956668 <== osd.10 v2:172.21.15.86:6824/1701691877 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0b10102ab0 con 0x7f0afc0ae010 2024-06-06T04:29:32.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.772+0000 7f0b1727f700 1 -- 172.21.15.45:0/3974956668 >> [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877] conn(0x7f0afc0ae010 msgr2=0x7f0afc0b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:32.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.772+0000 7f0b1727f700 1 --2- 172.21.15.45:0/3974956668 >> [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877] conn(0x7f0afc0ae010 0x7f0afc0b04b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:32.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.772+0000 7f0b1727f700 1 -- 172.21.15.45:0/3974956668 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0afc0a2d30 msgr2=0x7f0afc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:32.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.772+0000 7f0b1727f700 1 --2- 172.21.15.45:0/3974956668 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0afc0a2d30 0x7f0afc0a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0b04005950 tx=0x7f0b0400a5b0 comp rx=0 tx=0).stop 2024-06-06T04:29:32.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.772+0000 7f0b1727f700 1 -- 172.21.15.45:0/3974956668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b10107400 msgr2=0x7f0b10198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:32.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.772+0000 7f0b1727f700 1 --2- 172.21.15.45:0/3974956668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b10107400 0x7f0b10198900 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0b08007d70 tx=0x7f0b0800a340 comp rx=0 tx=0).stop 2024-06-06T04:29:32.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.772+0000 7f0b1581c700 1 -- 172.21.15.45:0/3974956668 reap_dead start 2024-06-06T04:29:32.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.773+0000 7f0b1727f700 1 -- 172.21.15.45:0/3974956668 shutdown_connections 2024-06-06T04:29:32.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.773+0000 7f0b1727f700 1 -- 172.21.15.45:0/3974956668 >> 172.21.15.45:0/3974956668 conn(0x7f0b10078340 msgr2=0x7f0b10100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:32.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.773+0000 7f0b1727f700 1 -- 172.21.15.45:0/3974956668 shutdown_connections 2024-06-06T04:29:32.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.773+0000 7f0b1727f700 1 -- 172.21.15.45:0/3974956668 wait complete. 2024-06-06T04:29:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.988+0000 7f43c704c700 1 -- 172.21.15.45:0/737616298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c01013c0 msgr2=0x7f43c0101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:32.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.988+0000 7f43c704c700 1 --2- 172.21.15.45:0/737616298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c01013c0 0x7f43c0101840 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009990 tx=0x7f43ac009ca0 comp rx=0 tx=0).stop 2024-06-06T04:29:32.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.989+0000 7f43c704c700 1 -- 172.21.15.45:0/737616298 shutdown_connections 2024-06-06T04:29:32.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.989+0000 7f43c704c700 1 --2- 172.21.15.45:0/737616298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c0101d80 0x7f43c010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:32.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.989+0000 7f43c704c700 1 --2- 172.21.15.45:0/737616298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43c0107440 0x7f43c0107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:32.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.989+0000 7f43c704c700 1 --2- 172.21.15.45:0/737616298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c01013c0 0x7f43c0101840 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:32.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.989+0000 7f43c704c700 1 -- 172.21.15.45:0/737616298 >> 172.21.15.45:0/737616298 conn(0x7f43c0077400 msgr2=0x7f43c0077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:32.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.989+0000 7f43c704c700 1 -- 172.21.15.45:0/737616298 shutdown_connections 2024-06-06T04:29:32.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.989+0000 7f43c704c700 1 -- 172.21.15.45:0/737616298 wait complete. 2024-06-06T04:29:32.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.989+0000 7f43c704c700 1 Processor -- start 2024-06-06T04:29:32.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.989+0000 7f43c704c700 1 -- start start 2024-06-06T04:29:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c704c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43c01013c0 0x7f43c019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c704c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c0101d80 0x7f43c019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c704c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c0107440 0x7f43c0196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c704c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f43c01104c0 con 0x7f43c0107440 2024-06-06T04:29:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c704c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0110340 con 0x7f43c0101d80 2024-06-06T04:29:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c704c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0110640 con 0x7f43c01013c0 2024-06-06T04:29:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c55e9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c0107440 0x7f43c0196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c55e9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c0107440 0x7f43c0196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33842/0 (socket says 172.21.15.45:33842) 2024-06-06T04:29:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c55e9700 1 -- 172.21.15.45:0/4236911610 learned_addr learned my addr 172.21.15.45:0/4236911610 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43bffff700 1 --2- 172.21.15.45:0/4236911610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c0101d80 0x7f43c019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c4de8700 1 --2- 172.21.15.45:0/4236911610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43c01013c0 0x7f43c019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c55e9700 1 -- 172.21.15.45:0/4236911610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43c01013c0 msgr2=0x7f43c019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c55e9700 1 --2- 172.21.15.45:0/4236911610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43c01013c0 0x7f43c019c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c55e9700 1 -- 172.21.15.45:0/4236911610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c0101d80 msgr2=0x7f43c019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:32.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c55e9700 1 --2- 172.21.15.45:0/4236911610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c0101d80 0x7f43c019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:32.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c55e9700 1 -- 172.21.15.45:0/4236911610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c0197020 con 0x7f43c0107440 2024-06-06T04:29:32.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c4de8700 1 --2- 172.21.15.45:0/4236911610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43c01013c0 0x7f43c019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:29:32.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43c55e9700 1 --2- 172.21.15.45:0/4236911610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c0107440 0x7f43c0196740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f43b8007ee0 tx=0x7f43b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:32.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43bdffb700 1 -- 172.21.15.45:0/4236911610 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b800dbd0 con 0x7f43c0107440 2024-06-06T04:29:32.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.990+0000 7f43bdffb700 1 -- 172.21.15.45:0/4236911610 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43b800dd70 con 0x7f43c0107440 2024-06-06T04:29:32.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.991+0000 7f43c704c700 1 -- 172.21.15.45:0/4236911610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c0197310 con 0x7f43c0107440 2024-06-06T04:29:32.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.991+0000 7f43bdffb700 1 -- 172.21.15.45:0/4236911610 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8005990 con 0x7f43c0107440 2024-06-06T04:29:32.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.991+0000 7f43c704c700 1 -- 172.21.15.45:0/4236911610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c01a35f0 con 0x7f43c0107440 2024-06-06T04:29:32.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.991+0000 7f43c704c700 1 -- 172.21.15.45:0/4236911610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f43c0062c20 con 0x7f43c0107440 2024-06-06T04:29:32.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.993+0000 7f43bdffb700 1 -- 172.21.15.45:0/4236911610 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f43b801a020 con 0x7f43c0107440 2024-06-06T04:29:32.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.993+0000 7f43bdffb700 1 --2- 172.21.15.45:0/4236911610 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43b00a2d30 0x7f43b00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:32.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.994+0000 7f43c4de8700 1 --2- 172.21.15.45:0/4236911610 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43b00a2d30 0x7f43b00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:32.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.994+0000 7f43bdffb700 1 -- 172.21.15.45:0/4236911610 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f43b80cf330 con 0x7f43c0107440 2024-06-06T04:29:32.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.994+0000 7f43bdffb700 1 --2- 172.21.15.45:0/4236911610 >> [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796] conn(0x7f43b00ae010 0x7f43b00b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:32.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.994+0000 7f43bdffb700 1 -- 172.21.15.45:0/4236911610 --> [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f43b00b0b80 con 0x7f43b00ae010 2024-06-06T04:29:32.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.994+0000 7f43bdffb700 1 -- 172.21.15.45:0/4236911610 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f43b80cf780 con 0x7f43c0107440 2024-06-06T04:29:32.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.994+0000 7f43c4de8700 1 --2- 172.21.15.45:0/4236911610 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43b00a2d30 0x7f43b00a51f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f43b400be40 tx=0x7f43b400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:32.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.994+0000 7f43bffff700 1 --2- 172.21.15.45:0/4236911610 >> [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796] conn(0x7f43b00ae010 0x7f43b00b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:32.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.995+0000 7f43bffff700 1 --2- 172.21.15.45:0/4236911610 >> [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796] conn(0x7f43b00ae010 0x7f43b00b04b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:32.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:32.995+0000 7f43bdffb700 1 -- 172.21.15.45:0/4236911610 <== osd.4 v2:172.21.15.86:6808/3030235796 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f43b00b0b80 con 0x7f43b00ae010 2024-06-06T04:29:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:33.012+0000 7f43c704c700 1 -- 172.21.15.45:0/4236911610 --> [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f43c00630d0 con 0x7f43b00ae010 2024-06-06T04:29:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:33.013+0000 7f43bdffb700 1 -- 172.21.15.45:0/4236911610 <== osd.4 v2:172.21.15.86:6808/3030235796 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f43c00630d0 con 0x7f43b00ae010 2024-06-06T04:29:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:33.013+0000 7f43c704c700 1 -- 172.21.15.45:0/4236911610 >> [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796] conn(0x7f43b00ae010 msgr2=0x7f43b00b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:33.013+0000 7f43c704c700 1 --2- 172.21.15.45:0/4236911610 >> [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796] conn(0x7f43b00ae010 0x7f43b00b04b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:33.013+0000 7f43c704c700 1 -- 172.21.15.45:0/4236911610 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43b00a2d30 msgr2=0x7f43b00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:33.013+0000 7f43c704c700 1 --2- 172.21.15.45:0/4236911610 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43b00a2d30 0x7f43b00a51f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f43b400be40 tx=0x7f43b400d040 comp rx=0 tx=0).stop 2024-06-06T04:29:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:33.013+0000 7f43c704c700 1 -- 172.21.15.45:0/4236911610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c0107440 msgr2=0x7f43c0196740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:33.013+0000 7f43c704c700 1 --2- 172.21.15.45:0/4236911610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c0107440 0x7f43c0196740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f43b8007ee0 tx=0x7f43b800d3b0 comp rx=0 tx=0).stop 2024-06-06T04:29:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:33.013+0000 7f43c55e9700 1 -- 172.21.15.45:0/4236911610 reap_dead start 2024-06-06T04:29:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:33.013+0000 7f43c704c700 1 -- 172.21.15.45:0/4236911610 shutdown_connections 2024-06-06T04:29:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:33.013+0000 7f43c704c700 1 -- 172.21.15.45:0/4236911610 >> 172.21.15.45:0/4236911610 conn(0x7f43c0077400 msgr2=0x7f43c0100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:33.014+0000 7f43c704c700 1 -- 172.21.15.45:0/4236911610 shutdown_connections 2024-06-06T04:29:33.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:33.014+0000 7f43c704c700 1 -- 172.21.15.45:0/4236911610 wait complete. 2024-06-06T04:29:33.881 INFO:teuthology.orchestra.run.smithi045.stdout:313532612656 2024-06-06T04:29:33.882 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-06-06T04:29:34.294 INFO:teuthology.orchestra.run.smithi045.stdout:223338299454 2024-06-06T04:29:34.295 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-06T04:29:34.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:34 smithi045 ceph-mon[32299]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:29:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:34 smithi086 ceph-mon[36053]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:29:34.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:34 smithi154 ceph-mon[36428]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:29:34.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.836+0000 7f9ad9320700 1 -- 172.21.15.45:0/1253780449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ad41073d0 msgr2=0x7f9ad41077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:34.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.836+0000 7f9ad9320700 1 --2- 172.21.15.45:0/1253780449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ad41073d0 0x7f9ad41077b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009990 tx=0x7f9ac4009ca0 comp rx=0 tx=0).stop 2024-06-06T04:29:34.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.836+0000 7f9ad9320700 1 -- 172.21.15.45:0/1253780449 shutdown_connections 2024-06-06T04:29:34.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.836+0000 7f9ad9320700 1 --2- 172.21.15.45:0/1253780449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ad4101d30 0x7f9ad410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:34.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.836+0000 7f9ad9320700 1 --2- 172.21.15.45:0/1253780449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ad41073d0 0x7f9ad41077b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:34.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.836+0000 7f9ad9320700 1 --2- 172.21.15.45:0/1253780449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ad4101390 0x7f9ad41017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:34.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.836+0000 7f9ad9320700 1 -- 172.21.15.45:0/1253780449 >> 172.21.15.45:0/1253780449 conn(0x7f9ad40782c0 msgr2=0x7f9ad40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:34.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.837+0000 7f9ad9320700 1 -- 172.21.15.45:0/1253780449 shutdown_connections 2024-06-06T04:29:34.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.837+0000 7f9ad9320700 1 -- 172.21.15.45:0/1253780449 wait complete. 2024-06-06T04:29:34.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.837+0000 7f9ad9320700 1 Processor -- start 2024-06-06T04:29:34.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.838+0000 7f9ad9320700 1 -- start start 2024-06-06T04:29:34.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.838+0000 7f9ad9320700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ad4101390 0x7f9ad4103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:34.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.838+0000 7f9ad9320700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ad4101d30 0x7f9ad4103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:34.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.838+0000 7f9ad9320700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ad41073d0 0x7f9ad4103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:34.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.838+0000 7f9ad9320700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad41114e0 con 0x7f9ad41073d0 2024-06-06T04:29:34.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.838+0000 7f9ad9320700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad4111360 con 0x7f9ad4101d30 2024-06-06T04:29:34.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.838+0000 7f9ad9320700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad4111660 con 0x7f9ad4101390 2024-06-06T04:29:34.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.838+0000 7f9ad37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ad41073d0 0x7f9ad4103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:34.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.838+0000 7f9ad37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ad41073d0 0x7f9ad4103ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33866/0 (socket says 172.21.15.45:33866) 2024-06-06T04:29:34.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.838+0000 7f9ad37fe700 1 -- 172.21.15.45:0/2803774501 learned_addr learned my addr 172.21.15.45:0/2803774501 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:34.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.838+0000 7f9ad2ffd700 1 --2- 172.21.15.45:0/2803774501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ad4101390 0x7f9ad4103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:34.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.838+0000 7f9ad27fc700 1 --2- 172.21.15.45:0/2803774501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ad4101d30 0x7f9ad4103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:34.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.839+0000 7f9ad37fe700 1 -- 172.21.15.45:0/2803774501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ad4101390 msgr2=0x7f9ad4103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:34.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.839+0000 7f9ad37fe700 1 --2- 172.21.15.45:0/2803774501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ad4101390 0x7f9ad4103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:34.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.839+0000 7f9ad37fe700 1 -- 172.21.15.45:0/2803774501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ad4101d30 msgr2=0x7f9ad4103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:34.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.839+0000 7f9ad37fe700 1 --2- 172.21.15.45:0/2803774501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ad4101d30 0x7f9ad4103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:34.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.839+0000 7f9ad37fe700 1 -- 172.21.15.45:0/2803774501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad41a7040 con 0x7f9ad41073d0 2024-06-06T04:29:34.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.839+0000 7f9ad27fc700 1 --2- 172.21.15.45:0/2803774501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ad4101d30 0x7f9ad4103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:29:34.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.839+0000 7f9ad37fe700 1 --2- 172.21.15.45:0/2803774501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ad41073d0 0x7f9ad4103ac0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9acc007ee0 tx=0x7f9acc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:34.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.839+0000 7f9acbfff700 1 -- 172.21.15.45:0/2803774501 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc00dbd0 con 0x7f9ad41073d0 2024-06-06T04:29:34.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.839+0000 7f9acbfff700 1 -- 172.21.15.45:0/2803774501 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9acc00dd70 con 0x7f9ad41073d0 2024-06-06T04:29:34.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.839+0000 7f9ad9320700 1 -- 172.21.15.45:0/2803774501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad41a7330 con 0x7f9ad41073d0 2024-06-06T04:29:34.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.839+0000 7f9acbfff700 1 -- 172.21.15.45:0/2803774501 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc005990 con 0x7f9ad41073d0 2024-06-06T04:29:34.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.839+0000 7f9ad9320700 1 -- 172.21.15.45:0/2803774501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad41a7820 con 0x7f9ad41073d0 2024-06-06T04:29:34.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.840+0000 7f9ad9320700 1 -- 172.21.15.45:0/2803774501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ad4105450 con 0x7f9ad41073d0 2024-06-06T04:29:34.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.841+0000 7f9acbfff700 1 -- 172.21.15.45:0/2803774501 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9acc01a020 con 0x7f9ad41073d0 2024-06-06T04:29:34.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.842+0000 7f9acbfff700 1 --2- 172.21.15.45:0/2803774501 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9ac00a2cb0 0x7f9ac00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:34.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.842+0000 7f9ad2ffd700 1 --2- 172.21.15.45:0/2803774501 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9ac00a2cb0 0x7f9ac00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:34.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.842+0000 7f9acbfff700 1 -- 172.21.15.45:0/2803774501 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f9acc0cf370 con 0x7f9ad41073d0 2024-06-06T04:29:34.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.842+0000 7f9acbfff700 1 --2- 172.21.15.45:0/2803774501 >> [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039] conn(0x7f9ac00adf90 0x7f9ac00b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:34.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.842+0000 7f9acbfff700 1 -- 172.21.15.45:0/2803774501 --> [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ac00b0b00 con 0x7f9ac00adf90 2024-06-06T04:29:34.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.842+0000 7f9acbfff700 1 -- 172.21.15.45:0/2803774501 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f9acc0cf720 con 0x7f9ad41073d0 2024-06-06T04:29:34.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.842+0000 7f9ad2ffd700 1 --2- 172.21.15.45:0/2803774501 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9ac00a2cb0 0x7f9ac00a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4006010 tx=0x7f9ac400b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:34.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.843+0000 7f9ad27fc700 1 --2- 172.21.15.45:0/2803774501 >> [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039] conn(0x7f9ac00adf90 0x7f9ac00b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:34.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.843+0000 7f9ad27fc700 1 --2- 172.21.15.45:0/2803774501 >> [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039] conn(0x7f9ac00adf90 0x7f9ac00b0430 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:34.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.844+0000 7f9acbfff700 1 -- 172.21.15.45:0/2803774501 <== osd.2 v2:172.21.15.154:6800/2898783039 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9ac00b0b00 con 0x7f9ac00adf90 2024-06-06T04:29:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.860+0000 7f9ad9320700 1 -- 172.21.15.45:0/2803774501 --> [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9ad4102a80 con 0x7f9ac00adf90 2024-06-06T04:29:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.861+0000 7f9acbfff700 1 -- 172.21.15.45:0/2803774501 <== osd.2 v2:172.21.15.154:6800/2898783039 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9ad4102a80 con 0x7f9ac00adf90 2024-06-06T04:29:34.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.861+0000 7f9ad9320700 1 -- 172.21.15.45:0/2803774501 >> [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039] conn(0x7f9ac00adf90 msgr2=0x7f9ac00b0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:34.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.861+0000 7f9ad9320700 1 --2- 172.21.15.45:0/2803774501 >> [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039] conn(0x7f9ac00adf90 0x7f9ac00b0430 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:34.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.862+0000 7f9ad9320700 1 -- 172.21.15.45:0/2803774501 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9ac00a2cb0 msgr2=0x7f9ac00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:34.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.862+0000 7f9ad9320700 1 --2- 172.21.15.45:0/2803774501 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9ac00a2cb0 0x7f9ac00a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4006010 tx=0x7f9ac400b560 comp rx=0 tx=0).stop 2024-06-06T04:29:34.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.862+0000 7f9ad9320700 1 -- 172.21.15.45:0/2803774501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ad41073d0 msgr2=0x7f9ad4103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:34.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.862+0000 7f9ad9320700 1 --2- 172.21.15.45:0/2803774501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ad41073d0 0x7f9ad4103ac0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9acc007ee0 tx=0x7f9acc00d3b0 comp rx=0 tx=0).stop 2024-06-06T04:29:34.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.862+0000 7f9ad37fe700 1 -- 172.21.15.45:0/2803774501 reap_dead start 2024-06-06T04:29:34.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.862+0000 7f9ad9320700 1 -- 172.21.15.45:0/2803774501 shutdown_connections 2024-06-06T04:29:34.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.862+0000 7f9ad9320700 1 -- 172.21.15.45:0/2803774501 >> 172.21.15.45:0/2803774501 conn(0x7f9ad40782c0 msgr2=0x7f9ad4100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:34.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.862+0000 7f9ad9320700 1 -- 172.21.15.45:0/2803774501 shutdown_connections 2024-06-06T04:29:34.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:34.862+0000 7f9ad9320700 1 -- 172.21.15.45:0/2803774501 wait complete. 2024-06-06T04:29:35.834 INFO:teuthology.orchestra.run.smithi045.stdout:146028888142 2024-06-06T04:29:35.835 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-06T04:29:36.302 INFO:teuthology.orchestra.run.smithi045.stdout:111669149778 2024-06-06T04:29:36.302 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-06T04:29:36.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:36 smithi045 ceph-mon[32299]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-06-06T04:29:36.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:36 smithi086 ceph-mon[36053]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-06-06T04:29:36.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:36 smithi154 ceph-mon[36428]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-06-06T04:29:36.770 INFO:teuthology.orchestra.run.smithi045.stdout:77309411417 2024-06-06T04:29:36.770 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-06T04:29:37.192 INFO:teuthology.orchestra.run.smithi045.stdout:34359738466 2024-06-06T04:29:37.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-06T04:29:37.407 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:37 smithi045 ceph-mon[32299]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-06T04:29:37.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:37 smithi086 ceph-mon[36053]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-06T04:29:37.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.612+0000 7f0211c52700 1 -- 172.21.15.45:0/1823199927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c103120 msgr2=0x7f020c107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:37.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.613+0000 7f0211c52700 1 --2- 172.21.15.45:0/1823199927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c103120 0x7f020c107130 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f02000099c0 tx=0x7f0200009cd0 comp rx=0 tx=0).stop 2024-06-06T04:29:37.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.613+0000 7f0211c52700 1 -- 172.21.15.45:0/1823199927 shutdown_connections 2024-06-06T04:29:37.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.613+0000 7f0211c52700 1 --2- 172.21.15.45:0/1823199927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f020c107860 0x7f020c10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:37.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.613+0000 7f0211c52700 1 --2- 172.21.15.45:0/1823199927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020c102770 0x7f020c102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:37.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.613+0000 7f0211c52700 1 --2- 172.21.15.45:0/1823199927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c103120 0x7f020c107130 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:37.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.613+0000 7f0211c52700 1 -- 172.21.15.45:0/1823199927 >> 172.21.15.45:0/1823199927 conn(0x7f020c079600 msgr2=0x7f020c079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:37.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.613+0000 7f0211c52700 1 -- 172.21.15.45:0/1823199927 shutdown_connections 2024-06-06T04:29:37.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.613+0000 7f0211c52700 1 -- 172.21.15.45:0/1823199927 wait complete. 2024-06-06T04:29:37.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.613+0000 7f0211c52700 1 Processor -- start 2024-06-06T04:29:37.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f0211c52700 1 -- start start 2024-06-06T04:29:37.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f0211c52700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c102770 0x7f020c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:37.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f0211c52700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020c103120 0x7f020c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:37.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f0211c52700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f020c107860 0x7f020c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:37.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f0211c52700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f020c112640 con 0x7f020c102770 2024-06-06T04:29:37.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f0211c52700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f020c1124c0 con 0x7f020c103120 2024-06-06T04:29:37.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f0211c52700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f020c1127c0 con 0x7f020c107860 2024-06-06T04:29:37.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f020b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c102770 0x7f020c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:37.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f020b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c102770 0x7f020c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33884/0 (socket says 172.21.15.45:33884) 2024-06-06T04:29:37.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f020b7fe700 1 -- 172.21.15.45:0/2099291560 learned_addr learned my addr 172.21.15.45:0/2099291560 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:37.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f020affd700 1 --2- 172.21.15.45:0/2099291560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020c103120 0x7f020c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:37.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f020bfff700 1 --2- 172.21.15.45:0/2099291560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f020c107860 0x7f020c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:37.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f020b7fe700 1 -- 172.21.15.45:0/2099291560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f020c107860 msgr2=0x7f020c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:37.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f020b7fe700 1 --2- 172.21.15.45:0/2099291560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f020c107860 0x7f020c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:37.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f020b7fe700 1 -- 172.21.15.45:0/2099291560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020c103120 msgr2=0x7f020c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:37.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f020b7fe700 1 --2- 172.21.15.45:0/2099291560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f020c103120 0x7f020c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:37.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.614+0000 7f020b7fe700 1 -- 172.21.15.45:0/2099291560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020c1a7080 con 0x7f020c102770 2024-06-06T04:29:37.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.615+0000 7f020b7fe700 1 --2- 172.21.15.45:0/2099291560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c102770 0x7f020c074790 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00d980 tx=0x7f01fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:37.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.615+0000 7f0208ff9700 1 -- 172.21.15.45:0/2099291560 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc016070 con 0x7f020c102770 2024-06-06T04:29:37.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.615+0000 7f0208ff9700 1 -- 172.21.15.45:0/2099291560 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01fc004830 con 0x7f020c102770 2024-06-06T04:29:37.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.615+0000 7f0211c52700 1 -- 172.21.15.45:0/2099291560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020c1a7370 con 0x7f020c102770 2024-06-06T04:29:37.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.615+0000 7f0208ff9700 1 -- 172.21.15.45:0/2099291560 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0054b0 con 0x7f020c102770 2024-06-06T04:29:37.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.615+0000 7f0211c52700 1 -- 172.21.15.45:0/2099291560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f020c1a78d0 con 0x7f020c102770 2024-06-06T04:29:37.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.616+0000 7f0211c52700 1 -- 172.21.15.45:0/2099291560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f020c062ba0 con 0x7f020c102770 2024-06-06T04:29:37.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.617+0000 7f0208ff9700 1 -- 172.21.15.45:0/2099291560 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f01fc0049e0 con 0x7f020c102770 2024-06-06T04:29:37.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.618+0000 7f0208ff9700 1 --2- 172.21.15.45:0/2099291560 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f01f40a2d30 0x7f01f40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:37.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.618+0000 7f020affd700 1 --2- 172.21.15.45:0/2099291560 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f01f40a2d30 0x7f01f40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:37.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.619+0000 7f0208ff9700 1 -- 172.21.15.45:0/2099291560 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f01fc0cf9e0 con 0x7f020c102770 2024-06-06T04:29:37.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.619+0000 7f0208ff9700 1 --2- 172.21.15.45:0/2099291560 >> [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510] conn(0x7f01f40ae010 0x7f01f40b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:37.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.619+0000 7f0208ff9700 1 -- 172.21.15.45:0/2099291560 --> [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01fc015ec0 con 0x7f01f40ae010 2024-06-06T04:29:37.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.619+0000 7f0208ff9700 1 -- 172.21.15.45:0/2099291560 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f01fc0cfd90 con 0x7f020c102770 2024-06-06T04:29:37.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.619+0000 7f020bfff700 1 --2- 172.21.15.45:0/2099291560 >> [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510] conn(0x7f01f40ae010 0x7f01f40b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:37.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.619+0000 7f020affd700 1 --2- 172.21.15.45:0/2099291560 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f01f40a2d30 0x7f01f40a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f020c076220 tx=0x7f0200005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:37.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.619+0000 7f020bfff700 1 --2- 172.21.15.45:0/2099291560 >> [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510] conn(0x7f01f40ae010 0x7f01f40b04b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:37.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.620+0000 7f0208ff9700 1 -- 172.21.15.45:0/2099291560 <== osd.9 v2:172.21.15.45:6826/1360655510 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f01fc015ec0 con 0x7f01f40ae010 2024-06-06T04:29:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.638+0000 7f0211c52700 1 -- 172.21.15.45:0/2099291560 --> [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f020c062fa0 con 0x7f01f40ae010 2024-06-06T04:29:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.639+0000 7f0208ff9700 1 -- 172.21.15.45:0/2099291560 <== osd.9 v2:172.21.15.45:6826/1360655510 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f020c062fa0 con 0x7f01f40ae010 2024-06-06T04:29:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.639+0000 7f0211c52700 1 -- 172.21.15.45:0/2099291560 >> [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510] conn(0x7f01f40ae010 msgr2=0x7f01f40b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.639+0000 7f0211c52700 1 --2- 172.21.15.45:0/2099291560 >> [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510] conn(0x7f01f40ae010 0x7f01f40b04b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.639+0000 7f0211c52700 1 -- 172.21.15.45:0/2099291560 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f01f40a2d30 msgr2=0x7f01f40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.639+0000 7f0211c52700 1 --2- 172.21.15.45:0/2099291560 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f01f40a2d30 0x7f01f40a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f020c076220 tx=0x7f0200005fb0 comp rx=0 tx=0).stop 2024-06-06T04:29:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.639+0000 7f0211c52700 1 -- 172.21.15.45:0/2099291560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c102770 msgr2=0x7f020c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.639+0000 7f0211c52700 1 --2- 172.21.15.45:0/2099291560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c102770 0x7f020c074790 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00d980 tx=0x7f01fc00dc90 comp rx=0 tx=0).stop 2024-06-06T04:29:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.639+0000 7f020bfff700 1 -- 172.21.15.45:0/2099291560 reap_dead start 2024-06-06T04:29:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.639+0000 7f0211c52700 1 -- 172.21.15.45:0/2099291560 shutdown_connections 2024-06-06T04:29:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.639+0000 7f0211c52700 1 -- 172.21.15.45:0/2099291560 >> 172.21.15.45:0/2099291560 conn(0x7f020c079600 msgr2=0x7f020c100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.639+0000 7f0211c52700 1 -- 172.21.15.45:0/2099291560 shutdown_connections 2024-06-06T04:29:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:37.639+0000 7f0211c52700 1 -- 172.21.15.45:0/2099291560 wait complete. 2024-06-06T04:29:37.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:37 smithi154 ceph-mon[36428]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-06T04:29:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.018+0000 7fe777923700 1 -- 172.21.15.45:0/1073230074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7701030f0 msgr2=0x7fe770107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.018+0000 7fe777923700 1 --2- 172.21.15.45:0/1073230074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7701030f0 0x7fe770107110 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe7600099c0 tx=0x7fe760009cd0 comp rx=0 tx=0).stop 2024-06-06T04:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.019+0000 7fe777923700 1 -- 172.21.15.45:0/1073230074 shutdown_connections 2024-06-06T04:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.019+0000 7fe777923700 1 --2- 172.21.15.45:0/1073230074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe770107840 0x7fe77010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.019+0000 7fe777923700 1 --2- 172.21.15.45:0/1073230074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770102740 0x7fe770102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.019+0000 7fe777923700 1 --2- 172.21.15.45:0/1073230074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7701030f0 0x7fe770107110 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.019+0000 7fe777923700 1 -- 172.21.15.45:0/1073230074 >> 172.21.15.45:0/1073230074 conn(0x7fe7700795f0 msgr2=0x7fe770079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.019+0000 7fe777923700 1 -- 172.21.15.45:0/1073230074 shutdown_connections 2024-06-06T04:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.019+0000 7fe777923700 1 -- 172.21.15.45:0/1073230074 wait complete. 2024-06-06T04:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.019+0000 7fe777923700 1 Processor -- start 2024-06-06T04:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.019+0000 7fe777923700 1 -- start start 2024-06-06T04:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.019+0000 7fe777923700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770102740 0x7fe770074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.019+0000 7fe777923700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7701030f0 0x7fe770074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe777923700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe770107840 0x7fe770075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe777923700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe770112600 con 0x7fe770107840 2024-06-06T04:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe777923700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe770112480 con 0x7fe770102740 2024-06-06T04:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe777923700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe770112780 con 0x7fe7701030f0 2024-06-06T04:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe775ec0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe770107840 0x7fe770075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe775ec0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe770107840 0x7fe770075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39942/0 (socket says 172.21.15.45:39942) 2024-06-06T04:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe7756bf700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770102740 0x7fe770074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe775ec0700 1 -- 172.21.15.45:0/163515295 learned_addr learned my addr 172.21.15.45:0/163515295 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe774ebe700 1 --2- 172.21.15.45:0/163515295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7701030f0 0x7fe770074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe775ec0700 1 -- 172.21.15.45:0/163515295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7701030f0 msgr2=0x7fe770074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe775ec0700 1 --2- 172.21.15.45:0/163515295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7701030f0 0x7fe770074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe775ec0700 1 -- 172.21.15.45:0/163515295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770102740 msgr2=0x7fe770074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe775ec0700 1 --2- 172.21.15.45:0/163515295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe770102740 0x7fe770074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe775ec0700 1 -- 172.21.15.45:0/163515295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7701a6fb0 con 0x7fe770107840 2024-06-06T04:29:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe775ec0700 1 --2- 172.21.15.45:0/163515295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe770107840 0x7fe770075210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe768007d70 tx=0x7fe76800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe7667fc700 1 -- 172.21.15.45:0/163515295 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768016020 con 0x7fe770107840 2024-06-06T04:29:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe7667fc700 1 -- 172.21.15.45:0/163515295 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe76800ac90 con 0x7fe770107840 2024-06-06T04:29:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe777923700 1 -- 172.21.15.45:0/163515295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7701a72a0 con 0x7fe770107840 2024-06-06T04:29:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe7667fc700 1 -- 172.21.15.45:0/163515295 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7680115e0 con 0x7fe770107840 2024-06-06T04:29:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.020+0000 7fe777923700 1 -- 172.21.15.45:0/163515295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7701a77b0 con 0x7fe770107840 2024-06-06T04:29:38.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.021+0000 7fe777923700 1 -- 172.21.15.45:0/163515295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe770062ba0 con 0x7fe770107840 2024-06-06T04:29:38.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.023+0000 7fe7667fc700 1 -- 172.21.15.45:0/163515295 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe768019020 con 0x7fe770107840 2024-06-06T04:29:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.024+0000 7fe7667fc700 1 --2- 172.21.15.45:0/163515295 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe75c0a2d30 0x7fe75c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.024+0000 7fe7667fc700 1 -- 172.21.15.45:0/163515295 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fe7680cf7c0 con 0x7fe770107840 2024-06-06T04:29:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.024+0000 7fe7667fc700 1 --2- 172.21.15.45:0/163515295 >> [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939] conn(0x7fe75c0ae010 0x7fe75c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.024+0000 7fe7667fc700 1 -- 172.21.15.45:0/163515295 --> [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe76801ce50 con 0x7fe75c0ae010 2024-06-06T04:29:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.024+0000 7fe7667fc700 1 -- 172.21.15.45:0/163515295 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fe7680cfc10 con 0x7fe770107840 2024-06-06T04:29:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.024+0000 7fe7756bf700 1 --2- 172.21.15.45:0/163515295 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe75c0a2d30 0x7fe75c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.024+0000 7fe774ebe700 1 --2- 172.21.15.45:0/163515295 >> [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939] conn(0x7fe75c0ae010 0x7fe75c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.025+0000 7fe774ebe700 1 --2- 172.21.15.45:0/163515295 >> [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939] conn(0x7fe75c0ae010 0x7fe75c0b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.026+0000 7fe7756bf700 1 --2- 172.21.15.45:0/163515295 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe75c0a2d30 0x7fe75c0a51f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00ac50 tx=0x7fe76c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.026+0000 7fe7667fc700 1 -- 172.21.15.45:0/163515295 <== osd.6 v2:172.21.15.45:6818/384734939 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe76801ce50 con 0x7fe75c0ae010 2024-06-06T04:29:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:38.042 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:38.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.066+0000 7fe777923700 1 -- 172.21.15.45:0/163515295 --> [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe770063020 con 0x7fe75c0ae010 2024-06-06T04:29:38.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.068+0000 7fe7667fc700 1 -- 172.21.15.45:0/163515295 <== osd.6 v2:172.21.15.45:6818/384734939 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe770063020 con 0x7fe75c0ae010 2024-06-06T04:29:38.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.069+0000 7fe777923700 1 -- 172.21.15.45:0/163515295 >> [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939] conn(0x7fe75c0ae010 msgr2=0x7fe75c0b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:38.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.069+0000 7fe777923700 1 --2- 172.21.15.45:0/163515295 >> [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939] conn(0x7fe75c0ae010 0x7fe75c0b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.069+0000 7fe777923700 1 -- 172.21.15.45:0/163515295 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe75c0a2d30 msgr2=0x7fe75c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.069+0000 7fe777923700 1 --2- 172.21.15.45:0/163515295 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe75c0a2d30 0x7fe75c0a51f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00ac50 tx=0x7fe76c00a380 comp rx=0 tx=0).stop 2024-06-06T04:29:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.069+0000 7fe777923700 1 -- 172.21.15.45:0/163515295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe770107840 msgr2=0x7fe770075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.069+0000 7fe777923700 1 --2- 172.21.15.45:0/163515295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe770107840 0x7fe770075210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe768007d70 tx=0x7fe76800a340 comp rx=0 tx=0).stop 2024-06-06T04:29:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.070+0000 7fe775ec0700 1 -- 172.21.15.45:0/163515295 reap_dead start 2024-06-06T04:29:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.070+0000 7fe777923700 1 -- 172.21.15.45:0/163515295 shutdown_connections 2024-06-06T04:29:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.070+0000 7fe777923700 1 -- 172.21.15.45:0/163515295 >> 172.21.15.45:0/163515295 conn(0x7fe7700795f0 msgr2=0x7fe770100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.071+0000 7fe777923700 1 -- 172.21.15.45:0/163515295 shutdown_connections 2024-06-06T04:29:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:38.071+0000 7fe777923700 1 -- 172.21.15.45:0/163515295 wait complete. 2024-06-06T04:29:38.838 INFO:teuthology.orchestra.run.smithi045.stdout:292057776182 2024-06-06T04:29:38.838 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-06-06T04:29:39.074 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:39.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:39 smithi045 ceph-mon[32299]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T04:29:39.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:39 smithi086 ceph-mon[36053]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T04:29:39.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:39 smithi154 ceph-mon[36428]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T04:29:41.400 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:41.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:41 smithi045 ceph-mon[32299]: pgmap v378: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:41.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:41 smithi086 ceph-mon[36053]: pgmap v378: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:41.926 INFO:teuthology.orchestra.run.smithi045.stdout:201863462981 2024-06-06T04:29:41.926 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-06T04:29:41.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:41 smithi154 ceph-mon[36428]: pgmap v378: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:43.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:43 smithi045 ceph-mon[32299]: pgmap v379: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:29:43.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:43 smithi086 ceph-mon[36053]: pgmap v379: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:29:43.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:43 smithi154 ceph-mon[36428]: pgmap v379: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:29:45.569 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:45.569 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:45.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:45 smithi045 ceph-mon[32299]: pgmap v380: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:45.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:45 smithi086 ceph-mon[36053]: pgmap v380: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:45.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:45 smithi154 ceph-mon[36428]: pgmap v380: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:46.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.367+0000 7f3d72740700 1 -- 172.21.15.45:0/3342766615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6c101d50 msgr2=0x7f3d6c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:46.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.367+0000 7f3d72740700 1 --2- 172.21.15.45:0/3342766615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6c101d50 0x7f3d6c10e240 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400b0a0 tx=0x7f3d6400b3b0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.367+0000 7f3d72740700 1 -- 172.21.15.45:0/3342766615 shutdown_connections 2024-06-06T04:29:46.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.367+0000 7f3d72740700 1 --2- 172.21.15.45:0/3342766615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6c101d50 0x7f3d6c10e240 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.367+0000 7f3d72740700 1 --2- 172.21.15.45:0/3342766615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d6c107400 0x7f3d6c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.367+0000 7f3d72740700 1 --2- 172.21.15.45:0/3342766615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d6c101390 0x7f3d6c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.367+0000 7f3d72740700 1 -- 172.21.15.45:0/3342766615 >> 172.21.15.45:0/3342766615 conn(0x7f3d6c078340 msgr2=0x7f3d6c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.367+0000 7f3d72740700 1 -- 172.21.15.45:0/3342766615 shutdown_connections 2024-06-06T04:29:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.367+0000 7f3d72740700 1 -- 172.21.15.45:0/3342766615 wait complete. 2024-06-06T04:29:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.368+0000 7f3d72740700 1 Processor -- start 2024-06-06T04:29:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.368+0000 7f3d72740700 1 -- start start 2024-06-06T04:29:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.368+0000 7f3d72740700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d6c101390 0x7f3d6c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.368+0000 7f3d72740700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6c101d50 0x7f3d6c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.368+0000 7f3d72740700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d6c107400 0x7f3d6c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.368+0000 7f3d72740700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c111550 con 0x7f3d6c101d50 2024-06-06T04:29:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.368+0000 7f3d72740700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c1113d0 con 0x7f3d6c101390 2024-06-06T04:29:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.368+0000 7f3d72740700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c1116d0 con 0x7f3d6c107400 2024-06-06T04:29:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.368+0000 7f3d6bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d6c101390 0x7f3d6c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.368+0000 7f3d63fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6c101d50 0x7f3d6c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.368+0000 7f3d6bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d6c101390 0x7f3d6c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:60820/0 (socket says 172.21.15.45:60820) 2024-06-06T04:29:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.369+0000 7f3d6bfff700 1 -- 172.21.15.45:0/2271624532 learned_addr learned my addr 172.21.15.45:0/2271624532 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.368+0000 7f3d63fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6c101d50 0x7f3d6c19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39952/0 (socket says 172.21.15.45:39952) 2024-06-06T04:29:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.369+0000 7f3d70cdd700 1 --2- 172.21.15.45:0/2271624532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d6c107400 0x7f3d6c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.369+0000 7f3d63fff700 1 -- 172.21.15.45:0/2271624532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d6c107400 msgr2=0x7f3d6c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.369+0000 7f3d63fff700 1 --2- 172.21.15.45:0/2271624532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d6c107400 0x7f3d6c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.369+0000 7f3d63fff700 1 -- 172.21.15.45:0/2271624532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d6c101390 msgr2=0x7f3d6c19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.369+0000 7f3d63fff700 1 --2- 172.21.15.45:0/2271624532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d6c101390 0x7f3d6c19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.369+0000 7f3d63fff700 1 -- 172.21.15.45:0/2271624532 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c1990a0 con 0x7f3d6c101d50 2024-06-06T04:29:46.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.369+0000 7f3d63fff700 1 --2- 172.21.15.45:0/2271624532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6c101d50 0x7f3d6c19ecc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400d750 tx=0x7f3d5400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:46.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.369+0000 7f3d69ffb700 1 -- 172.21.15.45:0/2271624532 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54016070 con 0x7f3d6c101d50 2024-06-06T04:29:46.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.369+0000 7f3d69ffb700 1 -- 172.21.15.45:0/2271624532 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d54005020 con 0x7f3d6c101d50 2024-06-06T04:29:46.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.369+0000 7f3d72740700 1 -- 172.21.15.45:0/2271624532 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6c199390 con 0x7f3d6c101d50 2024-06-06T04:29:46.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.369+0000 7f3d69ffb700 1 -- 172.21.15.45:0/2271624532 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5400f9f0 con 0x7f3d6c101d50 2024-06-06T04:29:46.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.369+0000 7f3d72740700 1 -- 172.21.15.45:0/2271624532 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d6c1a5650 con 0x7f3d6c101d50 2024-06-06T04:29:46.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.370+0000 7f3d72740700 1 -- 172.21.15.45:0/2271624532 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c069180 con 0x7f3d6c101d50 2024-06-06T04:29:46.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.371+0000 7f3d69ffb700 1 -- 172.21.15.45:0/2271624532 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3d540051d0 con 0x7f3d6c101d50 2024-06-06T04:29:46.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.372+0000 7f3d69ffb700 1 --2- 172.21.15.45:0/2271624532 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3d580a2d30 0x7f3d580a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:46.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.372+0000 7f3d6bfff700 1 --2- 172.21.15.45:0/2271624532 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3d580a2d30 0x7f3d580a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:46.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.372+0000 7f3d69ffb700 1 -- 172.21.15.45:0/2271624532 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f3d540cf420 con 0x7f3d6c101d50 2024-06-06T04:29:46.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.373+0000 7f3d6bfff700 1 --2- 172.21.15.45:0/2271624532 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3d580a2d30 0x7f3d580a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c199a80 tx=0x7f3d5c00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:46.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.376+0000 7f3d69ffb700 1 -- 172.21.15.45:0/2271624532 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3d5409be00 con 0x7f3d6c101d50 2024-06-06T04:29:46.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.593+0000 7f3d72740700 1 -- 172.21.15.45:0/2271624532 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f3d6c1a5940 con 0x7f3d6c101d50 2024-06-06T04:29:46.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.586+0000 7f7547f75700 1 -- 172.21.15.45:0/2785287581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7540101d40 msgr2=0x7f754010e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:46.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.586+0000 7f7547f75700 1 --2- 172.21.15.45:0/2785287581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7540101d40 0x7f754010e230 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f753800b0a0 tx=0x7f753800b3b0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.587+0000 7f7547f75700 1 -- 172.21.15.45:0/2785287581 shutdown_connections 2024-06-06T04:29:46.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.587+0000 7f7547f75700 1 --2- 172.21.15.45:0/2785287581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7540101d40 0x7f754010e230 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.587+0000 7f7547f75700 1 --2- 172.21.15.45:0/2785287581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75401073f0 0x7f75401077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.587+0000 7f7547f75700 1 --2- 172.21.15.45:0/2785287581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7540101380 0x7f7540101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.587+0000 7f7547f75700 1 -- 172.21.15.45:0/2785287581 >> 172.21.15.45:0/2785287581 conn(0x7f7540078300 msgr2=0x7f7540078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:46.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.587+0000 7f7547f75700 1 -- 172.21.15.45:0/2785287581 shutdown_connections 2024-06-06T04:29:46.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.587+0000 7f7547f75700 1 -- 172.21.15.45:0/2785287581 wait complete. 2024-06-06T04:29:46.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.587+0000 7f7547f75700 1 Processor -- start 2024-06-06T04:29:46.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.587+0000 7f7547f75700 1 -- start start 2024-06-06T04:29:46.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7547f75700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7540101380 0x7f7540103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:46.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7547f75700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7540101d40 0x7f75401035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:46.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7547f75700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75401073f0 0x7f7540103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:46.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7547f75700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7540111510 con 0x7f7540101d40 2024-06-06T04:29:46.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7547f75700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7540111390 con 0x7f75401073f0 2024-06-06T04:29:46.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7547f75700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7540111690 con 0x7f7540101380 2024-06-06T04:29:46.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7545510700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7540101d40 0x7f75401035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:46.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7545510700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7540101d40 0x7f75401035a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39960/0 (socket says 172.21.15.45:39960) 2024-06-06T04:29:46.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7545510700 1 -- 172.21.15.45:0/1851687177 learned_addr learned my addr 172.21.15.45:0/1851687177 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:46.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7545d11700 1 --2- 172.21.15.45:0/1851687177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7540101380 0x7f7540103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:46.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7546512700 1 --2- 172.21.15.45:0/1851687177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75401073f0 0x7f7540103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7545510700 1 -- 172.21.15.45:0/1851687177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7540101380 msgr2=0x7f7540103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7545510700 1 --2- 172.21.15.45:0/1851687177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7540101380 0x7f7540103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7545510700 1 -- 172.21.15.45:0/1851687177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75401073f0 msgr2=0x7f7540103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7545510700 1 --2- 172.21.15.45:0/1851687177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75401073f0 0x7f7540103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7545510700 1 -- 172.21.15.45:0/1851687177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75401a6fd0 con 0x7f7540101d40 2024-06-06T04:29:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7546512700 1 --2- 172.21.15.45:0/1851687177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75401073f0 0x7f7540103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:29:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7545510700 1 --2- 172.21.15.45:0/1851687177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7540101d40 0x7f75401035a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f753000c9b0 tx=0x7f753000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.588+0000 7f7536ffd700 1 -- 172.21.15.45:0/1851687177 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530007b00 con 0x7f7540101d40 2024-06-06T04:29:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.589+0000 7f7547f75700 1 -- 172.21.15.45:0/1851687177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75401a72c0 con 0x7f7540101d40 2024-06-06T04:29:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.589+0000 7f7536ffd700 1 -- 172.21.15.45:0/1851687177 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7530007ca0 con 0x7f7540101d40 2024-06-06T04:29:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.589+0000 7f7536ffd700 1 -- 172.21.15.45:0/1851687177 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530018660 con 0x7f7540101d40 2024-06-06T04:29:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.589+0000 7f7547f75700 1 -- 172.21.15.45:0/1851687177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75401a7850 con 0x7f7540101d40 2024-06-06T04:29:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.590+0000 7f7547f75700 1 -- 172.21.15.45:0/1851687177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7540069180 con 0x7f7540101d40 2024-06-06T04:29:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.591+0000 7f7536ffd700 1 -- 172.21.15.45:0/1851687177 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f753000f450 con 0x7f7540101d40 2024-06-06T04:29:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.592+0000 7f7536ffd700 1 --2- 172.21.15.45:0/1851687177 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f752c0a2cb0 0x7f752c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.592+0000 7f7545d11700 1 --2- 172.21.15.45:0/1851687177 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f752c0a2cb0 0x7f752c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.592+0000 7f7536ffd700 1 -- 172.21.15.45:0/1851687177 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f75300cf870 con 0x7f7540101d40 2024-06-06T04:29:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.593+0000 7f7545d11700 1 --2- 172.21.15.45:0/1851687177 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f752c0a2cb0 0x7f752c0a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f753c005950 tx=0x7f753c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.595+0000 7f7536ffd700 1 -- 172.21.15.45:0/1851687177 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7530014030 con 0x7f7540101d40 2024-06-06T04:29:46.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.596+0000 7f3d69ffb700 1 -- 172.21.15.45:0/2271624532 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3d5409b700 con 0x7f3d6c101d50 2024-06-06T04:29:46.599 INFO:teuthology.orchestra.run.smithi045.stdout:257698037820 2024-06-06T04:29:46.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.598+0000 7f3d72740700 1 -- 172.21.15.45:0/2271624532 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3d580a2d30 msgr2=0x7f3d580a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:46.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.598+0000 7f3d72740700 1 --2- 172.21.15.45:0/2271624532 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3d580a2d30 0x7f3d580a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c199a80 tx=0x7f3d5c00b500 comp rx=0 tx=0).stop 2024-06-06T04:29:46.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.598+0000 7f3d72740700 1 -- 172.21.15.45:0/2271624532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6c101d50 msgr2=0x7f3d6c19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.598+0000 7f3d72740700 1 --2- 172.21.15.45:0/2271624532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6c101d50 0x7f3d6c19ecc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400d750 tx=0x7f3d5400da60 comp rx=0 tx=0).stop 2024-06-06T04:29:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.598+0000 7f3d72740700 1 -- 172.21.15.45:0/2271624532 shutdown_connections 2024-06-06T04:29:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.598+0000 7f3d72740700 1 --2- 172.21.15.45:0/2271624532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d6c107400 0x7f3d6c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.598+0000 7f3d72740700 1 --2- 172.21.15.45:0/2271624532 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3d580a2d30 0x7f3d580a51f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.598+0000 7f3d72740700 1 --2- 172.21.15.45:0/2271624532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d6c101390 0x7f3d6c19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.598+0000 7f3d72740700 1 --2- 172.21.15.45:0/2271624532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6c101d50 0x7f3d6c19ecc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.598+0000 7f3d72740700 1 -- 172.21.15.45:0/2271624532 >> 172.21.15.45:0/2271624532 conn(0x7f3d6c078340 msgr2=0x7f3d6c100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.598+0000 7f3d72740700 1 -- 172.21.15.45:0/2271624532 shutdown_connections 2024-06-06T04:29:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.598+0000 7f3d72740700 1 -- 172.21.15.45:0/2271624532 wait complete. 2024-06-06T04:29:46.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.811+0000 7f7547f75700 1 -- 172.21.15.45:0/1851687177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f75401a7b70 con 0x7f7540101d40 2024-06-06T04:29:46.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.812+0000 7f7536ffd700 1 -- 172.21.15.45:0/1851687177 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f753009c2e0 con 0x7f7540101d40 2024-06-06T04:29:46.812 INFO:teuthology.orchestra.run.smithi045.stdout:223338299457 2024-06-06T04:29:46.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.814+0000 7f7547f75700 1 -- 172.21.15.45:0/1851687177 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f752c0a2cb0 msgr2=0x7f752c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:46.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.814+0000 7f7547f75700 1 --2- 172.21.15.45:0/1851687177 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f752c0a2cb0 0x7f752c0a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f753c005950 tx=0x7f753c00b410 comp rx=0 tx=0).stop 2024-06-06T04:29:46.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.814+0000 7f7547f75700 1 -- 172.21.15.45:0/1851687177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7540101d40 msgr2=0x7f75401035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:46.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.814+0000 7f7547f75700 1 --2- 172.21.15.45:0/1851687177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7540101d40 0x7f75401035a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f753000c9b0 tx=0x7f753000ccc0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.814+0000 7f7547f75700 1 -- 172.21.15.45:0/1851687177 shutdown_connections 2024-06-06T04:29:46.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.814+0000 7f7547f75700 1 --2- 172.21.15.45:0/1851687177 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f752c0a2cb0 0x7f752c0a5170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.814+0000 7f7547f75700 1 --2- 172.21.15.45:0/1851687177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75401073f0 0x7f7540103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.814+0000 7f7547f75700 1 --2- 172.21.15.45:0/1851687177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7540101380 0x7f7540103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.814+0000 7f7547f75700 1 --2- 172.21.15.45:0/1851687177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7540101d40 0x7f75401035a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:46.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.814+0000 7f7547f75700 1 -- 172.21.15.45:0/1851687177 >> 172.21.15.45:0/1851687177 conn(0x7f7540078300 msgr2=0x7f75401004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:46.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.814+0000 7f7547f75700 1 -- 172.21.15.45:0/1851687177 shutdown_connections 2024-06-06T04:29:46.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:46.814+0000 7f7547f75700 1 -- 172.21.15.45:0/1851687177 wait complete. 2024-06-06T04:29:47.405 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:47.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:47 smithi045 ceph-mon[32299]: pgmap v381: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-06-06T04:29:47.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:47 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2271624532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-06T04:29:47.717 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:47 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1851687177' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-06T04:29:47.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:47 smithi086 ceph-mon[36053]: pgmap v381: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-06-06T04:29:47.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:47 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2271624532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-06T04:29:47.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:47 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1851687177' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-06T04:29:47.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:47 smithi154 ceph-mon[36428]: pgmap v381: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-06-06T04:29:47.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:47 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2271624532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-06T04:29:47.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:47 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1851687177' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-06T04:29:48.985 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299454 got 223338299457 for osd.7 2024-06-06T04:29:48.985 DEBUG:teuthology.parallel:result is None 2024-06-06T04:29:49.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.515+0000 7ff851021700 1 -- 172.21.15.45:0/3774170086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff84c107400 msgr2=0x7ff84c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:49.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.515+0000 7ff851021700 1 --2- 172.21.15.45:0/3774170086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff84c107400 0x7ff84c1077e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009990 tx=0x7ff83c009ca0 comp rx=0 tx=0).stop 2024-06-06T04:29:49.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.515+0000 7ff851021700 1 -- 172.21.15.45:0/3774170086 shutdown_connections 2024-06-06T04:29:49.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.515+0000 7ff851021700 1 --2- 172.21.15.45:0/3774170086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff84c101d50 0x7ff84c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:49.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.515+0000 7ff851021700 1 --2- 172.21.15.45:0/3774170086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff84c107400 0x7ff84c1077e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:49.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.515+0000 7ff851021700 1 --2- 172.21.15.45:0/3774170086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff84c101390 0x7ff84c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:49.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.515+0000 7ff851021700 1 -- 172.21.15.45:0/3774170086 >> 172.21.15.45:0/3774170086 conn(0x7ff84c078340 msgr2=0x7ff84c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:49.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.515+0000 7ff851021700 1 -- 172.21.15.45:0/3774170086 shutdown_connections 2024-06-06T04:29:49.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.515+0000 7ff851021700 1 -- 172.21.15.45:0/3774170086 wait complete. 2024-06-06T04:29:49.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.516+0000 7ff851021700 1 Processor -- start 2024-06-06T04:29:49.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.516+0000 7ff851021700 1 -- start start 2024-06-06T04:29:49.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.516+0000 7ff851021700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff84c101390 0x7ff84c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:49.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.516+0000 7ff851021700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff84c101d50 0x7ff84c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:49.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.516+0000 7ff851021700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff84c107400 0x7ff84c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.516+0000 7ff851021700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c111550 con 0x7ff84c107400 2024-06-06T04:29:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.516+0000 7ff851021700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c1113d0 con 0x7ff84c101d50 2024-06-06T04:29:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.516+0000 7ff851021700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c1116d0 con 0x7ff84c101390 2024-06-06T04:29:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.516+0000 7ff84b59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff84c107400 0x7ff84c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.516+0000 7ff84ad9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff84c101390 0x7ff84c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.516+0000 7ff84b59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff84c107400 0x7ff84c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51622/0 (socket says 172.21.15.45:51622) 2024-06-06T04:29:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.516+0000 7ff84ad9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff84c101390 0x7ff84c19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:34386/0 (socket says 172.21.15.45:34386) 2024-06-06T04:29:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.516+0000 7ff84b59e700 1 -- 172.21.15.45:0/1682816287 learned_addr learned my addr 172.21.15.45:0/1682816287 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.517+0000 7ff84a59c700 1 --2- 172.21.15.45:0/1682816287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff84c101d50 0x7ff84c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.517+0000 7ff84b59e700 1 -- 172.21.15.45:0/1682816287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff84c101390 msgr2=0x7ff84c19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.517+0000 7ff84b59e700 1 --2- 172.21.15.45:0/1682816287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff84c101390 0x7ff84c19e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.517+0000 7ff84b59e700 1 -- 172.21.15.45:0/1682816287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff84c101d50 msgr2=0x7ff84c19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.517+0000 7ff84b59e700 1 --2- 172.21.15.45:0/1682816287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff84c101d50 0x7ff84c19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:49.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.517+0000 7ff84b59e700 1 -- 172.21.15.45:0/1682816287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84c1990c0 con 0x7ff84c107400 2024-06-06T04:29:49.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.517+0000 7ff84ad9d700 1 --2- 172.21.15.45:0/1682816287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff84c101390 0x7ff84c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:29:49.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.517+0000 7ff84b59e700 1 --2- 172.21.15.45:0/1682816287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff84c107400 0x7ff84c198960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff84400ada0 tx=0x7ff84400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:49.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.517+0000 7ff833fff700 1 -- 172.21.15.45:0/1682816287 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844017020 con 0x7ff84c107400 2024-06-06T04:29:49.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.517+0000 7ff833fff700 1 -- 172.21.15.45:0/1682816287 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff84400cd80 con 0x7ff84c107400 2024-06-06T04:29:49.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.517+0000 7ff851021700 1 -- 172.21.15.45:0/1682816287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84c1993b0 con 0x7ff84c107400 2024-06-06T04:29:49.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.517+0000 7ff833fff700 1 -- 172.21.15.45:0/1682816287 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844005260 con 0x7ff84c107400 2024-06-06T04:29:49.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.517+0000 7ff851021700 1 -- 172.21.15.45:0/1682816287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff84c1a5670 con 0x7ff84c107400 2024-06-06T04:29:49.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.518+0000 7ff851021700 1 -- 172.21.15.45:0/1682816287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84c069180 con 0x7ff84c107400 2024-06-06T04:29:49.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.520+0000 7ff833fff700 1 -- 172.21.15.45:0/1682816287 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff84400c8d0 con 0x7ff84c107400 2024-06-06T04:29:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.521+0000 7ff833fff700 1 --2- 172.21.15.45:0/1682816287 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff8340a2cb0 0x7ff8340a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.521+0000 7ff84ad9d700 1 --2- 172.21.15.45:0/1682816287 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff8340a2cb0 0x7ff8340a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.521+0000 7ff833fff700 1 -- 172.21.15.45:0/1682816287 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7ff8440cf250 con 0x7ff84c107400 2024-06-06T04:29:49.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.521+0000 7ff84ad9d700 1 --2- 172.21.15.45:0/1682816287 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff8340a2cb0 0x7ff8340a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00b5c0 tx=0x7ff83c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:49.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.525+0000 7ff833fff700 1 -- 172.21.15.45:0/1682816287 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff844021020 con 0x7ff84c107400 2024-06-06T04:29:49.693 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:49 smithi045 ceph-mon[32299]: pgmap v382: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T04:29:49.693 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:49 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1897340988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-06T04:29:49.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.740+0000 7ff851021700 1 -- 172.21.15.45:0/1682816287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff84c1a5840 con 0x7ff84c107400 2024-06-06T04:29:49.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.740+0000 7ff833fff700 1 -- 172.21.15.45:0/1682816287 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff84409bcc0 con 0x7ff84c107400 2024-06-06T04:29:49.741 INFO:teuthology.orchestra.run.smithi045.stdout:55834574944 2024-06-06T04:29:49.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.743+0000 7ff851021700 1 -- 172.21.15.45:0/1682816287 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff8340a2cb0 msgr2=0x7ff8340a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:49.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.743+0000 7ff851021700 1 --2- 172.21.15.45:0/1682816287 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff8340a2cb0 0x7ff8340a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00b5c0 tx=0x7ff83c005960 comp rx=0 tx=0).stop 2024-06-06T04:29:49.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.743+0000 7ff851021700 1 -- 172.21.15.45:0/1682816287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff84c107400 msgr2=0x7ff84c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:49.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.743+0000 7ff851021700 1 --2- 172.21.15.45:0/1682816287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff84c107400 0x7ff84c198960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff84400ada0 tx=0x7ff84400c330 comp rx=0 tx=0).stop 2024-06-06T04:29:49.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.743+0000 7ff851021700 1 -- 172.21.15.45:0/1682816287 shutdown_connections 2024-06-06T04:29:49.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.743+0000 7ff851021700 1 --2- 172.21.15.45:0/1682816287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff84c107400 0x7ff84c198960 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:49.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.743+0000 7ff851021700 1 --2- 172.21.15.45:0/1682816287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff84c101390 0x7ff84c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:49.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.743+0000 7ff851021700 1 --2- 172.21.15.45:0/1682816287 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff8340a2cb0 0x7ff8340a5170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:49.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.743+0000 7ff851021700 1 --2- 172.21.15.45:0/1682816287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff84c101d50 0x7ff84c19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:49.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.743+0000 7ff851021700 1 -- 172.21.15.45:0/1682816287 >> 172.21.15.45:0/1682816287 conn(0x7ff84c078340 msgr2=0x7ff84c100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:49.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.743+0000 7ff851021700 1 -- 172.21.15.45:0/1682816287 shutdown_connections 2024-06-06T04:29:49.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.743+0000 7ff851021700 1 -- 172.21.15.45:0/1682816287 wait complete. 2024-06-06T04:29:49.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:49 smithi086 ceph-mon[36053]: pgmap v382: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T04:29:49.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:49 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1897340988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-06T04:29:49.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:49 smithi154 ceph-mon[36428]: pgmap v382: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-06T04:29:49.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:49 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1897340988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-06T04:29:50.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:50 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1682816287' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-06T04:29:50.760 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:50 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1682816287' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-06T04:29:50.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:50 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1682816287' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-06T04:29:51.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.167+0000 7fb749194700 1 -- 172.21.15.45:0/3110904096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb744101390 msgr2=0x7fb744101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:51.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.167+0000 7fb749194700 1 --2- 172.21.15.45:0/3110904096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb744101390 0x7fb744101810 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb738009990 tx=0x7fb738009ca0 comp rx=0 tx=0).stop 2024-06-06T04:29:51.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.167+0000 7fb749194700 1 -- 172.21.15.45:0/3110904096 shutdown_connections 2024-06-06T04:29:51.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.167+0000 7fb749194700 1 --2- 172.21.15.45:0/3110904096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb744101d50 0x7fb74410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:51.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.167+0000 7fb749194700 1 --2- 172.21.15.45:0/3110904096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb744107400 0x7fb7441077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:51.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.167+0000 7fb749194700 1 --2- 172.21.15.45:0/3110904096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb744101390 0x7fb744101810 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:51.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.167+0000 7fb749194700 1 -- 172.21.15.45:0/3110904096 >> 172.21.15.45:0/3110904096 conn(0x7fb744078340 msgr2=0x7fb744078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:51.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.167+0000 7fb749194700 1 -- 172.21.15.45:0/3110904096 shutdown_connections 2024-06-06T04:29:51.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.167+0000 7fb749194700 1 -- 172.21.15.45:0/3110904096 wait complete. 2024-06-06T04:29:51.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.168+0000 7fb749194700 1 Processor -- start 2024-06-06T04:29:51.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.168+0000 7fb749194700 1 -- start start 2024-06-06T04:29:51.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb749194700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb744101390 0x7fb74419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:51.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb749194700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb744101d50 0x7fb74419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:51.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb749194700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb744107400 0x7fb7441989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:51.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb749194700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb7441115a0 con 0x7fb744101390 2024-06-06T04:29:51.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb749194700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb744111420 con 0x7fb744101d50 2024-06-06T04:29:51.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb749194700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb744111720 con 0x7fb744107400 2024-06-06T04:29:51.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb742d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb744101390 0x7fb74419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:51.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb742d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb744101390 0x7fb74419e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51602/0 (socket says 172.21.15.45:51602) 2024-06-06T04:29:51.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb742d9d700 1 -- 172.21.15.45:0/1897340988 learned_addr learned my addr 172.21.15.45:0/1897340988 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:51.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb74259c700 1 --2- 172.21.15.45:0/1897340988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb744101d50 0x7fb74419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:51.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb74359e700 1 --2- 172.21.15.45:0/1897340988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb744107400 0x7fb7441989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:51.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb742d9d700 1 -- 172.21.15.45:0/1897340988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb744107400 msgr2=0x7fb7441989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:51.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb742d9d700 1 --2- 172.21.15.45:0/1897340988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb744107400 0x7fb7441989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:51.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb742d9d700 1 -- 172.21.15.45:0/1897340988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb744101d50 msgr2=0x7fb74419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:51.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb742d9d700 1 --2- 172.21.15.45:0/1897340988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb744101d50 0x7fb74419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:51.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb742d9d700 1 -- 172.21.15.45:0/1897340988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb744199110 con 0x7fb744101390 2024-06-06T04:29:51.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.169+0000 7fb742d9d700 1 --2- 172.21.15.45:0/1897340988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb744101390 0x7fb74419e7f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb73400d950 tx=0x7fb73400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:51.090 INFO:teuthology.orchestra.run.smithi045.stdout:313532612659 2024-06-06T04:29:51.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.170+0000 7fb72bfff700 1 -- 172.21.15.45:0/1897340988 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734016070 con 0x7fb744101390 2024-06-06T04:29:51.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.170+0000 7fb749194700 1 -- 172.21.15.45:0/1897340988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb744199400 con 0x7fb744101390 2024-06-06T04:29:51.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.170+0000 7fb72bfff700 1 -- 172.21.15.45:0/1897340988 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb73400fd40 con 0x7fb744101390 2024-06-06T04:29:51.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.170+0000 7fb72bfff700 1 -- 172.21.15.45:0/1897340988 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73400e3b0 con 0x7fb744101390 2024-06-06T04:29:51.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.170+0000 7fb749194700 1 -- 172.21.15.45:0/1897340988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7441a56c0 con 0x7fb744101390 2024-06-06T04:29:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.171+0000 7fb749194700 1 -- 172.21.15.45:0/1897340988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb744069180 con 0x7fb744101390 2024-06-06T04:29:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.172+0000 7fb72bfff700 1 -- 172.21.15.45:0/1897340988 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb734019070 con 0x7fb744101390 2024-06-06T04:29:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.173+0000 7fb72bfff700 1 --2- 172.21.15.45:0/1897340988 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb72c0a2cb0 0x7fb72c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.173+0000 7fb74259c700 1 --2- 172.21.15.45:0/1897340988 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb72c0a2cb0 0x7fb72c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.173+0000 7fb72bfff700 1 -- 172.21.15.45:0/1897340988 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fb7340ceff0 con 0x7fb744101390 2024-06-06T04:29:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.173+0000 7fb74259c700 1 --2- 172.21.15.45:0/1897340988 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb72c0a2cb0 0x7fb72c0a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb7380098e0 tx=0x7fb738005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.176+0000 7fb72bfff700 1 -- 172.21.15.45:0/1897340988 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb73409ba30 con 0x7fb744101390 2024-06-06T04:29:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.387+0000 7fb749194700 1 -- 172.21.15.45:0/1897340988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fb7441a58f0 con 0x7fb744101390 2024-06-06T04:29:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.387+0000 7fb72bfff700 1 -- 172.21.15.45:0/1897340988 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb73409b330 con 0x7fb744101390 2024-06-06T04:29:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.389+0000 7fb749194700 1 -- 172.21.15.45:0/1897340988 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb72c0a2cb0 msgr2=0x7fb72c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.389+0000 7fb749194700 1 --2- 172.21.15.45:0/1897340988 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb72c0a2cb0 0x7fb72c0a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb7380098e0 tx=0x7fb738005960 comp rx=0 tx=0).stop 2024-06-06T04:29:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.389+0000 7fb749194700 1 -- 172.21.15.45:0/1897340988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb744101390 msgr2=0x7fb74419e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.389+0000 7fb749194700 1 --2- 172.21.15.45:0/1897340988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb744101390 0x7fb74419e7f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb73400d950 tx=0x7fb73400dc60 comp rx=0 tx=0).stop 2024-06-06T04:29:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.389+0000 7fb749194700 1 -- 172.21.15.45:0/1897340988 shutdown_connections 2024-06-06T04:29:51.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.389+0000 7fb749194700 1 --2- 172.21.15.45:0/1897340988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb744107400 0x7fb7441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:51.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.390+0000 7fb749194700 1 --2- 172.21.15.45:0/1897340988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb744101390 0x7fb74419e7f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:51.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.390+0000 7fb749194700 1 --2- 172.21.15.45:0/1897340988 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb72c0a2cb0 0x7fb72c0a5170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:51.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.390+0000 7fb749194700 1 --2- 172.21.15.45:0/1897340988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb744101d50 0x7fb74419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:51.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.390+0000 7fb749194700 1 -- 172.21.15.45:0/1897340988 >> 172.21.15.45:0/1897340988 conn(0x7fb744078340 msgr2=0x7fb74406a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:51.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.390+0000 7fb749194700 1 -- 172.21.15.45:0/1897340988 shutdown_connections 2024-06-06T04:29:51.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:49.390+0000 7fb749194700 1 -- 172.21.15.45:0/1897340988 wait complete. 2024-06-06T04:29:51.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:51 smithi045 ceph-mon[32299]: pgmap v383: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:29:51.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:51 smithi086 ceph-mon[36053]: pgmap v383: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:29:51.903 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574939 got 55834574944 for osd.1 2024-06-06T04:29:51.903 DEBUG:teuthology.parallel:result is None 2024-06-06T04:29:51.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:51 smithi154 ceph-mon[36428]: pgmap v383: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:29:52.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.902+0000 7f1c131cc700 1 -- 172.21.15.45:0/3064940522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c0c103150 msgr2=0x7f1c0c107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:52.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.902+0000 7f1c131cc700 1 --2- 172.21.15.45:0/3064940522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c0c103150 0x7f1c0c107140 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0099c0 tx=0x7f1bfc009cd0 comp rx=0 tx=0).stop 2024-06-06T04:29:52.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.902+0000 7f1c131cc700 1 -- 172.21.15.45:0/3064940522 shutdown_connections 2024-06-06T04:29:52.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.902+0000 7f1c131cc700 1 --2- 172.21.15.45:0/3064940522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c0c107870 0x7f1c0c10f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:52.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.902+0000 7f1c131cc700 1 --2- 172.21.15.45:0/3064940522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c0c1027a0 0x7f1c0c102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:52.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.902+0000 7f1c131cc700 1 --2- 172.21.15.45:0/3064940522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c0c103150 0x7f1c0c107140 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:52.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.902+0000 7f1c131cc700 1 -- 172.21.15.45:0/3064940522 >> 172.21.15.45:0/3064940522 conn(0x7f1c0c079570 msgr2=0x7f1c0c079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.903+0000 7f1c131cc700 1 -- 172.21.15.45:0/3064940522 shutdown_connections 2024-06-06T04:29:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.903+0000 7f1c131cc700 1 -- 172.21.15.45:0/3064940522 wait complete. 2024-06-06T04:29:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.903+0000 7f1c131cc700 1 Processor -- start 2024-06-06T04:29:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.903+0000 7f1c131cc700 1 -- start start 2024-06-06T04:29:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.903+0000 7f1c131cc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c0c1027a0 0x7f1c0c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.903+0000 7f1c131cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c0c103150 0x7f1c0c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.903+0000 7f1c131cc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c0c107870 0x7f1c0c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.903+0000 7f1c131cc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c112630 con 0x7f1c0c103150 2024-06-06T04:29:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.903+0000 7f1c131cc700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1124b0 con 0x7f1c0c107870 2024-06-06T04:29:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.903+0000 7f1c131cc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1127b0 con 0x7f1c0c1027a0 2024-06-06T04:29:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.904+0000 7f1c03fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c0c103150 0x7f1c0c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.904+0000 7f1c03fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c0c103150 0x7f1c0c074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51640/0 (socket says 172.21.15.45:51640) 2024-06-06T04:29:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.904+0000 7f1c03fff700 1 -- 172.21.15.45:0/3085038348 learned_addr learned my addr 172.21.15.45:0/3085038348 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.904+0000 7f1c11769700 1 --2- 172.21.15.45:0/3085038348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c0c107870 0x7f1c0c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.904+0000 7f1c10f68700 1 --2- 172.21.15.45:0/3085038348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c0c1027a0 0x7f1c0c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.904+0000 7f1c03fff700 1 -- 172.21.15.45:0/3085038348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c0c1027a0 msgr2=0x7f1c0c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.904+0000 7f1c03fff700 1 --2- 172.21.15.45:0/3085038348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c0c1027a0 0x7f1c0c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.904+0000 7f1c03fff700 1 -- 172.21.15.45:0/3085038348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c0c107870 msgr2=0x7f1c0c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.904+0000 7f1c03fff700 1 --2- 172.21.15.45:0/3085038348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c0c107870 0x7f1c0c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.904+0000 7f1c03fff700 1 -- 172.21.15.45:0/3085038348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0c1a6f90 con 0x7f1c0c103150 2024-06-06T04:29:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.904+0000 7f1c03fff700 1 --2- 172.21.15.45:0/3085038348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c0c103150 0x7f1c0c074cd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc006010 tx=0x7f1bfc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.904+0000 7f1c01ffb700 1 -- 172.21.15.45:0/3085038348 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc027070 con 0x7f1c0c103150 2024-06-06T04:29:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.904+0000 7f1c01ffb700 1 -- 172.21.15.45:0/3085038348 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bfc004050 con 0x7f1c0c103150 2024-06-06T04:29:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.904+0000 7f1c131cc700 1 -- 172.21.15.45:0/3085038348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0c1a7220 con 0x7f1c0c103150 2024-06-06T04:29:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.904+0000 7f1c01ffb700 1 -- 172.21.15.45:0/3085038348 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc022da0 con 0x7f1c0c103150 2024-06-06T04:29:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.905+0000 7f1c131cc700 1 -- 172.21.15.45:0/3085038348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c0c1a7670 con 0x7f1c0c103150 2024-06-06T04:29:52.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.905+0000 7f1c131cc700 1 -- 172.21.15.45:0/3085038348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c0c069180 con 0x7f1c0c103150 2024-06-06T04:29:52.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.907+0000 7f1c01ffb700 1 -- 172.21.15.45:0/3085038348 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1bfc02a070 con 0x7f1c0c103150 2024-06-06T04:29:52.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.908+0000 7f1c01ffb700 1 --2- 172.21.15.45:0/3085038348 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1bf40a2cb0 0x7f1bf40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:52.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.908+0000 7f1c10f68700 1 --2- 172.21.15.45:0/3085038348 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1bf40a2cb0 0x7f1bf40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:52.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.908+0000 7f1c01ffb700 1 -- 172.21.15.45:0/3085038348 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f1bfc0d0b60 con 0x7f1c0c103150 2024-06-06T04:29:52.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.908+0000 7f1c10f68700 1 --2- 172.21.15.45:0/3085038348 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1bf40a2cb0 0x7f1bf40a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1c080060b0 tx=0x7f1c08007b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:52.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:52.912+0000 7f1c01ffb700 1 -- 172.21.15.45:0/3085038348 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1bfc031020 con 0x7f1c0c103150 2024-06-06T04:29:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:53.122+0000 7f1c131cc700 1 -- 172.21.15.45:0/3085038348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1c0c02df00 con 0x7f1c0c103150 2024-06-06T04:29:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:53.122+0000 7f1c01ffb700 1 -- 172.21.15.45:0/3085038348 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1bfc013070 con 0x7f1c0c103150 2024-06-06T04:29:53.123 INFO:teuthology.orchestra.run.smithi045.stdout:167503724621 2024-06-06T04:29:53.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:53.125+0000 7f1c131cc700 1 -- 172.21.15.45:0/3085038348 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1bf40a2cb0 msgr2=0x7f1bf40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:53.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:53.125+0000 7f1c131cc700 1 --2- 172.21.15.45:0/3085038348 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1bf40a2cb0 0x7f1bf40a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1c080060b0 tx=0x7f1c08007b80 comp rx=0 tx=0).stop 2024-06-06T04:29:53.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:53.125+0000 7f1c131cc700 1 -- 172.21.15.45:0/3085038348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c0c103150 msgr2=0x7f1c0c074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:53.125+0000 7f1c131cc700 1 --2- 172.21.15.45:0/3085038348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c0c103150 0x7f1c0c074cd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc006010 tx=0x7f1bfc0057c0 comp rx=0 tx=0).stop 2024-06-06T04:29:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:53.125+0000 7f1c131cc700 1 -- 172.21.15.45:0/3085038348 shutdown_connections 2024-06-06T04:29:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:53.125+0000 7f1c131cc700 1 --2- 172.21.15.45:0/3085038348 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1bf40a2cb0 0x7f1bf40a5170 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:53.125+0000 7f1c131cc700 1 --2- 172.21.15.45:0/3085038348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c0c107870 0x7f1c0c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:53.125+0000 7f1c131cc700 1 --2- 172.21.15.45:0/3085038348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c0c1027a0 0x7f1c0c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:53.125+0000 7f1c131cc700 1 --2- 172.21.15.45:0/3085038348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c0c103150 0x7f1c0c074cd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:53.125+0000 7f1c131cc700 1 -- 172.21.15.45:0/3085038348 >> 172.21.15.45:0/3085038348 conn(0x7f1c0c079570 msgr2=0x7f1c0c100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:53.125+0000 7f1c131cc700 1 -- 172.21.15.45:0/3085038348 shutdown_connections 2024-06-06T04:29:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:53.125+0000 7f1c131cc700 1 -- 172.21.15.45:0/3085038348 wait complete. 2024-06-06T04:29:53.613 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:53 smithi045 ceph-mon[32299]: pgmap v384: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:29:53.613 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:53 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3085038348' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-06T04:29:53.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:53 smithi086 ceph-mon[36053]: pgmap v384: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:29:53.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:53 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3085038348' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-06T04:29:53.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:53 smithi154 ceph-mon[36428]: pgmap v384: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:29:53.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:53 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3085038348' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-06T04:29:54.089 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612656 got 313532612659 for osd.10 2024-06-06T04:29:54.089 DEBUG:teuthology.parallel:result is None 2024-06-06T04:29:54.476 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037816 got 257698037820 for osd.8 2024-06-06T04:29:54.476 DEBUG:teuthology.parallel:result is None 2024-06-06T04:29:55.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:55 smithi045 ceph-mon[32299]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:55.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.586+0000 7f10bddb6700 1 -- 172.21.15.45:0/1892928266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b81085d0 msgr2=0x7f10b81089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:55.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.586+0000 7f10bddb6700 1 --2- 172.21.15.45:0/1892928266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b81085d0 0x7f10b81089b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f10a80099c0 tx=0x7f10a8009cd0 comp rx=0 tx=0).stop 2024-06-06T04:29:55.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.586+0000 7f10bddb6700 1 -- 172.21.15.45:0/1892928266 shutdown_connections 2024-06-06T04:29:55.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.586+0000 7f10bddb6700 1 --2- 172.21.15.45:0/1892928266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b8102ef0 0x7f10b810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:55.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.586+0000 7f10bddb6700 1 --2- 172.21.15.45:0/1892928266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b81085d0 0x7f10b81089b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:55.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.586+0000 7f10bddb6700 1 --2- 172.21.15.45:0/1892928266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b8102530 0x7f10b81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:55.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.586+0000 7f10bddb6700 1 -- 172.21.15.45:0/1892928266 >> 172.21.15.45:0/1892928266 conn(0x7f10b8079640 msgr2=0x7f10b8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:55.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.586+0000 7f10bddb6700 1 -- 172.21.15.45:0/1892928266 shutdown_connections 2024-06-06T04:29:55.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.586+0000 7f10bddb6700 1 -- 172.21.15.45:0/1892928266 wait complete. 2024-06-06T04:29:55.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.587+0000 7f10bddb6700 1 Processor -- start 2024-06-06T04:29:55.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.587+0000 7f10bddb6700 1 -- start start 2024-06-06T04:29:55.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.587+0000 7f10bddb6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b8102530 0x7f10b8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:55.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.587+0000 7f10bddb6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b8102ef0 0x7f10b8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:55.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.587+0000 7f10bddb6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b81085d0 0x7f10b8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:55.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.587+0000 7f10bddb6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81127b0 con 0x7f10b8102530 2024-06-06T04:29:55.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.587+0000 7f10bddb6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10b8112630 con 0x7f10b81085d0 2024-06-06T04:29:55.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.587+0000 7f10bddb6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f10b8112930 con 0x7f10b8102ef0 2024-06-06T04:29:55.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.587+0000 7f10b77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b8102530 0x7f10b8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:55.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.588+0000 7f10b77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b8102530 0x7f10b8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51666/0 (socket says 172.21.15.45:51666) 2024-06-06T04:29:55.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.588+0000 7f10b77fe700 1 -- 172.21.15.45:0/3890821436 learned_addr learned my addr 172.21.15.45:0/3890821436 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:55.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.588+0000 7f10b7fff700 1 --2- 172.21.15.45:0/3890821436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b81085d0 0x7f10b8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:55.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.588+0000 7f10b6ffd700 1 --2- 172.21.15.45:0/3890821436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b8102ef0 0x7f10b8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:55.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.588+0000 7f10b77fe700 1 -- 172.21.15.45:0/3890821436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b8102ef0 msgr2=0x7f10b8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:55.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.588+0000 7f10b77fe700 1 --2- 172.21.15.45:0/3890821436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b8102ef0 0x7f10b8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:55.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.588+0000 7f10b77fe700 1 -- 172.21.15.45:0/3890821436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b81085d0 msgr2=0x7f10b8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:55.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.588+0000 7f10b77fe700 1 --2- 172.21.15.45:0/3890821436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b81085d0 0x7f10b8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:55.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.588+0000 7f10b77fe700 1 -- 172.21.15.45:0/3890821436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b81a7160 con 0x7f10b8102530 2024-06-06T04:29:55.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.588+0000 7f10b77fe700 1 --2- 172.21.15.45:0/3890821436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b8102530 0x7f10b8074790 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f10a8006010 tx=0x7f10a80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:55.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.588+0000 7f10b4ff9700 1 -- 172.21.15.45:0/3890821436 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8027070 con 0x7f10b8102530 2024-06-06T04:29:55.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.588+0000 7f10b4ff9700 1 -- 172.21.15.45:0/3890821436 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10a8003f50 con 0x7f10b8102530 2024-06-06T04:29:55.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.588+0000 7f10b4ff9700 1 -- 172.21.15.45:0/3890821436 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8022d10 con 0x7f10b8102530 2024-06-06T04:29:55.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.588+0000 7f10bddb6700 1 -- 172.21.15.45:0/3890821436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b81a73f0 con 0x7f10b8102530 2024-06-06T04:29:55.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.589+0000 7f10bddb6700 1 -- 172.21.15.45:0/3890821436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b81a77f0 con 0x7f10b8102530 2024-06-06T04:29:55.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.590+0000 7f10bddb6700 1 -- 172.21.15.45:0/3890821436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10b810aa60 con 0x7f10b8102530 2024-06-06T04:29:55.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.592+0000 7f10b4ff9700 1 -- 172.21.15.45:0/3890821436 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f10a802a070 con 0x7f10b8102530 2024-06-06T04:29:55.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.593+0000 7f10b4ff9700 1 --2- 172.21.15.45:0/3890821436 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f10a40a2d30 0x7f10a40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:55.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.593+0000 7f10b6ffd700 1 --2- 172.21.15.45:0/3890821436 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f10a40a2d30 0x7f10a40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:55.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.593+0000 7f10b4ff9700 1 -- 172.21.15.45:0/3890821436 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f10a80d0370 con 0x7f10b8102530 2024-06-06T04:29:55.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.593+0000 7f10b6ffd700 1 --2- 172.21.15.45:0/3890821436 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f10a40a2d30 0x7f10a40a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f10b8079c90 tx=0x7f10a0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:55.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.601+0000 7f10b4ff9700 1 -- 172.21.15.45:0/3890821436 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f10a805e850 con 0x7f10b8102530 2024-06-06T04:29:55.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.818+0000 7f10bddb6700 1 -- 172.21.15.45:0/3890821436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f10b802dca0 con 0x7f10b8102530 2024-06-06T04:29:55.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.818+0000 7f10b4ff9700 1 -- 172.21.15.45:0/3890821436 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f10a8013070 con 0x7f10b8102530 2024-06-06T04:29:55.819 INFO:teuthology.orchestra.run.smithi045.stdout:146028888146 2024-06-06T04:29:55.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.821+0000 7f10bddb6700 1 -- 172.21.15.45:0/3890821436 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f10a40a2d30 msgr2=0x7f10a40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.821+0000 7f10bddb6700 1 --2- 172.21.15.45:0/3890821436 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f10a40a2d30 0x7f10a40a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f10b8079c90 tx=0x7f10a0009190 comp rx=0 tx=0).stop 2024-06-06T04:29:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.821+0000 7f10bddb6700 1 -- 172.21.15.45:0/3890821436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b8102530 msgr2=0x7f10b8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.821+0000 7f10bddb6700 1 --2- 172.21.15.45:0/3890821436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b8102530 0x7f10b8074790 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f10a8006010 tx=0x7f10a80057c0 comp rx=0 tx=0).stop 2024-06-06T04:29:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.821+0000 7f10bddb6700 1 -- 172.21.15.45:0/3890821436 shutdown_connections 2024-06-06T04:29:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.821+0000 7f10bddb6700 1 --2- 172.21.15.45:0/3890821436 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f10a40a2d30 0x7f10a40a51f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.821+0000 7f10bddb6700 1 --2- 172.21.15.45:0/3890821436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b81085d0 0x7f10b8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.821+0000 7f10bddb6700 1 --2- 172.21.15.45:0/3890821436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b8102530 0x7f10b8074790 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.821+0000 7f10bddb6700 1 --2- 172.21.15.45:0/3890821436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b8102ef0 0x7f10b8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.821+0000 7f10bddb6700 1 -- 172.21.15.45:0/3890821436 >> 172.21.15.45:0/3890821436 conn(0x7f10b8079640 msgr2=0x7f10b8101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.821+0000 7f10bddb6700 1 -- 172.21.15.45:0/3890821436 shutdown_connections 2024-06-06T04:29:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:55.821+0000 7f10bddb6700 1 -- 172.21.15.45:0/3890821436 wait complete. 2024-06-06T04:29:55.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:55 smithi086 ceph-mon[36053]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:55.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:55 smithi154 ceph-mon[36428]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:29:56.373 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:29:56.821 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:56 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3890821436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-06T04:29:56.856 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888142 got 146028888146 for osd.4 2024-06-06T04:29:56.856 DEBUG:teuthology.parallel:result is None 2024-06-06T04:29:56.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:56 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3890821436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-06T04:29:56.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:56 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3890821436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-06T04:29:57.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.525+0000 7fe5ea133700 1 -- 172.21.15.45:0/1837417203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5e4107840 msgr2=0x7fe5e410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:57.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.525+0000 7fe5ea133700 1 --2- 172.21.15.45:0/1837417203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5e4107840 0x7fe5e410f3c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00b0d0 tx=0x7fe5dc00b3e0 comp rx=0 tx=0).stop 2024-06-06T04:29:57.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.525+0000 7fe5ea133700 1 -- 172.21.15.45:0/1837417203 shutdown_connections 2024-06-06T04:29:57.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.525+0000 7fe5ea133700 1 --2- 172.21.15.45:0/1837417203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5e4107840 0x7fe5e410f3c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:57.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.525+0000 7fe5ea133700 1 --2- 172.21.15.45:0/1837417203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e4102740 0x7fe5e4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:57.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.525+0000 7fe5ea133700 1 --2- 172.21.15.45:0/1837417203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e41030f0 0x7fe5e4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:57.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.525+0000 7fe5ea133700 1 -- 172.21.15.45:0/1837417203 >> 172.21.15.45:0/1837417203 conn(0x7fe5e40795f0 msgr2=0x7fe5e4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.525+0000 7fe5ea133700 1 -- 172.21.15.45:0/1837417203 shutdown_connections 2024-06-06T04:29:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.525+0000 7fe5ea133700 1 -- 172.21.15.45:0/1837417203 wait complete. 2024-06-06T04:29:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.525+0000 7fe5ea133700 1 Processor -- start 2024-06-06T04:29:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.525+0000 7fe5ea133700 1 -- start start 2024-06-06T04:29:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5ea133700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5e4102740 0x7fe5e4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5ea133700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e41030f0 0x7fe5e4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5ea133700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4107840 0x7fe5e4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5ea133700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4112690 con 0x7fe5e4102740 2024-06-06T04:29:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5ea133700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4112510 con 0x7fe5e4107840 2024-06-06T04:29:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5ea133700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4112810 con 0x7fe5e41030f0 2024-06-06T04:29:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5e37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5e4102740 0x7fe5e4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5e37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5e4102740 0x7fe5e4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51686/0 (socket says 172.21.15.45:51686) 2024-06-06T04:29:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5e37fe700 1 -- 172.21.15.45:0/361423928 learned_addr learned my addr 172.21.15.45:0/361423928 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5e2ffd700 1 --2- 172.21.15.45:0/361423928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e41030f0 0x7fe5e4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5e3fff700 1 --2- 172.21.15.45:0/361423928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4107840 0x7fe5e4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5e37fe700 1 -- 172.21.15.45:0/361423928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e41030f0 msgr2=0x7fe5e4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5e37fe700 1 --2- 172.21.15.45:0/361423928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e41030f0 0x7fe5e4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5e37fe700 1 -- 172.21.15.45:0/361423928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4107840 msgr2=0x7fe5e4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5e37fe700 1 --2- 172.21.15.45:0/361423928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4107840 0x7fe5e4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5e37fe700 1 -- 172.21.15.45:0/361423928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e41a6f60 con 0x7fe5e4102740 2024-06-06T04:29:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5e37fe700 1 --2- 172.21.15.45:0/361423928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5e4102740 0x7fe5e4074790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400b530 tx=0x7fe5d400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5e0ff9700 1 -- 172.21.15.45:0/361423928 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4015020 con 0x7fe5e4102740 2024-06-06T04:29:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5e0ff9700 1 -- 172.21.15.45:0/361423928 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5d4011ce0 con 0x7fe5e4102740 2024-06-06T04:29:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.526+0000 7fe5e0ff9700 1 -- 172.21.15.45:0/361423928 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4010e40 con 0x7fe5e4102740 2024-06-06T04:29:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.527+0000 7fe5ea133700 1 -- 172.21.15.45:0/361423928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e41a7250 con 0x7fe5e4102740 2024-06-06T04:29:57.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.527+0000 7fe5ea133700 1 -- 172.21.15.45:0/361423928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e41a7790 con 0x7fe5e4102740 2024-06-06T04:29:57.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.527+0000 7fe5ea133700 1 -- 172.21.15.45:0/361423928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e410aa80 con 0x7fe5e4102740 2024-06-06T04:29:57.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.529+0000 7fe5e0ff9700 1 -- 172.21.15.45:0/361423928 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe5d4018020 con 0x7fe5e4102740 2024-06-06T04:29:57.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.529+0000 7fe5e0ff9700 1 --2- 172.21.15.45:0/361423928 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5cc0a2cb0 0x7fe5cc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:57.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.530+0000 7fe5e0ff9700 1 -- 172.21.15.45:0/361423928 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fe5d40cf260 con 0x7fe5e4102740 2024-06-06T04:29:57.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.530+0000 7fe5e2ffd700 1 --2- 172.21.15.45:0/361423928 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5cc0a2cb0 0x7fe5cc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:57.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.530+0000 7fe5e2ffd700 1 --2- 172.21.15.45:0/361423928 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5cc0a2cb0 0x7fe5cc0a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4101f80 tx=0x7fe5d800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:57.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.533+0000 7fe5e0ff9700 1 -- 172.21.15.45:0/361423928 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe5d401f020 con 0x7fe5e4102740 2024-06-06T04:29:57.721 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:57 smithi045 ceph-mon[32299]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:29:57.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.744+0000 7fe5ea133700 1 -- 172.21.15.45:0/361423928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe5e410a460 con 0x7fe5e4102740 2024-06-06T04:29:57.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.744+0000 7fe5e0ff9700 1 -- 172.21.15.45:0/361423928 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe5d4010380 con 0x7fe5e4102740 2024-06-06T04:29:57.745 INFO:teuthology.orchestra.run.smithi045.stdout:111669149783 2024-06-06T04:29:57.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.746+0000 7fe5ea133700 1 -- 172.21.15.45:0/361423928 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5cc0a2cb0 msgr2=0x7fe5cc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:57.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.746+0000 7fe5ea133700 1 --2- 172.21.15.45:0/361423928 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5cc0a2cb0 0x7fe5cc0a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4101f80 tx=0x7fe5d800a380 comp rx=0 tx=0).stop 2024-06-06T04:29:57.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.746+0000 7fe5ea133700 1 -- 172.21.15.45:0/361423928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5e4102740 msgr2=0x7fe5e4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:57.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.746+0000 7fe5ea133700 1 --2- 172.21.15.45:0/361423928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5e4102740 0x7fe5e4074790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400b530 tx=0x7fe5d400b840 comp rx=0 tx=0).stop 2024-06-06T04:29:57.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.747+0000 7fe5ea133700 1 -- 172.21.15.45:0/361423928 shutdown_connections 2024-06-06T04:29:57.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.747+0000 7fe5ea133700 1 --2- 172.21.15.45:0/361423928 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe5cc0a2cb0 0x7fe5cc0a5170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:57.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.747+0000 7fe5ea133700 1 --2- 172.21.15.45:0/361423928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5e4107840 0x7fe5e4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:57.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.747+0000 7fe5ea133700 1 --2- 172.21.15.45:0/361423928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5e4102740 0x7fe5e4074790 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:57.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.747+0000 7fe5ea133700 1 --2- 172.21.15.45:0/361423928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e41030f0 0x7fe5e4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:57.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.747+0000 7fe5ea133700 1 -- 172.21.15.45:0/361423928 >> 172.21.15.45:0/361423928 conn(0x7fe5e40795f0 msgr2=0x7fe5e4100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:57.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.747+0000 7fe5ea133700 1 -- 172.21.15.45:0/361423928 shutdown_connections 2024-06-06T04:29:57.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:57.747+0000 7fe5ea133700 1 -- 172.21.15.45:0/361423928 wait complete. 2024-06-06T04:29:57.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:57 smithi086 ceph-mon[36053]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:29:57.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:57 smithi154 ceph-mon[36428]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:29:58.504 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:58 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/361423928' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-06T04:29:58.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:58 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/361423928' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-06T04:29:58.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:58 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/361423928' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-06T04:29:59.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.093+0000 7fd157393700 1 -- 172.21.15.45:0/849500757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd150108500 msgr2=0x7fd1501088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:59.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.093+0000 7fd157393700 1 --2- 172.21.15.45:0/849500757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd150108500 0x7fd1501088e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009990 tx=0x7fd14c009ca0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.093+0000 7fd157393700 1 -- 172.21.15.45:0/849500757 shutdown_connections 2024-06-06T04:29:59.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.093+0000 7fd157393700 1 --2- 172.21.15.45:0/849500757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd15006ab30 0x7fd15010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.093+0000 7fd157393700 1 --2- 172.21.15.45:0/849500757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd150108500 0x7fd1501088e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.093+0000 7fd157393700 1 --2- 172.21.15.45:0/849500757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd15006a170 0x7fd15006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.093+0000 7fd157393700 1 -- 172.21.15.45:0/849500757 >> 172.21.15.45:0/849500757 conn(0x7fd150078340 msgr2=0x7fd150078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.093+0000 7fd157393700 1 -- 172.21.15.45:0/849500757 shutdown_connections 2024-06-06T04:29:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.093+0000 7fd157393700 1 -- 172.21.15.45:0/849500757 wait complete. 2024-06-06T04:29:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.094+0000 7fd157393700 1 Processor -- start 2024-06-06T04:29:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.094+0000 7fd157393700 1 -- start start 2024-06-06T04:29:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.094+0000 7fd157393700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd15006a170 0x7fd15019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.094+0000 7fd157393700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd15006ab30 0x7fd15019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.094+0000 7fd157393700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd150108500 0x7fd150198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.094+0000 7fd157393700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd150112610 con 0x7fd15006a170 2024-06-06T04:29:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.094+0000 7fd157393700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd150112490 con 0x7fd150108500 2024-06-06T04:29:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.094+0000 7fd157393700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd150112790 con 0x7fd15006ab30 2024-06-06T04:29:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.094+0000 7fd15512f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd15006a170 0x7fd15019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.094+0000 7fd15512f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd15006a170 0x7fd15019e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50340/0 (socket says 172.21.15.45:50340) 2024-06-06T04:29:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.094+0000 7fd15512f700 1 -- 172.21.15.45:0/3622158491 learned_addr learned my addr 172.21.15.45:0/3622158491 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.095+0000 7fd15492e700 1 --2- 172.21.15.45:0/3622158491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd15006ab30 0x7fd15019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.095+0000 7fd155930700 1 --2- 172.21.15.45:0/3622158491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd150108500 0x7fd150198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.095+0000 7fd15512f700 1 -- 172.21.15.45:0/3622158491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd15006ab30 msgr2=0x7fd15019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.095+0000 7fd15512f700 1 --2- 172.21.15.45:0/3622158491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd15006ab30 0x7fd15019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.095+0000 7fd15512f700 1 -- 172.21.15.45:0/3622158491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd150108500 msgr2=0x7fd150198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.095+0000 7fd15512f700 1 --2- 172.21.15.45:0/3622158491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd150108500 0x7fd150198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.095+0000 7fd15512f700 1 -- 172.21.15.45:0/3622158491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1501990d0 con 0x7fd15006a170 2024-06-06T04:29:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.095+0000 7fd15512f700 1 --2- 172.21.15.45:0/3622158491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd15006a170 0x7fd15019e7b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd14c003ab0 tx=0x7fd14c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.095+0000 7fd1467fc700 1 -- 172.21.15.45:0/3622158491 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c027070 con 0x7fd15006a170 2024-06-06T04:29:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.095+0000 7fd1467fc700 1 -- 172.21.15.45:0/3622158491 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd14c00bdc0 con 0x7fd15006a170 2024-06-06T04:29:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.095+0000 7fd157393700 1 -- 172.21.15.45:0/3622158491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd150199360 con 0x7fd15006a170 2024-06-06T04:29:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.095+0000 7fd1467fc700 1 -- 172.21.15.45:0/3622158491 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c022d40 con 0x7fd15006a170 2024-06-06T04:29:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.095+0000 7fd157393700 1 -- 172.21.15.45:0/3622158491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1501a5680 con 0x7fd15006a170 2024-06-06T04:29:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.096+0000 7fd157393700 1 -- 172.21.15.45:0/3622158491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd15010a9e0 con 0x7fd15006a170 2024-06-06T04:29:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.097+0000 7fd1467fc700 1 -- 172.21.15.45:0/3622158491 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd14c02a070 con 0x7fd15006a170 2024-06-06T04:29:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.098+0000 7fd1467fc700 1 --2- 172.21.15.45:0/3622158491 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd13c0a2cb0 0x7fd13c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.098+0000 7fd15492e700 1 --2- 172.21.15.45:0/3622158491 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd13c0a2cb0 0x7fd13c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.098+0000 7fd1467fc700 1 -- 172.21.15.45:0/3622158491 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fd14c0d0680 con 0x7fd15006a170 2024-06-06T04:29:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.098+0000 7fd15492e700 1 --2- 172.21.15.45:0/3622158491 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd13c0a2cb0 0x7fd13c0a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd140005950 tx=0x7fd1400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:59.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.102+0000 7fd1467fc700 1 -- 172.21.15.45:0/3622158491 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd14c013070 con 0x7fd15006a170 2024-06-06T04:29:59.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.312+0000 7fd157393700 1 -- 172.21.15.45:0/3622158491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd150199ea0 con 0x7fd15006a170 2024-06-06T04:29:59.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.313+0000 7fd1467fc700 1 -- 172.21.15.45:0/3622158491 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd14c09d0f0 con 0x7fd15006a170 2024-06-06T04:29:59.313 INFO:teuthology.orchestra.run.smithi045.stdout:77309411422 2024-06-06T04:29:59.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.315+0000 7fd157393700 1 -- 172.21.15.45:0/3622158491 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd13c0a2cb0 msgr2=0x7fd13c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:59.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.315+0000 7fd157393700 1 --2- 172.21.15.45:0/3622158491 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd13c0a2cb0 0x7fd13c0a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd140005950 tx=0x7fd1400058e0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.315+0000 7fd157393700 1 -- 172.21.15.45:0/3622158491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd15006a170 msgr2=0x7fd15019e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:59.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.315+0000 7fd157393700 1 --2- 172.21.15.45:0/3622158491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd15006a170 0x7fd15019e7b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd14c003ab0 tx=0x7fd14c003b90 comp rx=0 tx=0).stop 2024-06-06T04:29:59.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.315+0000 7fd157393700 1 -- 172.21.15.45:0/3622158491 shutdown_connections 2024-06-06T04:29:59.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.315+0000 7fd157393700 1 --2- 172.21.15.45:0/3622158491 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd13c0a2cb0 0x7fd13c0a5170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.315+0000 7fd157393700 1 --2- 172.21.15.45:0/3622158491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd150108500 0x7fd150198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.315+0000 7fd157393700 1 --2- 172.21.15.45:0/3622158491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd15006a170 0x7fd15019e7b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.315+0000 7fd157393700 1 --2- 172.21.15.45:0/3622158491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd15006ab30 0x7fd15019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.315+0000 7fd157393700 1 -- 172.21.15.45:0/3622158491 >> 172.21.15.45:0/3622158491 conn(0x7fd150078340 msgr2=0x7fd1501001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:59.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.315+0000 7fd157393700 1 -- 172.21.15.45:0/3622158491 shutdown_connections 2024-06-06T04:29:59.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.316+0000 7fd157393700 1 -- 172.21.15.45:0/3622158491 wait complete. 2024-06-06T04:29:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:59 smithi045 ceph-mon[32299]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:29:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:29:59 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3622158491' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-06T04:29:59.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.852+0000 7f454eab8700 1 -- 172.21.15.45:0/1319394366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4548108510 msgr2=0x7f45481088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:59.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.852+0000 7f454eab8700 1 --2- 172.21.15.45:0/1319394366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4548108510 0x7f45481088f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f453c009990 tx=0x7f453c009ca0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.852+0000 7f454eab8700 1 -- 172.21.15.45:0/1319394366 shutdown_connections 2024-06-06T04:29:59.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.852+0000 7f454eab8700 1 --2- 172.21.15.45:0/1319394366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4548101440 0x7f454810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.852+0000 7f454eab8700 1 --2- 172.21.15.45:0/1319394366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4548108510 0x7f45481088f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.852+0000 7f454eab8700 1 --2- 172.21.15.45:0/1319394366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4548100a80 0x7f4548100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.852+0000 7f454eab8700 1 -- 172.21.15.45:0/1319394366 >> 172.21.15.45:0/1319394366 conn(0x7f4548077a00 msgr2=0x7f45480fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:29:59.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.852+0000 7f454eab8700 1 -- 172.21.15.45:0/1319394366 shutdown_connections 2024-06-06T04:29:59.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.852+0000 7f454eab8700 1 -- 172.21.15.45:0/1319394366 wait complete. 2024-06-06T04:29:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.853+0000 7f454eab8700 1 Processor -- start 2024-06-06T04:29:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.853+0000 7f454eab8700 1 -- start start 2024-06-06T04:29:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.853+0000 7f454eab8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4548100a80 0x7f4548101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.853+0000 7f454eab8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4548101440 0x7f45481021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.853+0000 7f454eab8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4548108510 0x7f45481026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.853+0000 7f454eab8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f45481126a0 con 0x7f4548101440 2024-06-06T04:29:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.853+0000 7f454eab8700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4548112520 con 0x7f4548108510 2024-06-06T04:29:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.853+0000 7f454eab8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4548112820 con 0x7f4548100a80 2024-06-06T04:29:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.853+0000 7f4547fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4548101440 0x7f45481021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.853+0000 7f4547fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4548101440 0x7f45481021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50346/0 (socket says 172.21.15.45:50346) 2024-06-06T04:29:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.853+0000 7f4547fff700 1 -- 172.21.15.45:0/1100651833 learned_addr learned my addr 172.21.15.45:0/1100651833 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:29:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.853+0000 7f454c854700 1 --2- 172.21.15.45:0/1100651833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4548100a80 0x7f4548101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.854+0000 7f454d055700 1 --2- 172.21.15.45:0/1100651833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4548108510 0x7f45481026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.854+0000 7f4547fff700 1 -- 172.21.15.45:0/1100651833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4548100a80 msgr2=0x7f4548101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.854+0000 7f4547fff700 1 --2- 172.21.15.45:0/1100651833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4548100a80 0x7f4548101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.854+0000 7f4547fff700 1 -- 172.21.15.45:0/1100651833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4548108510 msgr2=0x7f45481026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:29:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.854+0000 7f4547fff700 1 --2- 172.21.15.45:0/1100651833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4548108510 0x7f45481026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:29:59.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.854+0000 7f4547fff700 1 -- 172.21.15.45:0/1100651833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45481a7090 con 0x7f4548101440 2024-06-06T04:29:59.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.854+0000 7f454c854700 1 --2- 172.21.15.45:0/1100651833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4548100a80 0x7f4548101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:29:59.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.854+0000 7f4547fff700 1 --2- 172.21.15.45:0/1100651833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4548101440 0x7f45481021a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f453400b530 tx=0x7f453400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:59.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.854+0000 7f4545ffb700 1 -- 172.21.15.45:0/1100651833 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534015020 con 0x7f4548101440 2024-06-06T04:29:59.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.854+0000 7f454eab8700 1 -- 172.21.15.45:0/1100651833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45481a7380 con 0x7f4548101440 2024-06-06T04:29:59.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.854+0000 7f4545ffb700 1 -- 172.21.15.45:0/1100651833 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4534011ce0 con 0x7f4548101440 2024-06-06T04:29:59.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.855+0000 7f4545ffb700 1 -- 172.21.15.45:0/1100651833 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534010e40 con 0x7f4548101440 2024-06-06T04:29:59.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.855+0000 7f454eab8700 1 -- 172.21.15.45:0/1100651833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45481a7910 con 0x7f4548101440 2024-06-06T04:29:59.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.857+0000 7f454eab8700 1 -- 172.21.15.45:0/1100651833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4548103140 con 0x7f4548101440 2024-06-06T04:29:59.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.857+0000 7f4545ffb700 1 -- 172.21.15.45:0/1100651833 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4534018020 con 0x7f4548101440 2024-06-06T04:29:59.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.858+0000 7f4545ffb700 1 --2- 172.21.15.45:0/1100651833 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f45380a2d30 0x7f45380a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:29:59.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.858+0000 7f454c854700 1 --2- 172.21.15.45:0/1100651833 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f45380a2d30 0x7f45380a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:29:59.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.858+0000 7f4545ffb700 1 -- 172.21.15.45:0/1100651833 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f45340cec80 con 0x7f4548101440 2024-06-06T04:29:59.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:59 smithi086 ceph-mon[36053]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:29:59.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:29:59 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3622158491' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-06T04:29:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.859+0000 7f454c854700 1 --2- 172.21.15.45:0/1100651833 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f45380a2d30 0x7f45380a51f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f453c00b5c0 tx=0x7f453c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:29:59.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:29:59.863+0000 7f4545ffb700 1 -- 172.21.15.45:0/1100651833 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f453401f020 con 0x7f4548101440 2024-06-06T04:29:59.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:59 smithi154 ceph-mon[36428]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:29:59.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:29:59 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3622158491' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-06T04:30:00.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:00.071+0000 7f454eab8700 1 -- 172.21.15.45:0/1100651833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f4548103d80 con 0x7f4548101440 2024-06-06T04:30:00.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:00.072+0000 7f4545ffb700 1 -- 172.21.15.45:0/1100651833 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f4534010380 con 0x7f4548101440 2024-06-06T04:30:00.072 INFO:teuthology.orchestra.run.smithi045.stdout:347892351024 2024-06-06T04:30:00.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:00.074+0000 7f454eab8700 1 -- 172.21.15.45:0/1100651833 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f45380a2d30 msgr2=0x7f45380a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:00.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:00.074+0000 7f454eab8700 1 --2- 172.21.15.45:0/1100651833 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f45380a2d30 0x7f45380a51f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f453c00b5c0 tx=0x7f453c005fb0 comp rx=0 tx=0).stop 2024-06-06T04:30:00.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:00.074+0000 7f454eab8700 1 -- 172.21.15.45:0/1100651833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4548101440 msgr2=0x7f45481021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:00.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:00.074+0000 7f454eab8700 1 --2- 172.21.15.45:0/1100651833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4548101440 0x7f45481021a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f453400b530 tx=0x7f453400b840 comp rx=0 tx=0).stop 2024-06-06T04:30:00.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:00.075+0000 7f454eab8700 1 -- 172.21.15.45:0/1100651833 shutdown_connections 2024-06-06T04:30:00.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:00.075+0000 7f454eab8700 1 --2- 172.21.15.45:0/1100651833 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f45380a2d30 0x7f45380a51f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:00.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:00.075+0000 7f454eab8700 1 --2- 172.21.15.45:0/1100651833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4548108510 0x7f45481026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:00.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:00.075+0000 7f454eab8700 1 --2- 172.21.15.45:0/1100651833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4548100a80 0x7f4548101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:00.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:00.075+0000 7f454eab8700 1 --2- 172.21.15.45:0/1100651833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4548101440 0x7f45481021a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:00.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:00.075+0000 7f454eab8700 1 -- 172.21.15.45:0/1100651833 >> 172.21.15.45:0/1100651833 conn(0x7f4548077a00 msgr2=0x7f45480ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:00.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:00.075+0000 7f454eab8700 1 -- 172.21.15.45:0/1100651833 shutdown_connections 2024-06-06T04:30:00.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:00.075+0000 7f454eab8700 1 -- 172.21.15.45:0/1100651833 wait complete. 2024-06-06T04:30:00.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:00 smithi045 ceph-mon[32299]: overall HEALTH_OK 2024-06-06T04:30:00.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:00 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1100651833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-06T04:30:00.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:30:00.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:00 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:30:00.841 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149778 got 111669149783 for osd.3 2024-06-06T04:30:00.841 DEBUG:teuthology.parallel:result is None 2024-06-06T04:30:00.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:00 smithi086 ceph-mon[36053]: overall HEALTH_OK 2024-06-06T04:30:00.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:00 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1100651833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-06T04:30:00.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:30:00.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:00 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:30:00.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:00 smithi154 ceph-mon[36428]: overall HEALTH_OK 2024-06-06T04:30:00.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:00 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1100651833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-06T04:30:00.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:30:00.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:00 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:30:01.498 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:01.528 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411417 got 77309411422 for osd.2 2024-06-06T04:30:01.528 DEBUG:teuthology.parallel:result is None 2024-06-06T04:30:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.713+0000 7fbdbcc0f700 1 -- 172.21.15.45:0/1412168912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb8107840 msgr2=0x7fbdb810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.713+0000 7fbdbcc0f700 1 --2- 172.21.15.45:0/1412168912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb8107840 0x7fbdb810f3c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000b0d0 tx=0x7fbdb000b3e0 comp rx=0 tx=0).stop 2024-06-06T04:30:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.713+0000 7fbdbcc0f700 1 -- 172.21.15.45:0/1412168912 shutdown_connections 2024-06-06T04:30:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.713+0000 7fbdbcc0f700 1 --2- 172.21.15.45:0/1412168912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb8107840 0x7fbdb810f3c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.713+0000 7fbdbcc0f700 1 --2- 172.21.15.45:0/1412168912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdb8102740 0x7fbdb8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.713+0000 7fbdbcc0f700 1 --2- 172.21.15.45:0/1412168912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdb81030f0 0x7fbdb8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.713+0000 7fbdbcc0f700 1 -- 172.21.15.45:0/1412168912 >> 172.21.15.45:0/1412168912 conn(0x7fbdb80795f0 msgr2=0x7fbdb8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.713+0000 7fbdbcc0f700 1 -- 172.21.15.45:0/1412168912 shutdown_connections 2024-06-06T04:30:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.713+0000 7fbdbcc0f700 1 -- 172.21.15.45:0/1412168912 wait complete. 2024-06-06T04:30:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdbcc0f700 1 Processor -- start 2024-06-06T04:30:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdbcc0f700 1 -- start start 2024-06-06T04:30:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdbcc0f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb8102740 0x7fbdb8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdbcc0f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdb81030f0 0x7fbdb8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdbcc0f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdb8107840 0x7fbdb8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdbcc0f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb8112610 con 0x7fbdb8102740 2024-06-06T04:30:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdbcc0f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb8112490 con 0x7fbdb81030f0 2024-06-06T04:30:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdbcc0f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb8112790 con 0x7fbdb8107840 2024-06-06T04:30:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdb659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb8102740 0x7fbdb8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdb659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb8102740 0x7fbdb8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50372/0 (socket says 172.21.15.45:50372) 2024-06-06T04:30:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdb659c700 1 -- 172.21.15.45:0/931816739 learned_addr learned my addr 172.21.15.45:0/931816739 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdb6d9d700 1 --2- 172.21.15.45:0/931816739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdb8107840 0x7fbdb8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdb5d9b700 1 --2- 172.21.15.45:0/931816739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdb81030f0 0x7fbdb8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdb659c700 1 -- 172.21.15.45:0/931816739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdb8107840 msgr2=0x7fbdb8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdb659c700 1 --2- 172.21.15.45:0/931816739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdb8107840 0x7fbdb8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdb659c700 1 -- 172.21.15.45:0/931816739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdb81030f0 msgr2=0x7fbdb8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdb659c700 1 --2- 172.21.15.45:0/931816739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdb81030f0 0x7fbdb8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdb659c700 1 -- 172.21.15.45:0/931816739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb81a6f20 con 0x7fbdb8102740 2024-06-06T04:30:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.714+0000 7fbdb5d9b700 1 --2- 172.21.15.45:0/931816739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdb81030f0 0x7fbdb8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:30:01.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.715+0000 7fbdb659c700 1 --2- 172.21.15.45:0/931816739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb8102740 0x7fbdb8074790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbda800c9b0 tx=0x7fbda800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.715+0000 7fbda77fe700 1 -- 172.21.15.45:0/931816739 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8007b00 con 0x7fbdb8102740 2024-06-06T04:30:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.715+0000 7fbda77fe700 1 -- 172.21.15.45:0/931816739 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbda8007ca0 con 0x7fbdb8102740 2024-06-06T04:30:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.715+0000 7fbdbcc0f700 1 -- 172.21.15.45:0/931816739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb81a7210 con 0x7fbdb8102740 2024-06-06T04:30:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.715+0000 7fbda77fe700 1 -- 172.21.15.45:0/931816739 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8018660 con 0x7fbdb8102740 2024-06-06T04:30:01.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.715+0000 7fbdbcc0f700 1 -- 172.21.15.45:0/931816739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb81a7750 con 0x7fbdb8102740 2024-06-06T04:30:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.716+0000 7fbdbcc0f700 1 -- 172.21.15.45:0/931816739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb8069180 con 0x7fbdb8102740 2024-06-06T04:30:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.717+0000 7fbda77fe700 1 -- 172.21.15.45:0/931816739 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbda800f450 con 0x7fbdb8102740 2024-06-06T04:30:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.718+0000 7fbda77fe700 1 --2- 172.21.15.45:0/931816739 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbda00a2cb0 0x7fbda00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.718+0000 7fbdb5d9b700 1 --2- 172.21.15.45:0/931816739 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbda00a2cb0 0x7fbda00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.718+0000 7fbda77fe700 1 -- 172.21.15.45:0/931816739 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fbda80cf830 con 0x7fbdb8102740 2024-06-06T04:30:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.718+0000 7fbdb5d9b700 1 --2- 172.21.15.45:0/931816739 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbda00a2cb0 0x7fbda00a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbdac005e10 tx=0x7fbdac005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.721+0000 7fbda77fe700 1 -- 172.21.15.45:0/931816739 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbda8014030 con 0x7fbdb8102740 2024-06-06T04:30:01.789 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:01 smithi045 ceph-mon[32299]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:01.789 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:01 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:30:01.789 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:01 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:30:01.790 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:01 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:30:01.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:01 smithi086 ceph-mon[36053]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:01.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:01 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:30:01.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:01 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:30:01.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:01 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:30:01.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:01 smithi154 ceph-mon[36428]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:01.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:01 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:30:01.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:01 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:30:01.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:01 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:30:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.936+0000 7fbdbcc0f700 1 -- 172.21.15.45:0/931816739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fbdb802dca0 con 0x7fbdb8102740 2024-06-06T04:30:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.936+0000 7fbda77fe700 1 -- 172.21.15.45:0/931816739 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbda809c2a0 con 0x7fbdb8102740 2024-06-06T04:30:01.936 INFO:teuthology.orchestra.run.smithi045.stdout:292057776186 2024-06-06T04:30:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.938+0000 7fbdbcc0f700 1 -- 172.21.15.45:0/931816739 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbda00a2cb0 msgr2=0x7fbda00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.938+0000 7fbdbcc0f700 1 --2- 172.21.15.45:0/931816739 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbda00a2cb0 0x7fbda00a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbdac005e10 tx=0x7fbdac005d60 comp rx=0 tx=0).stop 2024-06-06T04:30:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.938+0000 7fbdbcc0f700 1 -- 172.21.15.45:0/931816739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb8102740 msgr2=0x7fbdb8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.938+0000 7fbdbcc0f700 1 --2- 172.21.15.45:0/931816739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb8102740 0x7fbdb8074790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbda800c9b0 tx=0x7fbda800ccc0 comp rx=0 tx=0).stop 2024-06-06T04:30:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.938+0000 7fbdbcc0f700 1 -- 172.21.15.45:0/931816739 shutdown_connections 2024-06-06T04:30:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.938+0000 7fbdbcc0f700 1 --2- 172.21.15.45:0/931816739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdb8107840 0x7fbdb8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.939+0000 7fbdbcc0f700 1 --2- 172.21.15.45:0/931816739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb8102740 0x7fbdb8074790 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.939+0000 7fbdbcc0f700 1 --2- 172.21.15.45:0/931816739 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbda00a2cb0 0x7fbda00a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.939+0000 7fbdbcc0f700 1 --2- 172.21.15.45:0/931816739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdb81030f0 0x7fbdb8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.939+0000 7fbdbcc0f700 1 -- 172.21.15.45:0/931816739 >> 172.21.15.45:0/931816739 conn(0x7fbdb80795f0 msgr2=0x7fbdb8100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.939+0000 7fbdbcc0f700 1 -- 172.21.15.45:0/931816739 shutdown_connections 2024-06-06T04:30:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:01.939+0000 7fbdbcc0f700 1 -- 172.21.15.45:0/931816739 wait complete. 2024-06-06T04:30:02.442 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724615 got 167503724621 for osd.5 2024-06-06T04:30:02.443 DEBUG:teuthology.parallel:result is None 2024-06-06T04:30:02.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:02 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/931816739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-06T04:30:02.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:02 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/931816739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-06T04:30:02.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:02 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/931816739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-06T04:30:02.921 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351016 got 347892351024 for osd.11 2024-06-06T04:30:02.921 DEBUG:teuthology.parallel:result is None 2024-06-06T04:30:03.507 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776182 got 292057776186 for osd.9 2024-06-06T04:30:03.507 DEBUG:teuthology.parallel:result is None 2024-06-06T04:30:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.695+0000 7f765a60a700 1 -- 172.21.15.45:0/4097783759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76541084f0 msgr2=0x7f76541088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.695+0000 7f765a60a700 1 --2- 172.21.15.45:0/4097783759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76541084f0 0x7f76541088d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7644009990 tx=0x7f7644009ca0 comp rx=0 tx=0).stop 2024-06-06T04:30:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.695+0000 7f765a60a700 1 -- 172.21.15.45:0/4097783759 shutdown_connections 2024-06-06T04:30:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.695+0000 7f765a60a700 1 --2- 172.21.15.45:0/4097783759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7654102e80 0x7f765410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.695+0000 7f765a60a700 1 --2- 172.21.15.45:0/4097783759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76541084f0 0x7f76541088d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.695+0000 7f765a60a700 1 --2- 172.21.15.45:0/4097783759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76541024e0 0x7f7654102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.695+0000 7f765a60a700 1 -- 172.21.15.45:0/4097783759 >> 172.21.15.45:0/4097783759 conn(0x7f7654079530 msgr2=0x7f7654079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.695+0000 7f765a60a700 1 -- 172.21.15.45:0/4097783759 shutdown_connections 2024-06-06T04:30:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.695+0000 7f765a60a700 1 -- 172.21.15.45:0/4097783759 wait complete. 2024-06-06T04:30:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.696+0000 7f765a60a700 1 Processor -- start 2024-06-06T04:30:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.696+0000 7f765a60a700 1 -- start start 2024-06-06T04:30:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f765a60a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76541024e0 0x7f765419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f765a60a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7654102e80 0x7f765419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f765a60a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76541084f0 0x7f7654198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f765a60a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7654112600 con 0x7f76541024e0 2024-06-06T04:30:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f765a60a700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7654112480 con 0x7f76541084f0 2024-06-06T04:30:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f765a60a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7654112780 con 0x7f7654102e80 2024-06-06T04:30:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f7653fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76541024e0 0x7f765419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f7653fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76541024e0 0x7f765419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50380/0 (socket says 172.21.15.45:50380) 2024-06-06T04:30:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f7653fff700 1 -- 172.21.15.45:0/414702396 learned_addr learned my addr 172.21.15.45:0/414702396 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f7658ba7700 1 --2- 172.21.15.45:0/414702396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76541084f0 0x7f7654198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f76537fe700 1 --2- 172.21.15.45:0/414702396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7654102e80 0x7f765419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f7653fff700 1 -- 172.21.15.45:0/414702396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7654102e80 msgr2=0x7f765419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f7653fff700 1 --2- 172.21.15.45:0/414702396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7654102e80 0x7f765419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f7653fff700 1 -- 172.21.15.45:0/414702396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76541084f0 msgr2=0x7f7654198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f7653fff700 1 --2- 172.21.15.45:0/414702396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76541084f0 0x7f7654198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f7653fff700 1 -- 172.21.15.45:0/414702396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76541990e0 con 0x7f76541024e0 2024-06-06T04:30:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.697+0000 7f7653fff700 1 --2- 172.21.15.45:0/414702396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76541024e0 0x7f765419e7c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7644000c00 tx=0x7f76440047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.698+0000 7f76517fa700 1 -- 172.21.15.45:0/414702396 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644027070 con 0x7f76541024e0 2024-06-06T04:30:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.698+0000 7f765a60a700 1 -- 172.21.15.45:0/414702396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7654199370 con 0x7f76541024e0 2024-06-06T04:30:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.698+0000 7f76517fa700 1 -- 172.21.15.45:0/414702396 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7644022470 con 0x7f76541024e0 2024-06-06T04:30:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.698+0000 7f76517fa700 1 -- 172.21.15.45:0/414702396 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644017650 con 0x7f76541024e0 2024-06-06T04:30:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.698+0000 7f765a60a700 1 -- 172.21.15.45:0/414702396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76541a5690 con 0x7f76541024e0 2024-06-06T04:30:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.699+0000 7f765a60a700 1 -- 172.21.15.45:0/414702396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7654069180 con 0x7f76541024e0 2024-06-06T04:30:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.700+0000 7f76517fa700 1 -- 172.21.15.45:0/414702396 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f764402a070 con 0x7f76541024e0 2024-06-06T04:30:03.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.701+0000 7f76517fa700 1 --2- 172.21.15.45:0/414702396 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f763c0a2cb0 0x7f763c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:03.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.701+0000 7f76537fe700 1 --2- 172.21.15.45:0/414702396 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f763c0a2cb0 0x7f763c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:03.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.701+0000 7f76517fa700 1 -- 172.21.15.45:0/414702396 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f76440d0770 con 0x7f76541024e0 2024-06-06T04:30:03.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.701+0000 7f76537fe700 1 --2- 172.21.15.45:0/414702396 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f763c0a2cb0 0x7f763c0a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7654079ba0 tx=0x7f7648009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.705+0000 7f76517fa700 1 -- 172.21.15.45:0/414702396 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7644013070 con 0x7f76541024e0 2024-06-06T04:30:03.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:03 smithi045 ceph-mon[32299]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:30:03.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:03 smithi086 ceph-mon[36053]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:30:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.916+0000 7f765a60a700 1 -- 172.21.15.45:0/414702396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f765402dca0 con 0x7f76541024e0 2024-06-06T04:30:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.917+0000 7f76517fa700 1 -- 172.21.15.45:0/414702396 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f764409d240 con 0x7f76541024e0 2024-06-06T04:30:03.917 INFO:teuthology.orchestra.run.smithi045.stdout:34359738472 2024-06-06T04:30:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.919+0000 7f765a60a700 1 -- 172.21.15.45:0/414702396 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f763c0a2cb0 msgr2=0x7f763c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.919+0000 7f765a60a700 1 --2- 172.21.15.45:0/414702396 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f763c0a2cb0 0x7f763c0a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7654079ba0 tx=0x7f7648009260 comp rx=0 tx=0).stop 2024-06-06T04:30:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.919+0000 7f765a60a700 1 -- 172.21.15.45:0/414702396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76541024e0 msgr2=0x7f765419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.919+0000 7f765a60a700 1 --2- 172.21.15.45:0/414702396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76541024e0 0x7f765419e7c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7644000c00 tx=0x7f76440047b0 comp rx=0 tx=0).stop 2024-06-06T04:30:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.919+0000 7f765a60a700 1 -- 172.21.15.45:0/414702396 shutdown_connections 2024-06-06T04:30:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.919+0000 7f765a60a700 1 --2- 172.21.15.45:0/414702396 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f763c0a2cb0 0x7f763c0a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.919+0000 7f765a60a700 1 --2- 172.21.15.45:0/414702396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76541084f0 0x7f7654198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.919+0000 7f765a60a700 1 --2- 172.21.15.45:0/414702396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76541024e0 0x7f765419e7c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.919+0000 7f765a60a700 1 --2- 172.21.15.45:0/414702396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7654102e80 0x7f765419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.919+0000 7f765a60a700 1 -- 172.21.15.45:0/414702396 >> 172.21.15.45:0/414702396 conn(0x7f7654079530 msgr2=0x7f76541005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.919+0000 7f765a60a700 1 -- 172.21.15.45:0/414702396 shutdown_connections 2024-06-06T04:30:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:03.919+0000 7f765a60a700 1 -- 172.21.15.45:0/414702396 wait complete. 2024-06-06T04:30:03.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:03 smithi154 ceph-mon[36428]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:30:04.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:04 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/414702396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-06T04:30:04.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:04 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/414702396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-06T04:30:04.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:04 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/414702396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-06T04:30:05.355 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738466 got 34359738472 for osd.0 2024-06-06T04:30:05.355 DEBUG:teuthology.parallel:result is None 2024-06-06T04:30:05.631 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:05 smithi045 ceph-mon[32299]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:05.632 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:05 smithi045 ceph-mon[32299]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-06-06T04:30:05.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:05 smithi086 ceph-mon[36053]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:05.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:05 smithi086 ceph-mon[36053]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-06-06T04:30:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.881+0000 7f670e2f3700 1 -- 172.21.15.45:0/3570662081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f670806a170 msgr2=0x7f670806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.881+0000 7f670e2f3700 1 --2- 172.21.15.45:0/3570662081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f670806a170 0x7f670806a5f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009990 tx=0x7f66fc009ca0 comp rx=0 tx=0).stop 2024-06-06T04:30:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.881+0000 7f670e2f3700 1 -- 172.21.15.45:0/3570662081 shutdown_connections 2024-06-06T04:30:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.881+0000 7f670e2f3700 1 --2- 172.21.15.45:0/3570662081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f670806ab30 0x7f670810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.881+0000 7f670e2f3700 1 --2- 172.21.15.45:0/3570662081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6708108500 0x7f67081088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.881+0000 7f670e2f3700 1 --2- 172.21.15.45:0/3570662081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f670806a170 0x7f670806a5f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.881+0000 7f670e2f3700 1 -- 172.21.15.45:0/3570662081 >> 172.21.15.45:0/3570662081 conn(0x7f6708078340 msgr2=0x7f6708078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.881+0000 7f670e2f3700 1 -- 172.21.15.45:0/3570662081 shutdown_connections 2024-06-06T04:30:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.881+0000 7f670e2f3700 1 -- 172.21.15.45:0/3570662081 wait complete. 2024-06-06T04:30:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.882+0000 7f670e2f3700 1 Processor -- start 2024-06-06T04:30:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.882+0000 7f670e2f3700 1 -- start start 2024-06-06T04:30:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.883+0000 7f670e2f3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f670806a170 0x7f670819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.883+0000 7f670e2f3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f670806ab30 0x7f670819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.883+0000 7f670e2f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6708108500 0x7f6708198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.883+0000 7f670e2f3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6708112630 con 0x7f6708108500 2024-06-06T04:30:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.883+0000 7f670e2f3700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67081124b0 con 0x7f670806ab30 2024-06-06T04:30:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.883+0000 7f670e2f3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f67081127b0 con 0x7f670806a170 2024-06-06T04:30:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.883+0000 7f6707fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f670806a170 0x7f670819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.883+0000 7f6707fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f670806a170 0x7f670819e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:47844/0 (socket says 172.21.15.45:47844) 2024-06-06T04:30:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.883+0000 7f6707fff700 1 -- 172.21.15.45:0/1829514956 learned_addr learned my addr 172.21.15.45:0/1829514956 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.883+0000 7f6707fff700 1 -- 172.21.15.45:0/1829514956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f670806ab30 msgr2=0x7f670819ecc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T04:30:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.883+0000 7f6707fff700 1 --2- 172.21.15.45:0/1829514956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f670806ab30 0x7f670819ecc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.883+0000 7f6707fff700 1 -- 172.21.15.45:0/1829514956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6708108500 msgr2=0x7f6708198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.883+0000 7f6707fff700 1 --2- 172.21.15.45:0/1829514956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6708108500 0x7f6708198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.883+0000 7f6707fff700 1 -- 172.21.15.45:0/1829514956 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67081990f0 con 0x7f670806a170 2024-06-06T04:30:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.884+0000 7f6707fff700 1 --2- 172.21.15.45:0/1829514956 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f670806a170 0x7f670819e780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f66f800d950 tx=0x7f66f800dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.884+0000 7f67057fa700 1 -- 172.21.15.45:0/1829514956 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8016070 con 0x7f670806a170 2024-06-06T04:30:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.884+0000 7f67057fa700 1 -- 172.21.15.45:0/1829514956 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66f8010460 con 0x7f670806a170 2024-06-06T04:30:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.884+0000 7f670e2f3700 1 -- 172.21.15.45:0/1829514956 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67081993e0 con 0x7f670806a170 2024-06-06T04:30:05.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.884+0000 7f67057fa700 1 -- 172.21.15.45:0/1829514956 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f800f620 con 0x7f670806a170 2024-06-06T04:30:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.884+0000 7f670e2f3700 1 -- 172.21.15.45:0/1829514956 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67081a5650 con 0x7f670806a170 2024-06-06T04:30:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.885+0000 7f670e2f3700 1 -- 172.21.15.45:0/1829514956 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6708069180 con 0x7f670806a170 2024-06-06T04:30:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.886+0000 7f67057fa700 1 -- 172.21.15.45:0/1829514956 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f66f8019070 con 0x7f670806a170 2024-06-06T04:30:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.887+0000 7f67057fa700 1 --2- 172.21.15.45:0/1829514956 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f66f00a2c20 0x7f66f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.887+0000 7f67077fe700 1 --2- 172.21.15.45:0/1829514956 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f66f00a2c20 0x7f66f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.887+0000 7f67057fa700 1 -- 172.21.15.45:0/1829514956 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f66f80cf060 con 0x7f670806a170 2024-06-06T04:30:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.887+0000 7f67077fe700 1 --2- 172.21.15.45:0/1829514956 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f66f00a2c20 0x7f66f00a50e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f66fc005b40 tx=0x7f66fc005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:05.891+0000 7f67057fa700 1 -- 172.21.15.45:0/1829514956 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f66f809bba0 con 0x7f670806a170 2024-06-06T04:30:05.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:05 smithi154 ceph-mon[36428]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:05.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:05 smithi154 ceph-mon[36428]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-06-06T04:30:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:06.096+0000 7f670e2f3700 1 -- 172.21.15.45:0/1829514956 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f670802ddc0 con 0x7f670806a170 2024-06-06T04:30:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:06.096+0000 7f67057fa700 1 -- 172.21.15.45:0/1829514956 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f66f809b4a0 con 0x7f670806a170 2024-06-06T04:30:06.097 INFO:teuthology.orchestra.run.smithi045.stdout:201863462987 2024-06-06T04:30:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:06.098+0000 7f670e2f3700 1 -- 172.21.15.45:0/1829514956 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f66f00a2c20 msgr2=0x7f66f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:06.098+0000 7f670e2f3700 1 --2- 172.21.15.45:0/1829514956 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f66f00a2c20 0x7f66f00a50e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f66fc005b40 tx=0x7f66fc005a90 comp rx=0 tx=0).stop 2024-06-06T04:30:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:06.098+0000 7f670e2f3700 1 -- 172.21.15.45:0/1829514956 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f670806a170 msgr2=0x7f670819e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:06.098+0000 7f670e2f3700 1 --2- 172.21.15.45:0/1829514956 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f670806a170 0x7f670819e780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f66f800d950 tx=0x7f66f800dd10 comp rx=0 tx=0).stop 2024-06-06T04:30:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:06.099+0000 7f670e2f3700 1 -- 172.21.15.45:0/1829514956 shutdown_connections 2024-06-06T04:30:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:06.099+0000 7f670e2f3700 1 --2- 172.21.15.45:0/1829514956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6708108500 0x7f6708198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:06.099+0000 7f670e2f3700 1 --2- 172.21.15.45:0/1829514956 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f670806a170 0x7f670819e780 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:06.099+0000 7f670e2f3700 1 --2- 172.21.15.45:0/1829514956 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f66f00a2c20 0x7f66f00a50e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:06.099+0000 7f670e2f3700 1 --2- 172.21.15.45:0/1829514956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f670806ab30 0x7f670819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:06.099+0000 7f670e2f3700 1 -- 172.21.15.45:0/1829514956 >> 172.21.15.45:0/1829514956 conn(0x7f6708078340 msgr2=0x7f6708100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:06.099+0000 7f670e2f3700 1 -- 172.21.15.45:0/1829514956 shutdown_connections 2024-06-06T04:30:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:06.099+0000 7f670e2f3700 1 -- 172.21.15.45:0/1829514956 wait complete. 2024-06-06T04:30:06.689 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:06 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1829514956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-06T04:30:06.715 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462981 got 201863462987 for osd.6 2024-06-06T04:30:06.715 DEBUG:teuthology.parallel:result is None 2024-06-06T04:30:06.715 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-06T04:30:06.715 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T04:30:06.832 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:06 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1829514956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-06T04:30:06.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:06 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1829514956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-06T04:30:06.966 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:07.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:07 smithi045 ceph-mon[32299]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T04:30:07.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:07 smithi086 ceph-mon[36053]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T04:30:07.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:07 smithi154 ceph-mon[36428]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T04:30:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.585+0000 7f3b23b56700 1 -- 172.21.15.45:0/107722435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b1c1066d0 msgr2=0x7f3b1c10e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.585+0000 7f3b23b56700 1 --2- 172.21.15.45:0/107722435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b1c1066d0 0x7f3b1c10e2a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400b0d0 tx=0x7f3b1400b3e0 comp rx=0 tx=0).stop 2024-06-06T04:30:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.585+0000 7f3b23b56700 1 -- 172.21.15.45:0/107722435 shutdown_connections 2024-06-06T04:30:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.585+0000 7f3b23b56700 1 --2- 172.21.15.45:0/107722435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b1c1066d0 0x7f3b1c10e2a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.585+0000 7f3b23b56700 1 --2- 172.21.15.45:0/107722435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b1c101680 0x7f3b1c101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.585+0000 7f3b23b56700 1 --2- 172.21.15.45:0/107722435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b1c102030 0x7f3b1c105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.585+0000 7f3b23b56700 1 -- 172.21.15.45:0/107722435 >> 172.21.15.45:0/107722435 conn(0x7f3b1c0773e0 msgr2=0x7f3b1c0777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.585+0000 7f3b23b56700 1 -- 172.21.15.45:0/107722435 shutdown_connections 2024-06-06T04:30:08.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.585+0000 7f3b23b56700 1 -- 172.21.15.45:0/107722435 wait complete. 2024-06-06T04:30:08.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b23b56700 1 Processor -- start 2024-06-06T04:30:08.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b23b56700 1 -- start start 2024-06-06T04:30:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b23b56700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b1c101680 0x7f3b1c19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b23b56700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b1c102030 0x7f3b1c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b23b56700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b1c1066d0 0x7f3b1c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b23b56700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c111590 con 0x7f3b1c101680 2024-06-06T04:30:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b23b56700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c111410 con 0x7f3b1c1066d0 2024-06-06T04:30:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b23b56700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c111710 con 0x7f3b1c102030 2024-06-06T04:30:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b218f2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b1c101680 0x7f3b1c19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b218f2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b1c101680 0x7f3b1c19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54154/0 (socket says 172.21.15.45:54154) 2024-06-06T04:30:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b210f1700 1 --2- 172.21.15.45:0/3639121241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b1c102030 0x7f3b1c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b218f2700 1 -- 172.21.15.45:0/3639121241 learned_addr learned my addr 172.21.15.45:0/3639121241 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b220f3700 1 --2- 172.21.15.45:0/3639121241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b1c1066d0 0x7f3b1c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b218f2700 1 -- 172.21.15.45:0/3639121241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b1c102030 msgr2=0x7f3b1c19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b218f2700 1 --2- 172.21.15.45:0/3639121241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b1c102030 0x7f3b1c19b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b218f2700 1 -- 172.21.15.45:0/3639121241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b1c1066d0 msgr2=0x7f3b1c19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b218f2700 1 --2- 172.21.15.45:0/3639121241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b1c1066d0 0x7f3b1c19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b218f2700 1 -- 172.21.15.45:0/3639121241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1c19fd40 con 0x7f3b1c101680 2024-06-06T04:30:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b220f3700 1 --2- 172.21.15.45:0/3639121241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b1c1066d0 0x7f3b1c19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:30:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.586+0000 7f3b210f1700 1 --2- 172.21.15.45:0/3639121241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b1c102030 0x7f3b1c19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:30:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.587+0000 7f3b218f2700 1 --2- 172.21.15.45:0/3639121241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b1c101680 0x7f3b1c19ad30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800e9c0 tx=0x7f3b1800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.587+0000 7f3b12ffd700 1 -- 172.21.15.45:0/3639121241 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1800cbd0 con 0x7f3b1c101680 2024-06-06T04:30:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.587+0000 7f3b12ffd700 1 -- 172.21.15.45:0/3639121241 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b1800cd70 con 0x7f3b1c101680 2024-06-06T04:30:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.587+0000 7f3b12ffd700 1 -- 172.21.15.45:0/3639121241 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b180056c0 con 0x7f3b1c101680 2024-06-06T04:30:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.587+0000 7f3b23b56700 1 -- 172.21.15.45:0/3639121241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1c1a0030 con 0x7f3b1c101680 2024-06-06T04:30:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.587+0000 7f3b23b56700 1 -- 172.21.15.45:0/3639121241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b1c1a78b0 con 0x7f3b1c101680 2024-06-06T04:30:08.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.589+0000 7f3b23b56700 1 -- 172.21.15.45:0/3639121241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b1c102ad0 con 0x7f3b1c101680 2024-06-06T04:30:08.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.590+0000 7f3b12ffd700 1 -- 172.21.15.45:0/3639121241 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3b18004710 con 0x7f3b1c101680 2024-06-06T04:30:08.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.590+0000 7f3b12ffd700 1 --2- 172.21.15.45:0/3639121241 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b080a2b80 0x7f3b080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:08.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.591+0000 7f3b210f1700 1 --2- 172.21.15.45:0/3639121241 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b080a2b80 0x7f3b080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:08.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.591+0000 7f3b12ffd700 1 -- 172.21.15.45:0/3639121241 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f3b180cf970 con 0x7f3b1c101680 2024-06-06T04:30:08.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.591+0000 7f3b210f1700 1 --2- 172.21.15.45:0/3639121241 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b080a2b80 0x7f3b080a5040 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c107e30 tx=0x7f3b0c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:08.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.594+0000 7f3b12ffd700 1 -- 172.21.15.45:0/3639121241 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3b1809c5a0 con 0x7f3b1c101680 2024-06-06T04:30:08.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.797+0000 7f3b23b56700 1 -- 172.21.15.45:0/3639121241 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b1c19c090 con 0x7f3b080a2b80 2024-06-06T04:30:08.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.806+0000 7f3b12ffd700 1 -- 172.21.15.45:0/3639121241 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340989 (secure 0 0 0) 0x7f3b1c19c090 con 0x7f3b080a2b80 2024-06-06T04:30:08.815 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:30:08.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.810+0000 7f3b23b56700 1 -- 172.21.15.45:0/3639121241 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b080a2b80 msgr2=0x7f3b080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:08.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.810+0000 7f3b23b56700 1 --2- 172.21.15.45:0/3639121241 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b080a2b80 0x7f3b080a5040 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c107e30 tx=0x7f3b0c00b410 comp rx=0 tx=0).stop 2024-06-06T04:30:08.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.810+0000 7f3b23b56700 1 -- 172.21.15.45:0/3639121241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b1c101680 msgr2=0x7f3b1c19ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:08.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.810+0000 7f3b23b56700 1 --2- 172.21.15.45:0/3639121241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b1c101680 0x7f3b1c19ad30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800e9c0 tx=0x7f3b1800ecd0 comp rx=0 tx=0).stop 2024-06-06T04:30:08.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.810+0000 7f3b23b56700 1 -- 172.21.15.45:0/3639121241 shutdown_connections 2024-06-06T04:30:08.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.810+0000 7f3b23b56700 1 --2- 172.21.15.45:0/3639121241 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b080a2b80 0x7f3b080a5040 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:08.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.810+0000 7f3b23b56700 1 --2- 172.21.15.45:0/3639121241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b1c1066d0 0x7f3b1c19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:08.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.810+0000 7f3b23b56700 1 --2- 172.21.15.45:0/3639121241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b1c101680 0x7f3b1c19ad30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:08.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.810+0000 7f3b23b56700 1 --2- 172.21.15.45:0/3639121241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b1c102030 0x7f3b1c19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:08.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.810+0000 7f3b23b56700 1 -- 172.21.15.45:0/3639121241 >> 172.21.15.45:0/3639121241 conn(0x7f3b1c0773e0 msgr2=0x7f3b1c10d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:08.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.810+0000 7f3b23b56700 1 -- 172.21.15.45:0/3639121241 shutdown_connections 2024-06-06T04:30:08.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:08.810+0000 7f3b23b56700 1 -- 172.21.15.45:0/3639121241 wait complete. 2024-06-06T04:30:08.821 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-06-06T04:30:09.540 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-06-06T04:30:08.403169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659608,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":736,"num_read_kb":486,"num_write":358,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":622,"ondisk_log_size":622,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375068,"kb_used_data":13412,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566756,"statfs":{"total":1151940427776,"available":1151556358144,"internally_reserved":0,"allocated":13733888,"data_stored":8829272,"data_compressed":497595,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":42,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003328"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.386822+0000","last_change":"2024-06-06T04:27:22.574900+0000","last_active":"2024-06-06T04:30:04.386822+0000","last_peered":"2024-06-06T04:30:04.386822+0000","last_clean":"2024-06-06T04:30:04.386822+0000","last_became_active":"2024-06-06T04:27:22.574389+0000","last_became_peered":"2024-06-06T04:27:22.574389+0000","last_unstale":"2024-06-06T04:30:04.386822+0000","last_undegraded":"2024-06-06T04:30:04.386822+0000","last_fullsized":"2024-06-06T04:30:04.386822+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:39.246463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.382712+0000","last_change":"2024-06-06T04:27:22.565543+0000","last_active":"2024-06-06T04:30:04.382712+0000","last_peered":"2024-06-06T04:30:04.382712+0000","last_clean":"2024-06-06T04:30:04.382712+0000","last_became_active":"2024-06-06T04:27:22.563833+0000","last_became_peered":"2024-06-06T04:27:22.563833+0000","last_unstale":"2024-06-06T04:30:04.382712+0000","last_undegraded":"2024-06-06T04:30:04.382712+0000","last_fullsized":"2024-06-06T04:30:04.382712+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:26:17.810782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388761+0000","last_change":"2024-06-06T04:27:22.570107+0000","last_active":"2024-06-06T04:30:04.388761+0000","last_peered":"2024-06-06T04:30:04.388761+0000","last_clean":"2024-06-06T04:30:04.388761+0000","last_became_active":"2024-06-06T04:27:22.569788+0000","last_became_peered":"2024-06-06T04:27:22.569788+0000","last_unstale":"2024-06-06T04:30:04.388761+0000","last_undegraded":"2024-06-06T04:30:04.388761+0000","last_fullsized":"2024-06-06T04:30:04.388761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:48:20.454782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385114+0000","last_change":"2024-06-06T04:27:22.559431+0000","last_active":"2024-06-06T04:30:04.385114+0000","last_peered":"2024-06-06T04:30:04.385114+0000","last_clean":"2024-06-06T04:30:04.385114+0000","last_became_active":"2024-06-06T04:27:22.558961+0000","last_became_peered":"2024-06-06T04:27:22.558961+0000","last_unstale":"2024-06-06T04:30:04.385114+0000","last_undegraded":"2024-06-06T04:30:04.385114+0000","last_fullsized":"2024-06-06T04:30:04.385114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:59.445283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.359959+0000","last_change":"2024-06-06T04:27:22.561598+0000","last_active":"2024-06-06T04:30:04.359959+0000","last_peered":"2024-06-06T04:30:04.359959+0000","last_clean":"2024-06-06T04:30:04.359959+0000","last_became_active":"2024-06-06T04:27:22.561198+0000","last_became_peered":"2024-06-06T04:27:22.561198+0000","last_unstale":"2024-06-06T04:30:04.359959+0000","last_undegraded":"2024-06-06T04:30:04.359959+0000","last_fullsized":"2024-06-06T04:30:04.359959+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:20:28.314941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380904+0000","last_change":"2024-06-06T04:27:22.569395+0000","last_active":"2024-06-06T04:30:04.380904+0000","last_peered":"2024-06-06T04:30:04.380904+0000","last_clean":"2024-06-06T04:30:04.380904+0000","last_became_active":"2024-06-06T04:27:22.569054+0000","last_became_peered":"2024-06-06T04:27:22.569054+0000","last_unstale":"2024-06-06T04:30:04.380904+0000","last_undegraded":"2024-06-06T04:30:04.380904+0000","last_fullsized":"2024-06-06T04:30:04.380904+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:03:56.630677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905289+0000","last_change":"2024-06-06T04:27:22.577121+0000","last_active":"2024-06-06T04:29:05.905289+0000","last_peered":"2024-06-06T04:29:05.905289+0000","last_clean":"2024-06-06T04:29:05.905289+0000","last_became_active":"2024-06-06T04:27:22.575574+0000","last_became_peered":"2024-06-06T04:27:22.575574+0000","last_unstale":"2024-06-06T04:29:05.905289+0000","last_undegraded":"2024-06-06T04:29:05.905289+0000","last_fullsized":"2024-06-06T04:29:05.905289+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:05:16.799335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.383226+0000","last_change":"2024-06-06T04:27:22.570051+0000","last_active":"2024-06-06T04:30:04.383226+0000","last_peered":"2024-06-06T04:30:04.383226+0000","last_clean":"2024-06-06T04:30:04.383226+0000","last_became_active":"2024-06-06T04:27:22.569640+0000","last_became_peered":"2024-06-06T04:27:22.569640+0000","last_unstale":"2024-06-06T04:30:04.383226+0000","last_undegraded":"2024-06-06T04:30:04.383226+0000","last_fullsized":"2024-06-06T04:30:04.383226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:48:00.744167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.907792+0000","last_change":"2024-06-06T04:27:22.574900+0000","last_active":"2024-06-06T04:29:06.907792+0000","last_peered":"2024-06-06T04:29:06.907792+0000","last_clean":"2024-06-06T04:29:06.907792+0000","last_became_active":"2024-06-06T04:27:22.574570+0000","last_became_peered":"2024-06-06T04:27:22.574570+0000","last_unstale":"2024-06-06T04:29:06.907792+0000","last_undegraded":"2024-06-06T04:29:06.907792+0000","last_fullsized":"2024-06-06T04:29:06.907792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:54.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380728+0000","last_change":"2024-06-06T04:27:22.570581+0000","last_active":"2024-06-06T04:30:04.380728+0000","last_peered":"2024-06-06T04:30:04.380728+0000","last_clean":"2024-06-06T04:30:04.380728+0000","last_became_active":"2024-06-06T04:27:22.570243+0000","last_became_peered":"2024-06-06T04:27:22.570243+0000","last_unstale":"2024-06-06T04:30:04.380728+0000","last_undegraded":"2024-06-06T04:30:04.380728+0000","last_fullsized":"2024-06-06T04:30:04.380728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:27.087527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905944+0000","last_change":"2024-06-06T04:27:22.571162+0000","last_active":"2024-06-06T04:29:05.905944+0000","last_peered":"2024-06-06T04:29:05.905944+0000","last_clean":"2024-06-06T04:29:05.905944+0000","last_became_active":"2024-06-06T04:27:22.570843+0000","last_became_peered":"2024-06-06T04:27:22.570843+0000","last_unstale":"2024-06-06T04:29:05.905944+0000","last_undegraded":"2024-06-06T04:29:05.905944+0000","last_fullsized":"2024-06-06T04:29:05.905944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:43:05.590240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.384953+0000","last_change":"2024-06-06T04:27:22.567420+0000","last_active":"2024-06-06T04:30:04.384953+0000","last_peered":"2024-06-06T04:30:04.384953+0000","last_clean":"2024-06-06T04:30:04.384953+0000","last_became_active":"2024-06-06T04:27:22.566750+0000","last_became_peered":"2024-06-06T04:27:22.566750+0000","last_unstale":"2024-06-06T04:30:04.384953+0000","last_undegraded":"2024-06-06T04:30:04.384953+0000","last_fullsized":"2024-06-06T04:30:04.384953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:21.669965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904374+0000","last_change":"2024-06-06T04:27:22.566831+0000","last_active":"2024-06-06T04:29:05.904374+0000","last_peered":"2024-06-06T04:29:05.904374+0000","last_clean":"2024-06-06T04:29:05.904374+0000","last_became_active":"2024-06-06T04:27:22.566255+0000","last_became_peered":"2024-06-06T04:27:22.566255+0000","last_unstale":"2024-06-06T04:29:05.904374+0000","last_undegraded":"2024-06-06T04:29:05.904374+0000","last_fullsized":"2024-06-06T04:29:05.904374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:40:22.486674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.384669+0000","last_change":"2024-06-06T04:27:22.564596+0000","last_active":"2024-06-06T04:30:04.384669+0000","last_peered":"2024-06-06T04:30:04.384669+0000","last_clean":"2024-06-06T04:30:04.384669+0000","last_became_active":"2024-06-06T04:27:22.563694+0000","last_became_peered":"2024-06-06T04:27:22.563694+0000","last_unstale":"2024-06-06T04:30:04.384669+0000","last_undegraded":"2024-06-06T04:30:04.384669+0000","last_fullsized":"2024-06-06T04:30:04.384669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:18.568266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385298+0000","last_change":"2024-06-06T04:27:22.559395+0000","last_active":"2024-06-06T04:30:04.385298+0000","last_peered":"2024-06-06T04:30:04.385298+0000","last_clean":"2024-06-06T04:30:04.385298+0000","last_became_active":"2024-06-06T04:27:22.559030+0000","last_became_peered":"2024-06-06T04:27:22.559030+0000","last_unstale":"2024-06-06T04:30:04.385298+0000","last_undegraded":"2024-06-06T04:30:04.385298+0000","last_fullsized":"2024-06-06T04:30:04.385298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:08.562184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.907434+0000","last_change":"2024-06-06T04:27:22.569657+0000","last_active":"2024-06-06T04:29:06.907434+0000","last_peered":"2024-06-06T04:29:06.907434+0000","last_clean":"2024-06-06T04:29:06.907434+0000","last_became_active":"2024-06-06T04:27:22.569359+0000","last_became_peered":"2024-06-06T04:27:22.569359+0000","last_unstale":"2024-06-06T04:29:06.907434+0000","last_undegraded":"2024-06-06T04:29:06.907434+0000","last_fullsized":"2024-06-06T04:29:06.907434+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:42:20.197715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394347+0000","last_change":"2024-06-06T04:27:22.574661+0000","last_active":"2024-06-06T04:30:04.394347+0000","last_peered":"2024-06-06T04:30:04.394347+0000","last_clean":"2024-06-06T04:30:04.394347+0000","last_became_active":"2024-06-06T04:27:22.574310+0000","last_became_peered":"2024-06-06T04:27:22.574310+0000","last_unstale":"2024-06-06T04:30:04.394347+0000","last_undegraded":"2024-06-06T04:30:04.394347+0000","last_fullsized":"2024-06-06T04:30:04.394347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:45:18.772270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388153+0000","last_change":"2024-06-06T04:27:22.565575+0000","last_active":"2024-06-06T04:30:04.388153+0000","last_peered":"2024-06-06T04:30:04.388153+0000","last_clean":"2024-06-06T04:30:04.388153+0000","last_became_active":"2024-06-06T04:27:22.565256+0000","last_became_peered":"2024-06-06T04:27:22.565256+0000","last_unstale":"2024-06-06T04:30:04.388153+0000","last_undegraded":"2024-06-06T04:30:04.388153+0000","last_fullsized":"2024-06-06T04:30:04.388153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:49.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.365082+0000","last_change":"2024-06-06T04:27:22.559606+0000","last_active":"2024-06-06T04:30:04.365082+0000","last_peered":"2024-06-06T04:30:04.365082+0000","last_clean":"2024-06-06T04:30:04.365082+0000","last_became_active":"2024-06-06T04:27:22.557881+0000","last_became_peered":"2024-06-06T04:27:22.557881+0000","last_unstale":"2024-06-06T04:30:04.365082+0000","last_undegraded":"2024-06-06T04:30:04.365082+0000","last_fullsized":"2024-06-06T04:30:04.365082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:31:06.269785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905476+0000","last_change":"2024-06-06T04:27:32.626612+0000","last_active":"2024-06-06T04:29:05.905476+0000","last_peered":"2024-06-06T04:29:05.905476+0000","last_clean":"2024-06-06T04:29:05.905476+0000","last_became_active":"2024-06-06T04:27:32.626113+0000","last_became_peered":"2024-06-06T04:27:32.626113+0000","last_unstale":"2024-06-06T04:29:05.905476+0000","last_undegraded":"2024-06-06T04:29:05.905476+0000","last_fullsized":"2024-06-06T04:29:05.905476+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.318738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908330+0000","last_change":"2024-06-06T04:27:19.302744+0000","last_active":"2024-06-06T04:29:05.908330+0000","last_peered":"2024-06-06T04:29:05.908330+0000","last_clean":"2024-06-06T04:29:05.908330+0000","last_became_active":"2024-06-06T04:27:19.302389+0000","last_became_peered":"2024-06-06T04:27:19.302389+0000","last_unstale":"2024-06-06T04:29:05.908330+0000","last_undegraded":"2024-06-06T04:29:05.908330+0000","last_fullsized":"2024-06-06T04:29:05.908330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:49.872618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380678+0000","last_change":"2024-06-06T04:27:22.570220+0000","last_active":"2024-06-06T04:30:04.380678+0000","last_peered":"2024-06-06T04:30:04.380678+0000","last_clean":"2024-06-06T04:30:04.380678+0000","last_became_active":"2024-06-06T04:27:22.569835+0000","last_became_peered":"2024-06-06T04:27:22.569835+0000","last_unstale":"2024-06-06T04:30:04.380678+0000","last_undegraded":"2024-06-06T04:30:04.380678+0000","last_fullsized":"2024-06-06T04:30:04.380678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:54:57.372561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401940+0000","last_change":"2024-06-06T04:27:32.622407+0000","last_active":"2024-06-06T04:29:06.401940+0000","last_peered":"2024-06-06T04:29:06.401940+0000","last_clean":"2024-06-06T04:29:06.401940+0000","last_became_active":"2024-06-06T04:27:32.621868+0000","last_became_peered":"2024-06-06T04:27:32.621868+0000","last_unstale":"2024-06-06T04:29:06.401940+0000","last_undegraded":"2024-06-06T04:29:06.401940+0000","last_fullsized":"2024-06-06T04:29:06.401940+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:50:05.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560827+0000","last_change":"2024-06-06T04:27:19.516235+0000","last_active":"2024-06-06T04:29:06.560827+0000","last_peered":"2024-06-06T04:29:06.560827+0000","last_clean":"2024-06-06T04:29:06.560827+0000","last_became_active":"2024-06-06T04:27:19.515849+0000","last_became_peered":"2024-06-06T04:27:19.515849+0000","last_unstale":"2024-06-06T04:29:06.560827+0000","last_undegraded":"2024-06-06T04:29:06.560827+0000","last_fullsized":"2024-06-06T04:29:06.560827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:52:42.722103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388234+0000","last_change":"2024-06-06T04:27:22.570797+0000","last_active":"2024-06-06T04:30:04.388234+0000","last_peered":"2024-06-06T04:30:04.388234+0000","last_clean":"2024-06-06T04:30:04.388234+0000","last_became_active":"2024-06-06T04:27:22.570305+0000","last_became_peered":"2024-06-06T04:27:22.570305+0000","last_unstale":"2024-06-06T04:30:04.388234+0000","last_undegraded":"2024-06-06T04:30:04.388234+0000","last_fullsized":"2024-06-06T04:30:04.388234+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:04:14.545306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401666+0000","last_change":"2024-06-06T04:27:32.623241+0000","last_active":"2024-06-06T04:29:06.401666+0000","last_peered":"2024-06-06T04:29:06.401666+0000","last_clean":"2024-06-06T04:29:06.401666+0000","last_became_active":"2024-06-06T04:27:32.622751+0000","last_became_peered":"2024-06-06T04:27:32.622751+0000","last_unstale":"2024-06-06T04:29:06.401666+0000","last_undegraded":"2024-06-06T04:29:06.401666+0000","last_fullsized":"2024-06-06T04:29:06.401666+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:24:53.446329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.909054+0000","last_change":"2024-06-06T04:28:05.733350+0000","last_active":"2024-06-06T04:29:05.909054+0000","last_peered":"2024-06-06T04:29:05.909054+0000","last_clean":"2024-06-06T04:29:05.909054+0000","last_became_active":"2024-06-06T04:28:05.726556+0000","last_became_peered":"2024-06-06T04:28:05.726556+0000","last_unstale":"2024-06-06T04:29:05.909054+0000","last_undegraded":"2024-06-06T04:29:05.909054+0000","last_fullsized":"2024-06-06T04:29:05.909054+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:26.636625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.907449+0000","last_change":"2024-06-06T04:27:22.571731+0000","last_active":"2024-06-06T04:29:06.907449+0000","last_peered":"2024-06-06T04:29:06.907449+0000","last_clean":"2024-06-06T04:29:06.907449+0000","last_became_active":"2024-06-06T04:27:22.571398+0000","last_became_peered":"2024-06-06T04:27:22.571398+0000","last_unstale":"2024-06-06T04:29:06.907449+0000","last_undegraded":"2024-06-06T04:29:06.907449+0000","last_fullsized":"2024-06-06T04:29:06.907449+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:22:45.144427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401661+0000","last_change":"2024-06-06T04:27:32.618465+0000","last_active":"2024-06-06T04:29:06.401661+0000","last_peered":"2024-06-06T04:29:06.401661+0000","last_clean":"2024-06-06T04:29:06.401661+0000","last_became_active":"2024-06-06T04:27:32.618066+0000","last_became_peered":"2024-06-06T04:27:32.618066+0000","last_unstale":"2024-06-06T04:29:06.401661+0000","last_undegraded":"2024-06-06T04:29:06.401661+0000","last_fullsized":"2024-06-06T04:29:06.401661+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:54:23.988687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908662+0000","last_change":"2024-06-06T04:28:05.730381+0000","last_active":"2024-06-06T04:29:05.908662+0000","last_peered":"2024-06-06T04:29:05.908662+0000","last_clean":"2024-06-06T04:29:05.908662+0000","last_became_active":"2024-06-06T04:28:05.726552+0000","last_became_peered":"2024-06-06T04:28:05.726552+0000","last_unstale":"2024-06-06T04:29:05.908662+0000","last_undegraded":"2024-06-06T04:29:05.908662+0000","last_fullsized":"2024-06-06T04:29:05.908662+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:52.903042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394789+0000","last_change":"2024-06-06T04:27:22.573769+0000","last_active":"2024-06-06T04:30:04.394789+0000","last_peered":"2024-06-06T04:30:04.394789+0000","last_clean":"2024-06-06T04:30:04.394789+0000","last_became_active":"2024-06-06T04:27:22.572674+0000","last_became_peered":"2024-06-06T04:27:22.572674+0000","last_unstale":"2024-06-06T04:30:04.394789+0000","last_undegraded":"2024-06-06T04:30:04.394789+0000","last_fullsized":"2024-06-06T04:30:04.394789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:57:22.653052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905675+0000","last_change":"2024-06-06T04:27:32.617934+0000","last_active":"2024-06-06T04:29:05.905675+0000","last_peered":"2024-06-06T04:29:05.905675+0000","last_clean":"2024-06-06T04:29:05.905675+0000","last_became_active":"2024-06-06T04:27:32.617537+0000","last_became_peered":"2024-06-06T04:27:32.617537+0000","last_unstale":"2024-06-06T04:29:05.905675+0000","last_undegraded":"2024-06-06T04:29:05.905675+0000","last_fullsized":"2024-06-06T04:29:05.905675+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:21:48.319434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'5","reported_seq":149,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.864491+0000","last_change":"2024-06-06T04:27:19.306236+0000","last_active":"2024-06-06T04:30:04.864491+0000","last_peered":"2024-06-06T04:30:04.864491+0000","last_clean":"2024-06-06T04:30:04.864491+0000","last_became_active":"2024-06-06T04:27:19.305850+0000","last_became_peered":"2024-06-06T04:27:19.305850+0000","last_unstale":"2024-06-06T04:30:04.864491+0000","last_undegraded":"2024-06-06T04:30:04.864491+0000","last_fullsized":"2024-06-06T04:30:04.864491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:58:47.310040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.365431+0000","last_change":"2024-06-06T04:27:22.567440+0000","last_active":"2024-06-06T04:30:04.365431+0000","last_peered":"2024-06-06T04:30:04.365431+0000","last_clean":"2024-06-06T04:30:04.365431+0000","last_became_active":"2024-06-06T04:27:22.566732+0000","last_became_peered":"2024-06-06T04:27:22.566732+0000","last_unstale":"2024-06-06T04:30:04.365431+0000","last_undegraded":"2024-06-06T04:30:04.365431+0000","last_fullsized":"2024-06-06T04:30:04.365431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:48:46.270103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905710+0000","last_change":"2024-06-06T04:27:32.626554+0000","last_active":"2024-06-06T04:29:05.905710+0000","last_peered":"2024-06-06T04:29:05.905710+0000","last_clean":"2024-06-06T04:29:05.905710+0000","last_became_active":"2024-06-06T04:27:32.626108+0000","last_became_peered":"2024-06-06T04:27:32.626108+0000","last_unstale":"2024-06-06T04:29:05.905710+0000","last_undegraded":"2024-06-06T04:29:05.905710+0000","last_fullsized":"2024-06-06T04:29:05.905710+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:37:43.811005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.262695+0000","last_change":"2024-06-06T04:27:19.306821+0000","last_active":"2024-06-06T04:29:06.262695+0000","last_peered":"2024-06-06T04:29:06.262695+0000","last_clean":"2024-06-06T04:29:06.262695+0000","last_became_active":"2024-06-06T04:27:19.306489+0000","last_became_peered":"2024-06-06T04:27:19.306489+0000","last_unstale":"2024-06-06T04:29:06.262695+0000","last_undegraded":"2024-06-06T04:29:06.262695+0000","last_fullsized":"2024-06-06T04:29:06.262695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:31:28.973112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.378929+0000","last_change":"2024-06-06T04:27:22.570617+0000","last_active":"2024-06-06T04:30:04.378929+0000","last_peered":"2024-06-06T04:30:04.378929+0000","last_clean":"2024-06-06T04:30:04.378929+0000","last_became_active":"2024-06-06T04:27:22.570271+0000","last_became_peered":"2024-06-06T04:27:22.570271+0000","last_unstale":"2024-06-06T04:30:04.378929+0000","last_undegraded":"2024-06-06T04:30:04.378929+0000","last_fullsized":"2024-06-06T04:30:04.378929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:45:33.890164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.909010+0000","last_change":"2024-06-06T04:27:32.628132+0000","last_active":"2024-06-06T04:29:05.909010+0000","last_peered":"2024-06-06T04:29:05.909010+0000","last_clean":"2024-06-06T04:29:05.909010+0000","last_became_active":"2024-06-06T04:27:32.627658+0000","last_became_peered":"2024-06-06T04:27:32.627658+0000","last_unstale":"2024-06-06T04:29:05.909010+0000","last_undegraded":"2024-06-06T04:29:05.909010+0000","last_fullsized":"2024-06-06T04:29:05.909010+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:24.817079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904125+0000","last_change":"2024-06-06T04:27:19.300510+0000","last_active":"2024-06-06T04:29:05.904125+0000","last_peered":"2024-06-06T04:29:05.904125+0000","last_clean":"2024-06-06T04:29:05.904125+0000","last_became_active":"2024-06-06T04:27:19.300150+0000","last_became_peered":"2024-06-06T04:27:19.300150+0000","last_unstale":"2024-06-06T04:29:05.904125+0000","last_undegraded":"2024-06-06T04:29:05.904125+0000","last_fullsized":"2024-06-06T04:29:05.904125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:38:25.084690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381098+0000","last_change":"2024-06-06T04:27:22.563864+0000","last_active":"2024-06-06T04:30:04.381098+0000","last_peered":"2024-06-06T04:30:04.381098+0000","last_clean":"2024-06-06T04:30:04.381098+0000","last_became_active":"2024-06-06T04:27:22.563555+0000","last_became_peered":"2024-06-06T04:27:22.563555+0000","last_unstale":"2024-06-06T04:30:04.381098+0000","last_undegraded":"2024-06-06T04:30:04.381098+0000","last_fullsized":"2024-06-06T04:30:04.381098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:39:33.707404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.165015+0000","last_change":"2024-06-06T04:27:32.617627+0000","last_active":"2024-06-06T04:29:06.165015+0000","last_peered":"2024-06-06T04:29:06.165015+0000","last_clean":"2024-06-06T04:29:06.165015+0000","last_became_active":"2024-06-06T04:27:32.617187+0000","last_became_peered":"2024-06-06T04:27:32.617187+0000","last_unstale":"2024-06-06T04:29:06.165015+0000","last_undegraded":"2024-06-06T04:29:06.165015+0000","last_fullsized":"2024-06-06T04:29:06.165015+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:24:52.554176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790234+0000","last_change":"2024-06-06T04:27:19.304316+0000","last_active":"2024-06-06T04:29:06.790234+0000","last_peered":"2024-06-06T04:29:06.790234+0000","last_clean":"2024-06-06T04:29:06.790234+0000","last_became_active":"2024-06-06T04:27:19.303952+0000","last_became_peered":"2024-06-06T04:27:19.303952+0000","last_unstale":"2024-06-06T04:29:06.790234+0000","last_undegraded":"2024-06-06T04:29:06.790234+0000","last_fullsized":"2024-06-06T04:29:06.790234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:26.886916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.371820+0000","last_change":"2024-06-06T04:27:22.561824+0000","last_active":"2024-06-06T04:30:04.371820+0000","last_peered":"2024-06-06T04:30:04.371820+0000","last_clean":"2024-06-06T04:30:04.371820+0000","last_became_active":"2024-06-06T04:27:22.561495+0000","last_became_peered":"2024-06-06T04:27:22.561495+0000","last_unstale":"2024-06-06T04:30:04.371820+0000","last_undegraded":"2024-06-06T04:30:04.371820+0000","last_fullsized":"2024-06-06T04:30:04.371820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:40:31.314627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560380+0000","last_change":"2024-06-06T04:27:32.624540+0000","last_active":"2024-06-06T04:29:06.560380+0000","last_peered":"2024-06-06T04:29:06.560380+0000","last_clean":"2024-06-06T04:29:06.560380+0000","last_became_active":"2024-06-06T04:27:32.624036+0000","last_became_peered":"2024-06-06T04:27:32.624036+0000","last_unstale":"2024-06-06T04:29:06.560380+0000","last_undegraded":"2024-06-06T04:29:06.560380+0000","last_fullsized":"2024-06-06T04:29:06.560380+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:16.983576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":143,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.864030+0000","last_change":"2024-06-06T04:27:19.304614+0000","last_active":"2024-06-06T04:30:04.864030+0000","last_peered":"2024-06-06T04:30:04.864030+0000","last_clean":"2024-06-06T04:30:04.864030+0000","last_became_active":"2024-06-06T04:27:19.304126+0000","last_became_peered":"2024-06-06T04:27:19.304126+0000","last_unstale":"2024-06-06T04:30:04.864030+0000","last_undegraded":"2024-06-06T04:30:04.864030+0000","last_fullsized":"2024-06-06T04:30:04.864030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:07:39.908630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.907555+0000","last_change":"2024-06-06T04:27:22.572534+0000","last_active":"2024-06-06T04:29:06.907555+0000","last_peered":"2024-06-06T04:29:06.907555+0000","last_clean":"2024-06-06T04:29:06.907555+0000","last_became_active":"2024-06-06T04:27:22.572214+0000","last_became_peered":"2024-06-06T04:27:22.572214+0000","last_unstale":"2024-06-06T04:29:06.907555+0000","last_undegraded":"2024-06-06T04:29:06.907555+0000","last_fullsized":"2024-06-06T04:29:06.907555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:13.176045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.262294+0000","last_change":"2024-06-06T04:28:05.723450+0000","last_active":"2024-06-06T04:29:06.262294+0000","last_peered":"2024-06-06T04:29:06.262294+0000","last_clean":"2024-06-06T04:29:06.262294+0000","last_became_active":"2024-06-06T04:28:05.722928+0000","last_became_peered":"2024-06-06T04:28:05.722928+0000","last_unstale":"2024-06-06T04:29:06.262294+0000","last_undegraded":"2024-06-06T04:29:06.262294+0000","last_fullsized":"2024-06-06T04:29:06.262294+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:53.093276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905918+0000","last_change":"2024-06-06T04:27:19.308465+0000","last_active":"2024-06-06T04:29:05.905918+0000","last_peered":"2024-06-06T04:29:05.905918+0000","last_clean":"2024-06-06T04:29:05.905918+0000","last_became_active":"2024-06-06T04:27:19.308077+0000","last_became_peered":"2024-06-06T04:27:19.308077+0000","last_unstale":"2024-06-06T04:29:05.905918+0000","last_undegraded":"2024-06-06T04:29:05.905918+0000","last_fullsized":"2024-06-06T04:29:05.905918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:37:48.282613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385739+0000","last_change":"2024-06-06T04:27:22.574141+0000","last_active":"2024-06-06T04:30:04.385739+0000","last_peered":"2024-06-06T04:30:04.385739+0000","last_clean":"2024-06-06T04:30:04.385739+0000","last_became_active":"2024-06-06T04:27:22.573679+0000","last_became_peered":"2024-06-06T04:27:22.573679+0000","last_unstale":"2024-06-06T04:30:04.385739+0000","last_undegraded":"2024-06-06T04:30:04.385739+0000","last_fullsized":"2024-06-06T04:30:04.385739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:27:25.177990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401856+0000","last_change":"2024-06-06T04:27:32.624148+0000","last_active":"2024-06-06T04:29:06.401856+0000","last_peered":"2024-06-06T04:29:06.401856+0000","last_clean":"2024-06-06T04:29:06.401856+0000","last_became_active":"2024-06-06T04:27:32.623448+0000","last_became_peered":"2024-06-06T04:27:32.623448+0000","last_unstale":"2024-06-06T04:29:06.401856+0000","last_undegraded":"2024-06-06T04:29:06.401856+0000","last_fullsized":"2024-06-06T04:29:06.401856+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:02.037296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790253+0000","last_change":"2024-06-06T04:27:19.516875+0000","last_active":"2024-06-06T04:29:06.790253+0000","last_peered":"2024-06-06T04:29:06.790253+0000","last_clean":"2024-06-06T04:29:06.790253+0000","last_became_active":"2024-06-06T04:27:19.516393+0000","last_became_peered":"2024-06-06T04:27:19.516393+0000","last_unstale":"2024-06-06T04:29:06.790253+0000","last_undegraded":"2024-06-06T04:29:06.790253+0000","last_fullsized":"2024-06-06T04:29:06.790253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:20.079218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.908265+0000","last_change":"2024-06-06T04:28:05.724043+0000","last_active":"2024-06-06T04:29:06.908265+0000","last_peered":"2024-06-06T04:29:06.908265+0000","last_clean":"2024-06-06T04:29:06.908265+0000","last_became_active":"2024-06-06T04:28:05.723622+0000","last_became_peered":"2024-06-06T04:28:05.723622+0000","last_unstale":"2024-06-06T04:29:06.908265+0000","last_undegraded":"2024-06-06T04:29:06.908265+0000","last_fullsized":"2024-06-06T04:29:06.908265+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:42.162400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.907660+0000","last_change":"2024-06-06T04:27:32.626366+0000","last_active":"2024-06-06T04:29:05.907660+0000","last_peered":"2024-06-06T04:29:05.907660+0000","last_clean":"2024-06-06T04:29:05.907660+0000","last_became_active":"2024-06-06T04:27:32.625847+0000","last_became_peered":"2024-06-06T04:27:32.625847+0000","last_unstale":"2024-06-06T04:29:05.907660+0000","last_undegraded":"2024-06-06T04:29:05.907660+0000","last_fullsized":"2024-06-06T04:29:05.907660+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:47:44.167531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'38","reported_seq":285,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:05.865651+0000","last_change":"2024-06-06T04:27:19.303581+0000","last_active":"2024-06-06T04:30:05.865651+0000","last_peered":"2024-06-06T04:30:05.865651+0000","last_clean":"2024-06-06T04:30:05.865651+0000","last_became_active":"2024-06-06T04:27:19.303234+0000","last_became_peered":"2024-06-06T04:27:19.303234+0000","last_unstale":"2024-06-06T04:30:05.865651+0000","last_undegraded":"2024-06-06T04:30:05.865651+0000","last_fullsized":"2024-06-06T04:30:05.865651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:37.721588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64865,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":204,"num_read_kb":166,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.391077+0000","last_change":"2024-06-06T04:27:22.567177+0000","last_active":"2024-06-06T04:30:04.391077+0000","last_peered":"2024-06-06T04:30:04.391077+0000","last_clean":"2024-06-06T04:30:04.391077+0000","last_became_active":"2024-06-06T04:27:22.566751+0000","last_became_peered":"2024-06-06T04:27:22.566751+0000","last_unstale":"2024-06-06T04:30:04.391077+0000","last_undegraded":"2024-06-06T04:30:04.391077+0000","last_fullsized":"2024-06-06T04:30:04.391077+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:37:15.914505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790613+0000","last_change":"2024-06-06T04:27:32.619319+0000","last_active":"2024-06-06T04:29:06.790613+0000","last_peered":"2024-06-06T04:29:06.790613+0000","last_clean":"2024-06-06T04:29:06.790613+0000","last_became_active":"2024-06-06T04:27:32.618843+0000","last_became_peered":"2024-06-06T04:27:32.618843+0000","last_unstale":"2024-06-06T04:29:06.790613+0000","last_undegraded":"2024-06-06T04:29:06.790613+0000","last_fullsized":"2024-06-06T04:29:06.790613+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:32:20.071512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905924+0000","last_change":"2024-06-06T04:27:19.304731+0000","last_active":"2024-06-06T04:29:05.905924+0000","last_peered":"2024-06-06T04:29:05.905924+0000","last_clean":"2024-06-06T04:29:05.905924+0000","last_became_active":"2024-06-06T04:27:19.304351+0000","last_became_peered":"2024-06-06T04:27:19.304351+0000","last_unstale":"2024-06-06T04:29:05.905924+0000","last_undegraded":"2024-06-06T04:29:05.905924+0000","last_fullsized":"2024-06-06T04:29:05.905924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:16:14.136633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.909492+0000","last_change":"2024-06-06T04:29:05.909041+0000","last_active":"2024-06-06T04:29:05.909492+0000","last_peered":"2024-06-06T04:29:05.909492+0000","last_clean":"2024-06-06T04:29:05.909492+0000","last_became_active":"2024-06-06T04:29:05.908666+0000","last_became_peered":"2024-06-06T04:29:05.908666+0000","last_unstale":"2024-06-06T04:29:05.909492+0000","last_undegraded":"2024-06-06T04:29:05.909492+0000","last_fullsized":"2024-06-06T04:29:05.909492+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:47:55.225572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908861+0000","last_change":"2024-06-06T04:27:32.628645+0000","last_active":"2024-06-06T04:29:05.908861+0000","last_peered":"2024-06-06T04:29:05.908861+0000","last_clean":"2024-06-06T04:29:05.908861+0000","last_became_active":"2024-06-06T04:27:32.627586+0000","last_became_peered":"2024-06-06T04:27:32.627586+0000","last_unstale":"2024-06-06T04:29:05.908861+0000","last_undegraded":"2024-06-06T04:29:05.908861+0000","last_fullsized":"2024-06-06T04:29:05.908861+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:46:53.094511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790029+0000","last_change":"2024-06-06T04:27:19.303805+0000","last_active":"2024-06-06T04:29:06.790029+0000","last_peered":"2024-06-06T04:29:06.790029+0000","last_clean":"2024-06-06T04:29:06.790029+0000","last_became_active":"2024-06-06T04:27:19.303391+0000","last_became_peered":"2024-06-06T04:27:19.303391+0000","last_unstale":"2024-06-06T04:29:06.790029+0000","last_undegraded":"2024-06-06T04:29:06.790029+0000","last_fullsized":"2024-06-06T04:29:06.790029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:30:27.336567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905833+0000","last_change":"2024-06-06T04:27:22.575148+0000","last_active":"2024-06-06T04:29:05.905833+0000","last_peered":"2024-06-06T04:29:05.905833+0000","last_clean":"2024-06-06T04:29:05.905833+0000","last_became_active":"2024-06-06T04:27:22.574591+0000","last_became_peered":"2024-06-06T04:27:22.574591+0000","last_unstale":"2024-06-06T04:29:05.905833+0000","last_undegraded":"2024-06-06T04:29:05.905833+0000","last_fullsized":"2024-06-06T04:29:05.905833+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:35:23.404644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214398+0000","last_change":"2024-06-06T04:28:05.726734+0000","last_active":"2024-06-06T04:29:06.214398+0000","last_peered":"2024-06-06T04:29:06.214398+0000","last_clean":"2024-06-06T04:29:06.214398+0000","last_became_active":"2024-06-06T04:28:05.726238+0000","last_became_peered":"2024-06-06T04:28:05.726238+0000","last_unstale":"2024-06-06T04:29:06.214398+0000","last_undegraded":"2024-06-06T04:29:06.214398+0000","last_fullsized":"2024-06-06T04:29:06.214398+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:47.841475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"100'5","reported_seq":151,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:05.866998+0000","last_change":"2024-06-06T04:27:19.305571+0000","last_active":"2024-06-06T04:30:05.866998+0000","last_peered":"2024-06-06T04:30:05.866998+0000","last_clean":"2024-06-06T04:30:05.866998+0000","last_became_active":"2024-06-06T04:27:19.305183+0000","last_became_peered":"2024-06-06T04:27:19.305183+0000","last_unstale":"2024-06-06T04:30:05.866998+0000","last_undegraded":"2024-06-06T04:30:05.866998+0000","last_fullsized":"2024-06-06T04:30:05.866998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:51.205209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.264354+0000","last_change":"2024-06-06T04:29:06.263872+0000","last_active":"2024-06-06T04:29:06.264354+0000","last_peered":"2024-06-06T04:29:06.264354+0000","last_clean":"2024-06-06T04:29:06.264354+0000","last_became_active":"2024-06-06T04:29:06.263487+0000","last_became_peered":"2024-06-06T04:29:06.263487+0000","last_unstale":"2024-06-06T04:29:06.264354+0000","last_undegraded":"2024-06-06T04:29:06.264354+0000","last_fullsized":"2024-06-06T04:29:06.264354+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:15.965926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904639+0000","last_change":"2024-06-06T04:28:05.724892+0000","last_active":"2024-06-06T04:29:05.904639+0000","last_peered":"2024-06-06T04:29:05.904639+0000","last_clean":"2024-06-06T04:29:05.904639+0000","last_became_active":"2024-06-06T04:28:05.724420+0000","last_became_peered":"2024-06-06T04:28:05.724420+0000","last_unstale":"2024-06-06T04:29:05.904639+0000","last_undegraded":"2024-06-06T04:29:05.904639+0000","last_fullsized":"2024-06-06T04:29:05.904639+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:39.879481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":60,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.213846+0000","last_change":"2024-06-06T04:27:19.516687+0000","last_active":"2024-06-06T04:29:06.213846+0000","last_peered":"2024-06-06T04:29:06.213846+0000","last_clean":"2024-06-06T04:29:06.213846+0000","last_became_active":"2024-06-06T04:27:19.516271+0000","last_became_peered":"2024-06-06T04:27:19.516271+0000","last_unstale":"2024-06-06T04:29:06.213846+0000","last_undegraded":"2024-06-06T04:29:06.213846+0000","last_fullsized":"2024-06-06T04:29:06.213846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:33:46.817727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394832+0000","last_change":"2024-06-06T04:27:22.572671+0000","last_active":"2024-06-06T04:30:04.394832+0000","last_peered":"2024-06-06T04:30:04.394832+0000","last_clean":"2024-06-06T04:30:04.394832+0000","last_became_active":"2024-06-06T04:27:22.572230+0000","last_became_peered":"2024-06-06T04:27:22.572230+0000","last_unstale":"2024-06-06T04:30:04.394832+0000","last_undegraded":"2024-06-06T04:30:04.394832+0000","last_fullsized":"2024-06-06T04:30:04.394832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:36:48.393092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.789710+0000","last_change":"2024-06-06T04:27:32.620131+0000","last_active":"2024-06-06T04:29:06.789710+0000","last_peered":"2024-06-06T04:29:06.789710+0000","last_clean":"2024-06-06T04:29:06.789710+0000","last_became_active":"2024-06-06T04:27:32.619686+0000","last_became_peered":"2024-06-06T04:27:32.619686+0000","last_unstale":"2024-06-06T04:29:06.789710+0000","last_undegraded":"2024-06-06T04:29:06.789710+0000","last_fullsized":"2024-06-06T04:29:06.789710+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:19:17.130410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214545+0000","last_change":"2024-06-06T04:27:19.304682+0000","last_active":"2024-06-06T04:29:06.214545+0000","last_peered":"2024-06-06T04:29:06.214545+0000","last_clean":"2024-06-06T04:29:06.214545+0000","last_became_active":"2024-06-06T04:27:19.304262+0000","last_became_peered":"2024-06-06T04:27:19.304262+0000","last_unstale":"2024-06-06T04:29:06.214545+0000","last_undegraded":"2024-06-06T04:29:06.214545+0000","last_fullsized":"2024-06-06T04:29:06.214545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:45.819520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.377096+0000","last_change":"2024-06-06T04:29:06.791555+0000","last_active":"2024-06-06T04:30:04.377096+0000","last_peered":"2024-06-06T04:30:04.377096+0000","last_clean":"2024-06-06T04:30:04.377096+0000","last_became_active":"2024-06-06T04:29:06.790925+0000","last_became_peered":"2024-06-06T04:29:06.790925+0000","last_unstale":"2024-06-06T04:30:04.377096+0000","last_undegraded":"2024-06-06T04:30:04.377096+0000","last_fullsized":"2024-06-06T04:30:04.377096+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:23:39.549238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.213893+0000","last_change":"2024-06-06T04:27:32.622092+0000","last_active":"2024-06-06T04:29:06.213893+0000","last_peered":"2024-06-06T04:29:06.213893+0000","last_clean":"2024-06-06T04:29:06.213893+0000","last_became_active":"2024-06-06T04:27:32.621618+0000","last_became_peered":"2024-06-06T04:27:32.621618+0000","last_unstale":"2024-06-06T04:29:06.213893+0000","last_undegraded":"2024-06-06T04:29:06.213893+0000","last_fullsized":"2024-06-06T04:29:06.213893+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:03.067292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.907896+0000","last_change":"2024-06-06T04:27:19.516803+0000","last_active":"2024-06-06T04:29:05.907896+0000","last_peered":"2024-06-06T04:29:05.907896+0000","last_clean":"2024-06-06T04:29:05.907896+0000","last_became_active":"2024-06-06T04:27:19.516458+0000","last_became_peered":"2024-06-06T04:27:19.516458+0000","last_unstale":"2024-06-06T04:29:05.907896+0000","last_undegraded":"2024-06-06T04:29:05.907896+0000","last_fullsized":"2024-06-06T04:29:05.907896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:34:53.593825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380334+0000","last_change":"2024-06-06T04:27:22.564306+0000","last_active":"2024-06-06T04:30:04.380334+0000","last_peered":"2024-06-06T04:30:04.380334+0000","last_clean":"2024-06-06T04:30:04.380334+0000","last_became_active":"2024-06-06T04:27:22.563922+0000","last_became_peered":"2024-06-06T04:27:22.563922+0000","last_unstale":"2024-06-06T04:30:04.380334+0000","last_undegraded":"2024-06-06T04:30:04.380334+0000","last_fullsized":"2024-06-06T04:30:04.380334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:42:23.189047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904530+0000","last_change":"2024-06-06T04:27:32.617014+0000","last_active":"2024-06-06T04:29:05.904530+0000","last_peered":"2024-06-06T04:29:05.904530+0000","last_clean":"2024-06-06T04:29:05.904530+0000","last_became_active":"2024-06-06T04:27:32.616639+0000","last_became_peered":"2024-06-06T04:27:32.616639+0000","last_unstale":"2024-06-06T04:29:05.904530+0000","last_undegraded":"2024-06-06T04:29:05.904530+0000","last_fullsized":"2024-06-06T04:29:05.904530+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:54:52.034711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905627+0000","last_change":"2024-06-06T04:27:19.305000+0000","last_active":"2024-06-06T04:29:05.905627+0000","last_peered":"2024-06-06T04:29:05.905627+0000","last_clean":"2024-06-06T04:29:05.905627+0000","last_became_active":"2024-06-06T04:27:19.304676+0000","last_became_peered":"2024-06-06T04:27:19.304676+0000","last_unstale":"2024-06-06T04:29:05.905627+0000","last_undegraded":"2024-06-06T04:29:05.905627+0000","last_fullsized":"2024-06-06T04:29:05.905627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:19:34.054455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.382734+0000","last_change":"2024-06-06T04:27:22.572680+0000","last_active":"2024-06-06T04:30:04.382734+0000","last_peered":"2024-06-06T04:30:04.382734+0000","last_clean":"2024-06-06T04:30:04.382734+0000","last_became_active":"2024-06-06T04:27:22.572348+0000","last_became_peered":"2024-06-06T04:27:22.572348+0000","last_unstale":"2024-06-06T04:30:04.382734+0000","last_undegraded":"2024-06-06T04:30:04.382734+0000","last_fullsized":"2024-06-06T04:30:04.382734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:05:26.215771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904503+0000","last_change":"2024-06-06T04:27:32.611587+0000","last_active":"2024-06-06T04:29:05.904503+0000","last_peered":"2024-06-06T04:29:05.904503+0000","last_clean":"2024-06-06T04:29:05.904503+0000","last_became_active":"2024-06-06T04:27:32.611183+0000","last_became_peered":"2024-06-06T04:27:32.611183+0000","last_unstale":"2024-06-06T04:29:05.904503+0000","last_undegraded":"2024-06-06T04:29:05.904503+0000","last_fullsized":"2024-06-06T04:29:05.904503+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:36.773254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.371612+0000","last_change":"2024-06-06T04:29:06.403321+0000","last_active":"2024-06-06T04:30:04.371612+0000","last_peered":"2024-06-06T04:30:04.371612+0000","last_clean":"2024-06-06T04:30:04.371612+0000","last_became_active":"2024-06-06T04:29:06.402909+0000","last_became_peered":"2024-06-06T04:29:06.402909+0000","last_unstale":"2024-06-06T04:30:04.371612+0000","last_undegraded":"2024-06-06T04:30:04.371612+0000","last_fullsized":"2024-06-06T04:30:04.371612+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:29:38.963372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":143,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:01.860156+0000","last_change":"2024-06-06T04:27:19.516361+0000","last_active":"2024-06-06T04:30:01.860156+0000","last_peered":"2024-06-06T04:30:01.860156+0000","last_clean":"2024-06-06T04:30:01.860156+0000","last_became_active":"2024-06-06T04:27:19.515973+0000","last_became_peered":"2024-06-06T04:27:19.515973+0000","last_unstale":"2024-06-06T04:30:01.860156+0000","last_undegraded":"2024-06-06T04:30:01.860156+0000","last_fullsized":"2024-06-06T04:30:01.860156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:37.298819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905945+0000","last_change":"2024-06-06T04:27:32.630528+0000","last_active":"2024-06-06T04:29:05.905945+0000","last_peered":"2024-06-06T04:29:05.905945+0000","last_clean":"2024-06-06T04:29:05.905945+0000","last_became_active":"2024-06-06T04:27:32.630215+0000","last_became_peered":"2024-06-06T04:27:32.630215+0000","last_unstale":"2024-06-06T04:29:05.905945+0000","last_undegraded":"2024-06-06T04:29:05.905945+0000","last_fullsized":"2024-06-06T04:29:05.905945+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:34:29.980193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392719+0000","last_change":"2024-06-06T04:27:22.567466+0000","last_active":"2024-06-06T04:30:04.392719+0000","last_peered":"2024-06-06T04:30:04.392719+0000","last_clean":"2024-06-06T04:30:04.392719+0000","last_became_active":"2024-06-06T04:27:22.567145+0000","last_became_peered":"2024-06-06T04:27:22.567145+0000","last_unstale":"2024-06-06T04:30:04.392719+0000","last_undegraded":"2024-06-06T04:30:04.392719+0000","last_fullsized":"2024-06-06T04:30:04.392719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:56:56.203027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401783+0000","last_change":"2024-06-06T04:27:19.302726+0000","last_active":"2024-06-06T04:29:06.401783+0000","last_peered":"2024-06-06T04:29:06.401783+0000","last_clean":"2024-06-06T04:29:06.401783+0000","last_became_active":"2024-06-06T04:27:19.302329+0000","last_became_peered":"2024-06-06T04:27:19.302329+0000","last_unstale":"2024-06-06T04:29:06.401783+0000","last_undegraded":"2024-06-06T04:29:06.401783+0000","last_fullsized":"2024-06-06T04:29:06.401783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:55:15.010632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":366,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905176+0000","last_change":"2024-06-06T04:25:53.854188+0000","last_active":"2024-06-06T04:29:05.905176+0000","last_peered":"2024-06-06T04:29:05.905176+0000","last_clean":"2024-06-06T04:29:05.905176+0000","last_became_active":"2024-06-06T04:25:53.853700+0000","last_became_peered":"2024-06-06T04:25:53.853700+0000","last_unstale":"2024-06-06T04:29:05.905176+0000","last_undegraded":"2024-06-06T04:29:05.905176+0000","last_fullsized":"2024-06-06T04:29:05.905176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:22:28.847333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:22:28.847333+0000","last_clean_scrub_stamp":"2024-06-06T04:22:28.847333+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:59:10.246373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790675+0000","last_change":"2024-06-06T04:27:32.623300+0000","last_active":"2024-06-06T04:29:06.790675+0000","last_peered":"2024-06-06T04:29:06.790675+0000","last_clean":"2024-06-06T04:29:06.790675+0000","last_became_active":"2024-06-06T04:27:32.622811+0000","last_became_peered":"2024-06-06T04:27:32.622811+0000","last_unstale":"2024-06-06T04:29:06.790675+0000","last_undegraded":"2024-06-06T04:29:06.790675+0000","last_fullsized":"2024-06-06T04:29:06.790675+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:28.383079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904009+0000","last_change":"2024-06-06T04:27:22.566617+0000","last_active":"2024-06-06T04:29:05.904009+0000","last_peered":"2024-06-06T04:29:05.904009+0000","last_clean":"2024-06-06T04:29:05.904009+0000","last_became_active":"2024-06-06T04:27:22.566256+0000","last_became_peered":"2024-06-06T04:27:22.566256+0000","last_unstale":"2024-06-06T04:29:05.904009+0000","last_undegraded":"2024-06-06T04:29:05.904009+0000","last_fullsized":"2024-06-06T04:29:05.904009+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:50:41.318475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905457+0000","last_change":"2024-06-06T04:27:19.305941+0000","last_active":"2024-06-06T04:29:05.905457+0000","last_peered":"2024-06-06T04:29:05.905457+0000","last_clean":"2024-06-06T04:29:05.905457+0000","last_became_active":"2024-06-06T04:27:19.305037+0000","last_became_peered":"2024-06-06T04:27:19.305037+0000","last_unstale":"2024-06-06T04:29:05.905457+0000","last_undegraded":"2024-06-06T04:29:05.905457+0000","last_fullsized":"2024-06-06T04:29:05.905457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:54.084427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908935+0000","last_change":"2024-06-06T04:27:32.621182+0000","last_active":"2024-06-06T04:29:05.908935+0000","last_peered":"2024-06-06T04:29:05.908935+0000","last_clean":"2024-06-06T04:29:05.908935+0000","last_became_active":"2024-06-06T04:27:32.620710+0000","last_became_peered":"2024-06-06T04:27:32.620710+0000","last_unstale":"2024-06-06T04:29:05.908935+0000","last_undegraded":"2024-06-06T04:29:05.908935+0000","last_fullsized":"2024-06-06T04:29:05.908935+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:05.512309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214246+0000","last_change":"2024-06-06T04:27:19.517033+0000","last_active":"2024-06-06T04:29:06.214246+0000","last_peered":"2024-06-06T04:29:06.214246+0000","last_clean":"2024-06-06T04:29:06.214246+0000","last_became_active":"2024-06-06T04:27:19.516278+0000","last_became_peered":"2024-06-06T04:27:19.516278+0000","last_unstale":"2024-06-06T04:29:06.214246+0000","last_undegraded":"2024-06-06T04:29:06.214246+0000","last_fullsized":"2024-06-06T04:29:06.214246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:11:20.336818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904346+0000","last_change":"2024-06-06T04:27:22.566261+0000","last_active":"2024-06-06T04:29:05.904346+0000","last_peered":"2024-06-06T04:29:05.904346+0000","last_clean":"2024-06-06T04:29:05.904346+0000","last_became_active":"2024-06-06T04:27:22.565875+0000","last_became_peered":"2024-06-06T04:27:22.565875+0000","last_unstale":"2024-06-06T04:29:05.904346+0000","last_undegraded":"2024-06-06T04:29:05.904346+0000","last_fullsized":"2024-06-06T04:29:05.904346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:49:55.923950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560350+0000","last_change":"2024-06-06T04:27:32.619854+0000","last_active":"2024-06-06T04:29:06.560350+0000","last_peered":"2024-06-06T04:29:06.560350+0000","last_clean":"2024-06-06T04:29:06.560350+0000","last_became_active":"2024-06-06T04:27:32.618765+0000","last_became_peered":"2024-06-06T04:27:32.618765+0000","last_unstale":"2024-06-06T04:29:06.560350+0000","last_undegraded":"2024-06-06T04:29:06.560350+0000","last_fullsized":"2024-06-06T04:29:06.560350+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:47:05.283751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'12","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:31.134232+0000","last_change":"2024-06-06T04:27:19.303057+0000","last_active":"2024-06-06T04:29:31.134232+0000","last_peered":"2024-06-06T04:29:31.134232+0000","last_clean":"2024-06-06T04:29:31.134232+0000","last_became_active":"2024-06-06T04:27:19.302716+0000","last_became_peered":"2024-06-06T04:27:19.302716+0000","last_unstale":"2024-06-06T04:29:31.134232+0000","last_undegraded":"2024-06-06T04:29:31.134232+0000","last_fullsized":"2024-06-06T04:29:31.134232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:35.273663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381463+0000","last_change":"2024-06-06T04:27:22.575274+0000","last_active":"2024-06-06T04:30:04.381463+0000","last_peered":"2024-06-06T04:30:04.381463+0000","last_clean":"2024-06-06T04:30:04.381463+0000","last_became_active":"2024-06-06T04:27:22.574979+0000","last_became_peered":"2024-06-06T04:27:22.574979+0000","last_unstale":"2024-06-06T04:30:04.381463+0000","last_undegraded":"2024-06-06T04:30:04.381463+0000","last_fullsized":"2024-06-06T04:30:04.381463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:30.718398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905413+0000","last_change":"2024-06-06T04:27:32.619939+0000","last_active":"2024-06-06T04:29:05.905413+0000","last_peered":"2024-06-06T04:29:05.905413+0000","last_clean":"2024-06-06T04:29:05.905413+0000","last_became_active":"2024-06-06T04:27:32.619499+0000","last_became_peered":"2024-06-06T04:27:32.619499+0000","last_unstale":"2024-06-06T04:29:05.905413+0000","last_undegraded":"2024-06-06T04:29:05.905413+0000","last_fullsized":"2024-06-06T04:29:05.905413+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:41:13.933546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402340+0000","last_change":"2024-06-06T04:27:19.397552+0000","last_active":"2024-06-06T04:29:06.402340+0000","last_peered":"2024-06-06T04:29:06.402340+0000","last_clean":"2024-06-06T04:29:06.402340+0000","last_became_active":"2024-06-06T04:27:19.397186+0000","last_became_peered":"2024-06-06T04:27:19.397186+0000","last_unstale":"2024-06-06T04:29:06.402340+0000","last_undegraded":"2024-06-06T04:29:06.402340+0000","last_fullsized":"2024-06-06T04:29:06.402340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:24:13.403860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.384537+0000","last_change":"2024-06-06T04:28:05.728212+0000","last_active":"2024-06-06T04:30:04.384537+0000","last_peered":"2024-06-06T04:30:04.384537+0000","last_clean":"2024-06-06T04:30:04.384537+0000","last_became_active":"2024-06-06T04:28:05.727748+0000","last_became_peered":"2024-06-06T04:28:05.727748+0000","last_unstale":"2024-06-06T04:30:04.384537+0000","last_undegraded":"2024-06-06T04:30:04.384537+0000","last_fullsized":"2024-06-06T04:30:04.384537+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:50.583716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402083+0000","last_change":"2024-06-06T04:28:05.896480+0000","last_active":"2024-06-06T04:29:06.402083+0000","last_peered":"2024-06-06T04:29:06.402083+0000","last_clean":"2024-06-06T04:29:06.402083+0000","last_became_active":"2024-06-06T04:28:05.895967+0000","last_became_peered":"2024-06-06T04:28:05.895967+0000","last_unstale":"2024-06-06T04:29:06.402083+0000","last_undegraded":"2024-06-06T04:29:06.402083+0000","last_fullsized":"2024-06-06T04:29:06.402083+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:26:19.846884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.907843+0000","last_change":"2024-06-06T04:27:19.516515+0000","last_active":"2024-06-06T04:29:06.907843+0000","last_peered":"2024-06-06T04:29:06.907843+0000","last_clean":"2024-06-06T04:29:06.907843+0000","last_became_active":"2024-06-06T04:27:19.516087+0000","last_became_peered":"2024-06-06T04:27:19.516087+0000","last_unstale":"2024-06-06T04:29:06.907843+0000","last_undegraded":"2024-06-06T04:29:06.907843+0000","last_fullsized":"2024-06-06T04:29:06.907843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:58.440226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":40,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.342173+0000","last_change":"2024-06-06T04:27:22.562367+0000","last_active":"2024-06-06T04:30:04.342173+0000","last_peered":"2024-06-06T04:30:04.342173+0000","last_clean":"2024-06-06T04:30:04.342173+0000","last_became_active":"2024-06-06T04:27:22.561947+0000","last_became_peered":"2024-06-06T04:27:22.561947+0000","last_unstale":"2024-06-06T04:30:04.342173+0000","last_undegraded":"2024-06-06T04:30:04.342173+0000","last_fullsized":"2024-06-06T04:30:04.342173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:28:06.200748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908183+0000","last_change":"2024-06-06T04:27:32.631577+0000","last_active":"2024-06-06T04:29:05.908183+0000","last_peered":"2024-06-06T04:29:05.908183+0000","last_clean":"2024-06-06T04:29:05.908183+0000","last_became_active":"2024-06-06T04:27:32.629555+0000","last_became_peered":"2024-06-06T04:27:32.629555+0000","last_unstale":"2024-06-06T04:29:05.908183+0000","last_undegraded":"2024-06-06T04:29:05.908183+0000","last_fullsized":"2024-06-06T04:29:05.908183+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:52:56.197281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560477+0000","last_change":"2024-06-06T04:27:19.298880+0000","last_active":"2024-06-06T04:29:06.560477+0000","last_peered":"2024-06-06T04:29:06.560477+0000","last_clean":"2024-06-06T04:29:06.560477+0000","last_became_active":"2024-06-06T04:27:19.298551+0000","last_became_peered":"2024-06-06T04:27:19.298551+0000","last_unstale":"2024-06-06T04:29:06.560477+0000","last_undegraded":"2024-06-06T04:29:06.560477+0000","last_fullsized":"2024-06-06T04:29:06.560477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:11:32.145401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.383001+0000","last_change":"2024-06-06T04:27:22.575065+0000","last_active":"2024-06-06T04:30:04.383001+0000","last_peered":"2024-06-06T04:30:04.383001+0000","last_clean":"2024-06-06T04:30:04.383001+0000","last_became_active":"2024-06-06T04:27:22.574469+0000","last_became_peered":"2024-06-06T04:27:22.574469+0000","last_unstale":"2024-06-06T04:30:04.383001+0000","last_undegraded":"2024-06-06T04:30:04.383001+0000","last_fullsized":"2024-06-06T04:30:04.383001+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:29:44.943833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402041+0000","last_change":"2024-06-06T04:27:32.626678+0000","last_active":"2024-06-06T04:29:06.402041+0000","last_peered":"2024-06-06T04:29:06.402041+0000","last_clean":"2024-06-06T04:29:06.402041+0000","last_became_active":"2024-06-06T04:27:32.626167+0000","last_became_peered":"2024-06-06T04:27:32.626167+0000","last_unstale":"2024-06-06T04:29:06.402041+0000","last_undegraded":"2024-06-06T04:29:06.402041+0000","last_fullsized":"2024-06-06T04:29:06.402041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:56:26.363195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905544+0000","last_change":"2024-06-06T04:27:19.306466+0000","last_active":"2024-06-06T04:29:05.905544+0000","last_peered":"2024-06-06T04:29:05.905544+0000","last_clean":"2024-06-06T04:29:05.905544+0000","last_became_active":"2024-06-06T04:27:19.303821+0000","last_became_peered":"2024-06-06T04:27:19.303821+0000","last_unstale":"2024-06-06T04:29:05.905544+0000","last_undegraded":"2024-06-06T04:29:05.905544+0000","last_fullsized":"2024-06-06T04:29:05.905544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:23:53.593998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.386917+0000","last_change":"2024-06-06T04:27:22.562662+0000","last_active":"2024-06-06T04:30:04.386917+0000","last_peered":"2024-06-06T04:30:04.386917+0000","last_clean":"2024-06-06T04:30:04.386917+0000","last_became_active":"2024-06-06T04:27:22.562318+0000","last_became_peered":"2024-06-06T04:27:22.562318+0000","last_unstale":"2024-06-06T04:30:04.386917+0000","last_undegraded":"2024-06-06T04:30:04.386917+0000","last_fullsized":"2024-06-06T04:30:04.386917+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:31:26.639275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402218+0000","last_change":"2024-06-06T04:27:32.619970+0000","last_active":"2024-06-06T04:29:06.402218+0000","last_peered":"2024-06-06T04:29:06.402218+0000","last_clean":"2024-06-06T04:29:06.402218+0000","last_became_active":"2024-06-06T04:27:32.619464+0000","last_became_peered":"2024-06-06T04:27:32.619464+0000","last_unstale":"2024-06-06T04:29:06.402218+0000","last_undegraded":"2024-06-06T04:29:06.402218+0000","last_fullsized":"2024-06-06T04:29:06.402218+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:44.844018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214056+0000","last_change":"2024-06-06T04:27:19.308172+0000","last_active":"2024-06-06T04:29:06.214056+0000","last_peered":"2024-06-06T04:29:06.214056+0000","last_clean":"2024-06-06T04:29:06.214056+0000","last_became_active":"2024-06-06T04:27:19.307780+0000","last_became_peered":"2024-06-06T04:27:19.307780+0000","last_unstale":"2024-06-06T04:29:06.214056+0000","last_undegraded":"2024-06-06T04:29:06.214056+0000","last_fullsized":"2024-06-06T04:29:06.214056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:39:11.653649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905770+0000","last_change":"2024-06-06T04:27:22.568881+0000","last_active":"2024-06-06T04:29:05.905770+0000","last_peered":"2024-06-06T04:29:05.905770+0000","last_clean":"2024-06-06T04:29:05.905770+0000","last_became_active":"2024-06-06T04:27:22.568489+0000","last_became_peered":"2024-06-06T04:27:22.568489+0000","last_unstale":"2024-06-06T04:29:05.905770+0000","last_undegraded":"2024-06-06T04:29:05.905770+0000","last_fullsized":"2024-06-06T04:29:05.905770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:00:22.339313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.907971+0000","last_change":"2024-06-06T04:27:32.622357+0000","last_active":"2024-06-06T04:29:06.907971+0000","last_peered":"2024-06-06T04:29:06.907971+0000","last_clean":"2024-06-06T04:29:06.907971+0000","last_became_active":"2024-06-06T04:27:32.621680+0000","last_became_peered":"2024-06-06T04:27:32.621680+0000","last_unstale":"2024-06-06T04:29:06.907971+0000","last_undegraded":"2024-06-06T04:29:06.907971+0000","last_fullsized":"2024-06-06T04:29:06.907971+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:00.677721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.906141+0000","last_change":"2024-06-06T04:27:19.516890+0000","last_active":"2024-06-06T04:29:05.906141+0000","last_peered":"2024-06-06T04:29:05.906141+0000","last_clean":"2024-06-06T04:29:05.906141+0000","last_became_active":"2024-06-06T04:27:19.516490+0000","last_became_peered":"2024-06-06T04:27:19.516490+0000","last_unstale":"2024-06-06T04:29:05.906141+0000","last_undegraded":"2024-06-06T04:29:05.906141+0000","last_fullsized":"2024-06-06T04:29:05.906141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:11:59.336250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.906081+0000","last_change":"2024-06-06T04:27:22.571003+0000","last_active":"2024-06-06T04:29:05.906081+0000","last_peered":"2024-06-06T04:29:05.906081+0000","last_clean":"2024-06-06T04:29:05.906081+0000","last_became_active":"2024-06-06T04:27:22.570442+0000","last_became_peered":"2024-06-06T04:27:22.570442+0000","last_unstale":"2024-06-06T04:29:05.906081+0000","last_undegraded":"2024-06-06T04:29:05.906081+0000","last_fullsized":"2024-06-06T04:29:05.906081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:28:26.870429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.164730+0000","last_change":"2024-06-06T04:27:32.624424+0000","last_active":"2024-06-06T04:29:06.164730+0000","last_peered":"2024-06-06T04:29:06.164730+0000","last_clean":"2024-06-06T04:29:06.164730+0000","last_became_active":"2024-06-06T04:27:32.624026+0000","last_became_peered":"2024-06-06T04:27:32.624026+0000","last_unstale":"2024-06-06T04:29:06.164730+0000","last_undegraded":"2024-06-06T04:29:06.164730+0000","last_fullsized":"2024-06-06T04:29:06.164730+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:30:08.814284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790341+0000","last_change":"2024-06-06T04:27:19.301382+0000","last_active":"2024-06-06T04:29:06.790341+0000","last_peered":"2024-06-06T04:29:06.790341+0000","last_clean":"2024-06-06T04:29:06.790341+0000","last_became_active":"2024-06-06T04:27:19.300882+0000","last_became_peered":"2024-06-06T04:27:19.300882+0000","last_unstale":"2024-06-06T04:29:06.790341+0000","last_undegraded":"2024-06-06T04:29:06.790341+0000","last_fullsized":"2024-06-06T04:29:06.790341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:50.514103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905686+0000","last_change":"2024-06-06T04:27:22.570661+0000","last_active":"2024-06-06T04:29:05.905686+0000","last_peered":"2024-06-06T04:29:05.905686+0000","last_clean":"2024-06-06T04:29:05.905686+0000","last_became_active":"2024-06-06T04:27:22.570287+0000","last_became_peered":"2024-06-06T04:27:22.570287+0000","last_unstale":"2024-06-06T04:29:05.905686+0000","last_undegraded":"2024-06-06T04:29:05.905686+0000","last_fullsized":"2024-06-06T04:29:05.905686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:48.790492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.789984+0000","last_change":"2024-06-06T04:27:32.618385+0000","last_active":"2024-06-06T04:29:06.789984+0000","last_peered":"2024-06-06T04:29:06.789984+0000","last_clean":"2024-06-06T04:29:06.789984+0000","last_became_active":"2024-06-06T04:27:32.617956+0000","last_became_peered":"2024-06-06T04:27:32.617956+0000","last_unstale":"2024-06-06T04:29:06.789984+0000","last_undegraded":"2024-06-06T04:29:06.789984+0000","last_fullsized":"2024-06-06T04:29:06.789984+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:26.308657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904541+0000","last_change":"2024-06-06T04:28:05.728633+0000","last_active":"2024-06-06T04:29:05.904541+0000","last_peered":"2024-06-06T04:29:05.904541+0000","last_clean":"2024-06-06T04:29:05.904541+0000","last_became_active":"2024-06-06T04:28:05.722427+0000","last_became_peered":"2024-06-06T04:28:05.722427+0000","last_unstale":"2024-06-06T04:29:05.904541+0000","last_undegraded":"2024-06-06T04:29:05.904541+0000","last_fullsized":"2024-06-06T04:29:05.904541+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:13:22.061798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905842+0000","last_change":"2024-06-06T04:27:22.574950+0000","last_active":"2024-06-06T04:29:05.905842+0000","last_peered":"2024-06-06T04:29:05.905842+0000","last_clean":"2024-06-06T04:29:05.905842+0000","last_became_active":"2024-06-06T04:27:22.574604+0000","last_became_peered":"2024-06-06T04:27:22.574604+0000","last_unstale":"2024-06-06T04:29:05.905842+0000","last_undegraded":"2024-06-06T04:29:05.905842+0000","last_fullsized":"2024-06-06T04:29:05.905842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:17:29.395137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380448+0000","last_change":"2024-06-06T04:27:22.568554+0000","last_active":"2024-06-06T04:30:04.380448+0000","last_peered":"2024-06-06T04:30:04.380448+0000","last_clean":"2024-06-06T04:30:04.380448+0000","last_became_active":"2024-06-06T04:27:22.568196+0000","last_became_peered":"2024-06-06T04:27:22.568196+0000","last_unstale":"2024-06-06T04:30:04.380448+0000","last_undegraded":"2024-06-06T04:30:04.380448+0000","last_fullsized":"2024-06-06T04:30:04.380448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:11.564067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.372439+0000","last_change":"2024-06-06T04:27:22.565472+0000","last_active":"2024-06-06T04:30:04.372439+0000","last_peered":"2024-06-06T04:30:04.372439+0000","last_clean":"2024-06-06T04:30:04.372439+0000","last_became_active":"2024-06-06T04:27:22.565116+0000","last_became_peered":"2024-06-06T04:27:22.565116+0000","last_unstale":"2024-06-06T04:30:04.372439+0000","last_undegraded":"2024-06-06T04:30:04.372439+0000","last_fullsized":"2024-06-06T04:30:04.372439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:03:58.232707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.377900+0000","last_change":"2024-06-06T04:27:22.560210+0000","last_active":"2024-06-06T04:30:04.377900+0000","last_peered":"2024-06-06T04:30:04.377900+0000","last_clean":"2024-06-06T04:30:04.377900+0000","last_became_active":"2024-06-06T04:27:22.559853+0000","last_became_peered":"2024-06-06T04:27:22.559853+0000","last_unstale":"2024-06-06T04:30:04.377900+0000","last_undegraded":"2024-06-06T04:30:04.377900+0000","last_fullsized":"2024-06-06T04:30:04.377900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:56:18.650427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385809+0000","last_change":"2024-06-06T04:27:22.565071+0000","last_active":"2024-06-06T04:30:04.385809+0000","last_peered":"2024-06-06T04:30:04.385809+0000","last_clean":"2024-06-06T04:30:04.385809+0000","last_became_active":"2024-06-06T04:27:22.561411+0000","last_became_peered":"2024-06-06T04:27:22.561411+0000","last_unstale":"2024-06-06T04:30:04.385809+0000","last_undegraded":"2024-06-06T04:30:04.385809+0000","last_fullsized":"2024-06-06T04:30:04.385809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:41.948481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904289+0000","last_change":"2024-06-06T04:27:22.563280+0000","last_active":"2024-06-06T04:29:05.904289+0000","last_peered":"2024-06-06T04:29:05.904289+0000","last_clean":"2024-06-06T04:29:05.904289+0000","last_became_active":"2024-06-06T04:27:22.563049+0000","last_became_peered":"2024-06-06T04:27:22.563049+0000","last_unstale":"2024-06-06T04:29:05.904289+0000","last_undegraded":"2024-06-06T04:29:05.904289+0000","last_fullsized":"2024-06-06T04:29:05.904289+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:41:48.287969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.377210+0000","last_change":"2024-06-06T04:27:22.567691+0000","last_active":"2024-06-06T04:30:04.377210+0000","last_peered":"2024-06-06T04:30:04.377210+0000","last_clean":"2024-06-06T04:30:04.377210+0000","last_became_active":"2024-06-06T04:27:22.567331+0000","last_became_peered":"2024-06-06T04:27:22.567331+0000","last_unstale":"2024-06-06T04:30:04.377210+0000","last_undegraded":"2024-06-06T04:30:04.377210+0000","last_fullsized":"2024-06-06T04:30:04.377210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:03.886084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.393172+0000","last_change":"2024-06-06T04:27:22.563853+0000","last_active":"2024-06-06T04:30:04.393172+0000","last_peered":"2024-06-06T04:30:04.393172+0000","last_clean":"2024-06-06T04:30:04.393172+0000","last_became_active":"2024-06-06T04:27:22.563436+0000","last_became_peered":"2024-06-06T04:27:22.563436+0000","last_unstale":"2024-06-06T04:30:04.393172+0000","last_undegraded":"2024-06-06T04:30:04.393172+0000","last_fullsized":"2024-06-06T04:30:04.393172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:26.201308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.387060+0000","last_change":"2024-06-06T04:27:22.568189+0000","last_active":"2024-06-06T04:30:04.387060+0000","last_peered":"2024-06-06T04:30:04.387060+0000","last_clean":"2024-06-06T04:30:04.387060+0000","last_became_active":"2024-06-06T04:27:22.567815+0000","last_became_peered":"2024-06-06T04:27:22.567815+0000","last_unstale":"2024-06-06T04:30:04.387060+0000","last_undegraded":"2024-06-06T04:30:04.387060+0000","last_fullsized":"2024-06-06T04:30:04.387060+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:13:39.259012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.359906+0000","last_change":"2024-06-06T04:27:22.569698+0000","last_active":"2024-06-06T04:30:04.359906+0000","last_peered":"2024-06-06T04:30:04.359906+0000","last_clean":"2024-06-06T04:30:04.359906+0000","last_became_active":"2024-06-06T04:27:22.569347+0000","last_became_peered":"2024-06-06T04:27:22.569347+0000","last_unstale":"2024-06-06T04:30:04.359906+0000","last_undegraded":"2024-06-06T04:30:04.359906+0000","last_fullsized":"2024-06-06T04:30:04.359906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:06:34.722798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381456+0000","last_change":"2024-06-06T04:27:22.561149+0000","last_active":"2024-06-06T04:30:04.381456+0000","last_peered":"2024-06-06T04:30:04.381456+0000","last_clean":"2024-06-06T04:30:04.381456+0000","last_became_active":"2024-06-06T04:27:22.560793+0000","last_became_peered":"2024-06-06T04:27:22.560793+0000","last_unstale":"2024-06-06T04:30:04.381456+0000","last_undegraded":"2024-06-06T04:30:04.381456+0000","last_fullsized":"2024-06-06T04:30:04.381456+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:00:41.563443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.907434+0000","last_change":"2024-06-06T04:27:22.569331+0000","last_active":"2024-06-06T04:29:06.907434+0000","last_peered":"2024-06-06T04:29:06.907434+0000","last_clean":"2024-06-06T04:29:06.907434+0000","last_became_active":"2024-06-06T04:27:22.569021+0000","last_became_peered":"2024-06-06T04:27:22.569021+0000","last_unstale":"2024-06-06T04:29:06.907434+0000","last_undegraded":"2024-06-06T04:29:06.907434+0000","last_fullsized":"2024-06-06T04:29:06.907434+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:52:37.064317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904265+0000","last_change":"2024-06-06T04:27:22.574558+0000","last_active":"2024-06-06T04:29:05.904265+0000","last_peered":"2024-06-06T04:29:05.904265+0000","last_clean":"2024-06-06T04:29:05.904265+0000","last_became_active":"2024-06-06T04:27:22.574215+0000","last_became_peered":"2024-06-06T04:27:22.574215+0000","last_unstale":"2024-06-06T04:29:05.904265+0000","last_undegraded":"2024-06-06T04:29:05.904265+0000","last_fullsized":"2024-06-06T04:29:05.904265+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:40:52.496322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385148+0000","last_change":"2024-06-06T04:27:22.565320+0000","last_active":"2024-06-06T04:30:04.385148+0000","last_peered":"2024-06-06T04:30:04.385148+0000","last_clean":"2024-06-06T04:30:04.385148+0000","last_became_active":"2024-06-06T04:27:22.564786+0000","last_became_peered":"2024-06-06T04:27:22.564786+0000","last_unstale":"2024-06-06T04:30:04.385148+0000","last_undegraded":"2024-06-06T04:30:04.385148+0000","last_fullsized":"2024-06-06T04:30:04.385148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:19:42.057666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":333,"ondisk_log_size":333,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249851,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69006,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":610,"num_read_kb":377,"num_write":175,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":206628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351026,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":572909,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612663,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28236,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716916,"statfs":{"total":95995035648,"available":95966121984,"internally_reserved":0,"allocated":1249280,"data_stored":1225573,"data_compressed":68587,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776188,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":577241,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037825,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573065,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299461,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1167360,"data_stored":1168973,"data_compressed":68587,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462987,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1097728,"data_stored":568321,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1167360,"data_stored":632847,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":569709,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":634446,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149786,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":568515,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888149,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":573502,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724624,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1175552,"data_stored":1164171,"data_compressed":68587,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34523,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34523,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T04:30:09.545 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T04:30:09.798 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:09.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:09 smithi086 ceph-mon[36053]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-06T04:30:09.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:09 smithi086 ceph-mon[36053]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:30:09.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:09 smithi045 ceph-mon[32299]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-06T04:30:09.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:09 smithi045 ceph-mon[32299]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:30:09.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:09 smithi154 ceph-mon[36428]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-06T04:30:09.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:09 smithi154 ceph-mon[36428]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:30:11.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:11 smithi045 ceph-mon[32299]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-06T04:30:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.654+0000 7f02d0ced700 1 -- 172.21.15.45:0/1070861448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02cc06b580 msgr2=0x7f02cc10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.654+0000 7f02d0ced700 1 --2- 172.21.15.45:0/1070861448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02cc06b580 0x7f02cc10f320 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f02c400b0a0 tx=0x7f02c400b3b0 comp rx=0 tx=0).stop 2024-06-06T04:30:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.654+0000 7f02d0ced700 1 -- 172.21.15.45:0/1070861448 shutdown_connections 2024-06-06T04:30:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.654+0000 7f02d0ced700 1 --2- 172.21.15.45:0/1070861448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02cc06b580 0x7f02cc10f320 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.654+0000 7f02d0ced700 1 --2- 172.21.15.45:0/1070861448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02cc108500 0x7f02cc1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.654+0000 7f02d0ced700 1 --2- 172.21.15.45:0/1070861448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02cc06abc0 0x7f02cc06b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.654+0000 7f02d0ced700 1 -- 172.21.15.45:0/1070861448 >> 172.21.15.45:0/1070861448 conn(0x7f02cc078340 msgr2=0x7f02cc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.654+0000 7f02d0ced700 1 -- 172.21.15.45:0/1070861448 shutdown_connections 2024-06-06T04:30:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.654+0000 7f02d0ced700 1 -- 172.21.15.45:0/1070861448 wait complete. 2024-06-06T04:30:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.655+0000 7f02d0ced700 1 Processor -- start 2024-06-06T04:30:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.655+0000 7f02d0ced700 1 -- start start 2024-06-06T04:30:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.655+0000 7f02d0ced700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02cc06abc0 0x7f02cc19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.655+0000 7f02d0ced700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02cc06b580 0x7f02cc19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.655+0000 7f02d0ced700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02cc108500 0x7f02cc19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.655+0000 7f02d0ced700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc112690 con 0x7f02cc06abc0 2024-06-06T04:30:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.655+0000 7f02d0ced700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc112510 con 0x7f02cc06b580 2024-06-06T04:30:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.655+0000 7f02d0ced700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc112810 con 0x7f02cc108500 2024-06-06T04:30:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.655+0000 7f02ca59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02cc06abc0 0x7f02cc19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.655+0000 7f02ca59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02cc06abc0 0x7f02cc19add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54176/0 (socket says 172.21.15.45:54176) 2024-06-06T04:30:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.655+0000 7f02ca59c700 1 -- 172.21.15.45:0/4098790141 learned_addr learned my addr 172.21.15.45:0/4098790141 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.655+0000 7f02cad9d700 1 --2- 172.21.15.45:0/4098790141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02cc108500 0x7f02cc19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.655+0000 7f02c9d9b700 1 --2- 172.21.15.45:0/4098790141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02cc06b580 0x7f02cc19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.655+0000 7f02ca59c700 1 -- 172.21.15.45:0/4098790141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02cc108500 msgr2=0x7f02cc19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.656+0000 7f02ca59c700 1 --2- 172.21.15.45:0/4098790141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02cc108500 0x7f02cc19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.656+0000 7f02ca59c700 1 -- 172.21.15.45:0/4098790141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02cc06b580 msgr2=0x7f02cc19b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.656+0000 7f02ca59c700 1 --2- 172.21.15.45:0/4098790141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02cc06b580 0x7f02cc19b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.656+0000 7f02ca59c700 1 -- 172.21.15.45:0/4098790141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02cc19fde0 con 0x7f02cc06abc0 2024-06-06T04:30:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.656+0000 7f02cad9d700 1 --2- 172.21.15.45:0/4098790141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02cc108500 0x7f02cc19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:30:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.656+0000 7f02ca59c700 1 --2- 172.21.15.45:0/4098790141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02cc06abc0 0x7f02cc19add0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00d950 tx=0x7f02bc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.656+0000 7f02bb7fe700 1 -- 172.21.15.45:0/4098790141 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc016070 con 0x7f02cc06abc0 2024-06-06T04:30:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.656+0000 7f02bb7fe700 1 -- 172.21.15.45:0/4098790141 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02bc010460 con 0x7f02cc06abc0 2024-06-06T04:30:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.656+0000 7f02bb7fe700 1 -- 172.21.15.45:0/4098790141 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc00f620 con 0x7f02cc06abc0 2024-06-06T04:30:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.656+0000 7f02d0ced700 1 -- 172.21.15.45:0/4098790141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02cc1a00d0 con 0x7f02cc06abc0 2024-06-06T04:30:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.656+0000 7f02d0ced700 1 -- 172.21.15.45:0/4098790141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02cc1a7950 con 0x7f02cc06abc0 2024-06-06T04:30:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.657+0000 7f02d0ced700 1 -- 172.21.15.45:0/4098790141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02cc069180 con 0x7f02cc06abc0 2024-06-06T04:30:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.658+0000 7f02bb7fe700 1 -- 172.21.15.45:0/4098790141 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f02bc019070 con 0x7f02cc06abc0 2024-06-06T04:30:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.659+0000 7f02bb7fe700 1 --2- 172.21.15.45:0/4098790141 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f02b40a2c20 0x7f02b40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.659+0000 7f02c9d9b700 1 --2- 172.21.15.45:0/4098790141 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f02b40a2c20 0x7f02b40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.659+0000 7f02bb7fe700 1 -- 172.21.15.45:0/4098790141 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f02bc0cf0e0 con 0x7f02cc06abc0 2024-06-06T04:30:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.659+0000 7f02c9d9b700 1 --2- 172.21.15.45:0/4098790141 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f02b40a2c20 0x7f02b40a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f02c0006fd0 tx=0x7f02c0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.663+0000 7f02bb7fe700 1 -- 172.21.15.45:0/4098790141 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f02bc09bc20 con 0x7f02cc06abc0 2024-06-06T04:30:11.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:11 smithi086 ceph-mon[36053]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-06T04:30:11.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.863+0000 7f02d0ced700 1 -- 172.21.15.45:0/4098790141 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02cc10dd60 con 0x7f02b40a2c20 2024-06-06T04:30:11.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.873+0000 7f02bb7fe700 1 -- 172.21.15.45:0/4098790141 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+341063 (secure 0 0 0) 0x7f02cc10dd60 con 0x7f02b40a2c20 2024-06-06T04:30:11.877 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:30:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.876+0000 7f02d0ced700 1 -- 172.21.15.45:0/4098790141 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f02b40a2c20 msgr2=0x7f02b40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.876+0000 7f02d0ced700 1 --2- 172.21.15.45:0/4098790141 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f02b40a2c20 0x7f02b40a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f02c0006fd0 tx=0x7f02c0008040 comp rx=0 tx=0).stop 2024-06-06T04:30:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.876+0000 7f02d0ced700 1 -- 172.21.15.45:0/4098790141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02cc06abc0 msgr2=0x7f02cc19add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.876+0000 7f02d0ced700 1 --2- 172.21.15.45:0/4098790141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02cc06abc0 0x7f02cc19add0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00d950 tx=0x7f02bc00dd10 comp rx=0 tx=0).stop 2024-06-06T04:30:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.876+0000 7f02d0ced700 1 -- 172.21.15.45:0/4098790141 shutdown_connections 2024-06-06T04:30:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.876+0000 7f02d0ced700 1 --2- 172.21.15.45:0/4098790141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02cc108500 0x7f02cc19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.876+0000 7f02d0ced700 1 --2- 172.21.15.45:0/4098790141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02cc06abc0 0x7f02cc19add0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.876+0000 7f02d0ced700 1 --2- 172.21.15.45:0/4098790141 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f02b40a2c20 0x7f02b40a50e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.876+0000 7f02d0ced700 1 --2- 172.21.15.45:0/4098790141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02cc06b580 0x7f02cc19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.876+0000 7f02d0ced700 1 -- 172.21.15.45:0/4098790141 >> 172.21.15.45:0/4098790141 conn(0x7f02cc078340 msgr2=0x7f02cc10d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:11.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.876+0000 7f02d0ced700 1 -- 172.21.15.45:0/4098790141 shutdown_connections 2024-06-06T04:30:11.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:11.876+0000 7f02d0ced700 1 -- 172.21.15.45:0/4098790141 wait complete. 2024-06-06T04:30:11.881 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-06-06T04:30:11.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:11 smithi154 ceph-mon[36428]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-06T04:30:12.545 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:12 smithi045 ceph-mon[32299]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:30:12.577 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-06-06T04:30:10.403762+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659608,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":741,"num_read_kb":491,"num_write":359,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375044,"kb_used_data":13412,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566780,"statfs":{"total":1151940427776,"available":1151556382720,"internally_reserved":0,"allocated":13733888,"data_stored":8829662,"data_compressed":497595,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":37,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003390"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.386822+0000","last_change":"2024-06-06T04:27:22.574900+0000","last_active":"2024-06-06T04:30:04.386822+0000","last_peered":"2024-06-06T04:30:04.386822+0000","last_clean":"2024-06-06T04:30:04.386822+0000","last_became_active":"2024-06-06T04:27:22.574389+0000","last_became_peered":"2024-06-06T04:27:22.574389+0000","last_unstale":"2024-06-06T04:30:04.386822+0000","last_undegraded":"2024-06-06T04:30:04.386822+0000","last_fullsized":"2024-06-06T04:30:04.386822+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:39.246463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.382712+0000","last_change":"2024-06-06T04:27:22.565543+0000","last_active":"2024-06-06T04:30:04.382712+0000","last_peered":"2024-06-06T04:30:04.382712+0000","last_clean":"2024-06-06T04:30:04.382712+0000","last_became_active":"2024-06-06T04:27:22.563833+0000","last_became_peered":"2024-06-06T04:27:22.563833+0000","last_unstale":"2024-06-06T04:30:04.382712+0000","last_undegraded":"2024-06-06T04:30:04.382712+0000","last_fullsized":"2024-06-06T04:30:04.382712+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:26:17.810782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388761+0000","last_change":"2024-06-06T04:27:22.570107+0000","last_active":"2024-06-06T04:30:04.388761+0000","last_peered":"2024-06-06T04:30:04.388761+0000","last_clean":"2024-06-06T04:30:04.388761+0000","last_became_active":"2024-06-06T04:27:22.569788+0000","last_became_peered":"2024-06-06T04:27:22.569788+0000","last_unstale":"2024-06-06T04:30:04.388761+0000","last_undegraded":"2024-06-06T04:30:04.388761+0000","last_fullsized":"2024-06-06T04:30:04.388761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:48:20.454782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385114+0000","last_change":"2024-06-06T04:27:22.559431+0000","last_active":"2024-06-06T04:30:04.385114+0000","last_peered":"2024-06-06T04:30:04.385114+0000","last_clean":"2024-06-06T04:30:04.385114+0000","last_became_active":"2024-06-06T04:27:22.558961+0000","last_became_peered":"2024-06-06T04:27:22.558961+0000","last_unstale":"2024-06-06T04:30:04.385114+0000","last_undegraded":"2024-06-06T04:30:04.385114+0000","last_fullsized":"2024-06-06T04:30:04.385114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:59.445283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.359959+0000","last_change":"2024-06-06T04:27:22.561598+0000","last_active":"2024-06-06T04:30:04.359959+0000","last_peered":"2024-06-06T04:30:04.359959+0000","last_clean":"2024-06-06T04:30:04.359959+0000","last_became_active":"2024-06-06T04:27:22.561198+0000","last_became_peered":"2024-06-06T04:27:22.561198+0000","last_unstale":"2024-06-06T04:30:04.359959+0000","last_undegraded":"2024-06-06T04:30:04.359959+0000","last_fullsized":"2024-06-06T04:30:04.359959+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:20:28.314941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380904+0000","last_change":"2024-06-06T04:27:22.569395+0000","last_active":"2024-06-06T04:30:04.380904+0000","last_peered":"2024-06-06T04:30:04.380904+0000","last_clean":"2024-06-06T04:30:04.380904+0000","last_became_active":"2024-06-06T04:27:22.569054+0000","last_became_peered":"2024-06-06T04:27:22.569054+0000","last_unstale":"2024-06-06T04:30:04.380904+0000","last_undegraded":"2024-06-06T04:30:04.380904+0000","last_fullsized":"2024-06-06T04:30:04.380904+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:03:56.630677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392227+0000","last_change":"2024-06-06T04:27:22.577121+0000","last_active":"2024-06-06T04:30:04.392227+0000","last_peered":"2024-06-06T04:30:04.392227+0000","last_clean":"2024-06-06T04:30:04.392227+0000","last_became_active":"2024-06-06T04:27:22.575574+0000","last_became_peered":"2024-06-06T04:27:22.575574+0000","last_unstale":"2024-06-06T04:30:04.392227+0000","last_undegraded":"2024-06-06T04:30:04.392227+0000","last_fullsized":"2024-06-06T04:30:04.392227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:05:16.799335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.383226+0000","last_change":"2024-06-06T04:27:22.570051+0000","last_active":"2024-06-06T04:30:04.383226+0000","last_peered":"2024-06-06T04:30:04.383226+0000","last_clean":"2024-06-06T04:30:04.383226+0000","last_became_active":"2024-06-06T04:27:22.569640+0000","last_became_peered":"2024-06-06T04:27:22.569640+0000","last_unstale":"2024-06-06T04:30:04.383226+0000","last_undegraded":"2024-06-06T04:30:04.383226+0000","last_fullsized":"2024-06-06T04:30:04.383226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:48:00.744167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.396353+0000","last_change":"2024-06-06T04:27:22.574900+0000","last_active":"2024-06-06T04:30:04.396353+0000","last_peered":"2024-06-06T04:30:04.396353+0000","last_clean":"2024-06-06T04:30:04.396353+0000","last_became_active":"2024-06-06T04:27:22.574570+0000","last_became_peered":"2024-06-06T04:27:22.574570+0000","last_unstale":"2024-06-06T04:30:04.396353+0000","last_undegraded":"2024-06-06T04:30:04.396353+0000","last_fullsized":"2024-06-06T04:30:04.396353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:54.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380728+0000","last_change":"2024-06-06T04:27:22.570581+0000","last_active":"2024-06-06T04:30:04.380728+0000","last_peered":"2024-06-06T04:30:04.380728+0000","last_clean":"2024-06-06T04:30:04.380728+0000","last_became_active":"2024-06-06T04:27:22.570243+0000","last_became_peered":"2024-06-06T04:27:22.570243+0000","last_unstale":"2024-06-06T04:30:04.380728+0000","last_undegraded":"2024-06-06T04:30:04.380728+0000","last_fullsized":"2024-06-06T04:30:04.380728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:27.087527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.395227+0000","last_change":"2024-06-06T04:27:22.571162+0000","last_active":"2024-06-06T04:30:04.395227+0000","last_peered":"2024-06-06T04:30:04.395227+0000","last_clean":"2024-06-06T04:30:04.395227+0000","last_became_active":"2024-06-06T04:27:22.570843+0000","last_became_peered":"2024-06-06T04:27:22.570843+0000","last_unstale":"2024-06-06T04:30:04.395227+0000","last_undegraded":"2024-06-06T04:30:04.395227+0000","last_fullsized":"2024-06-06T04:30:04.395227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:43:05.590240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.384953+0000","last_change":"2024-06-06T04:27:22.567420+0000","last_active":"2024-06-06T04:30:04.384953+0000","last_peered":"2024-06-06T04:30:04.384953+0000","last_clean":"2024-06-06T04:30:04.384953+0000","last_became_active":"2024-06-06T04:27:22.566750+0000","last_became_peered":"2024-06-06T04:27:22.566750+0000","last_unstale":"2024-06-06T04:30:04.384953+0000","last_undegraded":"2024-06-06T04:30:04.384953+0000","last_fullsized":"2024-06-06T04:30:04.384953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:21.669965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.364827+0000","last_change":"2024-06-06T04:27:22.566831+0000","last_active":"2024-06-06T04:30:04.364827+0000","last_peered":"2024-06-06T04:30:04.364827+0000","last_clean":"2024-06-06T04:30:04.364827+0000","last_became_active":"2024-06-06T04:27:22.566255+0000","last_became_peered":"2024-06-06T04:27:22.566255+0000","last_unstale":"2024-06-06T04:30:04.364827+0000","last_undegraded":"2024-06-06T04:30:04.364827+0000","last_fullsized":"2024-06-06T04:30:04.364827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:40:22.486674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.384669+0000","last_change":"2024-06-06T04:27:22.564596+0000","last_active":"2024-06-06T04:30:04.384669+0000","last_peered":"2024-06-06T04:30:04.384669+0000","last_clean":"2024-06-06T04:30:04.384669+0000","last_became_active":"2024-06-06T04:27:22.563694+0000","last_became_peered":"2024-06-06T04:27:22.563694+0000","last_unstale":"2024-06-06T04:30:04.384669+0000","last_undegraded":"2024-06-06T04:30:04.384669+0000","last_fullsized":"2024-06-06T04:30:04.384669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:18.568266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385298+0000","last_change":"2024-06-06T04:27:22.559395+0000","last_active":"2024-06-06T04:30:04.385298+0000","last_peered":"2024-06-06T04:30:04.385298+0000","last_clean":"2024-06-06T04:30:04.385298+0000","last_became_active":"2024-06-06T04:27:22.559030+0000","last_became_peered":"2024-06-06T04:27:22.559030+0000","last_unstale":"2024-06-06T04:30:04.385298+0000","last_undegraded":"2024-06-06T04:30:04.385298+0000","last_fullsized":"2024-06-06T04:30:04.385298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:08.562184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394517+0000","last_change":"2024-06-06T04:27:22.569657+0000","last_active":"2024-06-06T04:30:04.394517+0000","last_peered":"2024-06-06T04:30:04.394517+0000","last_clean":"2024-06-06T04:30:04.394517+0000","last_became_active":"2024-06-06T04:27:22.569359+0000","last_became_peered":"2024-06-06T04:27:22.569359+0000","last_unstale":"2024-06-06T04:30:04.394517+0000","last_undegraded":"2024-06-06T04:30:04.394517+0000","last_fullsized":"2024-06-06T04:30:04.394517+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:42:20.197715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394347+0000","last_change":"2024-06-06T04:27:22.574661+0000","last_active":"2024-06-06T04:30:04.394347+0000","last_peered":"2024-06-06T04:30:04.394347+0000","last_clean":"2024-06-06T04:30:04.394347+0000","last_became_active":"2024-06-06T04:27:22.574310+0000","last_became_peered":"2024-06-06T04:27:22.574310+0000","last_unstale":"2024-06-06T04:30:04.394347+0000","last_undegraded":"2024-06-06T04:30:04.394347+0000","last_fullsized":"2024-06-06T04:30:04.394347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:45:18.772270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388153+0000","last_change":"2024-06-06T04:27:22.565575+0000","last_active":"2024-06-06T04:30:04.388153+0000","last_peered":"2024-06-06T04:30:04.388153+0000","last_clean":"2024-06-06T04:30:04.388153+0000","last_became_active":"2024-06-06T04:27:22.565256+0000","last_became_peered":"2024-06-06T04:27:22.565256+0000","last_unstale":"2024-06-06T04:30:04.388153+0000","last_undegraded":"2024-06-06T04:30:04.388153+0000","last_fullsized":"2024-06-06T04:30:04.388153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:49.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.365082+0000","last_change":"2024-06-06T04:27:22.559606+0000","last_active":"2024-06-06T04:30:04.365082+0000","last_peered":"2024-06-06T04:30:04.365082+0000","last_clean":"2024-06-06T04:30:04.365082+0000","last_became_active":"2024-06-06T04:27:22.557881+0000","last_became_peered":"2024-06-06T04:27:22.557881+0000","last_unstale":"2024-06-06T04:30:04.365082+0000","last_undegraded":"2024-06-06T04:30:04.365082+0000","last_fullsized":"2024-06-06T04:30:04.365082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:31:06.269785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905476+0000","last_change":"2024-06-06T04:27:32.626612+0000","last_active":"2024-06-06T04:29:05.905476+0000","last_peered":"2024-06-06T04:29:05.905476+0000","last_clean":"2024-06-06T04:29:05.905476+0000","last_became_active":"2024-06-06T04:27:32.626113+0000","last_became_peered":"2024-06-06T04:27:32.626113+0000","last_unstale":"2024-06-06T04:29:05.905476+0000","last_undegraded":"2024-06-06T04:29:05.905476+0000","last_fullsized":"2024-06-06T04:29:05.905476+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.318738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908330+0000","last_change":"2024-06-06T04:27:19.302744+0000","last_active":"2024-06-06T04:29:05.908330+0000","last_peered":"2024-06-06T04:29:05.908330+0000","last_clean":"2024-06-06T04:29:05.908330+0000","last_became_active":"2024-06-06T04:27:19.302389+0000","last_became_peered":"2024-06-06T04:27:19.302389+0000","last_unstale":"2024-06-06T04:29:05.908330+0000","last_undegraded":"2024-06-06T04:29:05.908330+0000","last_fullsized":"2024-06-06T04:29:05.908330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:49.872618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380678+0000","last_change":"2024-06-06T04:27:22.570220+0000","last_active":"2024-06-06T04:30:04.380678+0000","last_peered":"2024-06-06T04:30:04.380678+0000","last_clean":"2024-06-06T04:30:04.380678+0000","last_became_active":"2024-06-06T04:27:22.569835+0000","last_became_peered":"2024-06-06T04:27:22.569835+0000","last_unstale":"2024-06-06T04:30:04.380678+0000","last_undegraded":"2024-06-06T04:30:04.380678+0000","last_fullsized":"2024-06-06T04:30:04.380678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:54:57.372561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401940+0000","last_change":"2024-06-06T04:27:32.622407+0000","last_active":"2024-06-06T04:29:06.401940+0000","last_peered":"2024-06-06T04:29:06.401940+0000","last_clean":"2024-06-06T04:29:06.401940+0000","last_became_active":"2024-06-06T04:27:32.621868+0000","last_became_peered":"2024-06-06T04:27:32.621868+0000","last_unstale":"2024-06-06T04:29:06.401940+0000","last_undegraded":"2024-06-06T04:29:06.401940+0000","last_fullsized":"2024-06-06T04:29:06.401940+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:50:05.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560827+0000","last_change":"2024-06-06T04:27:19.516235+0000","last_active":"2024-06-06T04:29:06.560827+0000","last_peered":"2024-06-06T04:29:06.560827+0000","last_clean":"2024-06-06T04:29:06.560827+0000","last_became_active":"2024-06-06T04:27:19.515849+0000","last_became_peered":"2024-06-06T04:27:19.515849+0000","last_unstale":"2024-06-06T04:29:06.560827+0000","last_undegraded":"2024-06-06T04:29:06.560827+0000","last_fullsized":"2024-06-06T04:29:06.560827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:52:42.722103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388234+0000","last_change":"2024-06-06T04:27:22.570797+0000","last_active":"2024-06-06T04:30:04.388234+0000","last_peered":"2024-06-06T04:30:04.388234+0000","last_clean":"2024-06-06T04:30:04.388234+0000","last_became_active":"2024-06-06T04:27:22.570305+0000","last_became_peered":"2024-06-06T04:27:22.570305+0000","last_unstale":"2024-06-06T04:30:04.388234+0000","last_undegraded":"2024-06-06T04:30:04.388234+0000","last_fullsized":"2024-06-06T04:30:04.388234+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:04:14.545306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401666+0000","last_change":"2024-06-06T04:27:32.623241+0000","last_active":"2024-06-06T04:29:06.401666+0000","last_peered":"2024-06-06T04:29:06.401666+0000","last_clean":"2024-06-06T04:29:06.401666+0000","last_became_active":"2024-06-06T04:27:32.622751+0000","last_became_peered":"2024-06-06T04:27:32.622751+0000","last_unstale":"2024-06-06T04:29:06.401666+0000","last_undegraded":"2024-06-06T04:29:06.401666+0000","last_fullsized":"2024-06-06T04:29:06.401666+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:24:53.446329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.909054+0000","last_change":"2024-06-06T04:28:05.733350+0000","last_active":"2024-06-06T04:29:05.909054+0000","last_peered":"2024-06-06T04:29:05.909054+0000","last_clean":"2024-06-06T04:29:05.909054+0000","last_became_active":"2024-06-06T04:28:05.726556+0000","last_became_peered":"2024-06-06T04:28:05.726556+0000","last_unstale":"2024-06-06T04:29:05.909054+0000","last_undegraded":"2024-06-06T04:29:05.909054+0000","last_fullsized":"2024-06-06T04:29:05.909054+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:26.636625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.396094+0000","last_change":"2024-06-06T04:27:22.571731+0000","last_active":"2024-06-06T04:30:04.396094+0000","last_peered":"2024-06-06T04:30:04.396094+0000","last_clean":"2024-06-06T04:30:04.396094+0000","last_became_active":"2024-06-06T04:27:22.571398+0000","last_became_peered":"2024-06-06T04:27:22.571398+0000","last_unstale":"2024-06-06T04:30:04.396094+0000","last_undegraded":"2024-06-06T04:30:04.396094+0000","last_fullsized":"2024-06-06T04:30:04.396094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:22:45.144427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401661+0000","last_change":"2024-06-06T04:27:32.618465+0000","last_active":"2024-06-06T04:29:06.401661+0000","last_peered":"2024-06-06T04:29:06.401661+0000","last_clean":"2024-06-06T04:29:06.401661+0000","last_became_active":"2024-06-06T04:27:32.618066+0000","last_became_peered":"2024-06-06T04:27:32.618066+0000","last_unstale":"2024-06-06T04:29:06.401661+0000","last_undegraded":"2024-06-06T04:29:06.401661+0000","last_fullsized":"2024-06-06T04:29:06.401661+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:54:23.988687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908662+0000","last_change":"2024-06-06T04:28:05.730381+0000","last_active":"2024-06-06T04:29:05.908662+0000","last_peered":"2024-06-06T04:29:05.908662+0000","last_clean":"2024-06-06T04:29:05.908662+0000","last_became_active":"2024-06-06T04:28:05.726552+0000","last_became_peered":"2024-06-06T04:28:05.726552+0000","last_unstale":"2024-06-06T04:29:05.908662+0000","last_undegraded":"2024-06-06T04:29:05.908662+0000","last_fullsized":"2024-06-06T04:29:05.908662+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:52.903042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394789+0000","last_change":"2024-06-06T04:27:22.573769+0000","last_active":"2024-06-06T04:30:04.394789+0000","last_peered":"2024-06-06T04:30:04.394789+0000","last_clean":"2024-06-06T04:30:04.394789+0000","last_became_active":"2024-06-06T04:27:22.572674+0000","last_became_peered":"2024-06-06T04:27:22.572674+0000","last_unstale":"2024-06-06T04:30:04.394789+0000","last_undegraded":"2024-06-06T04:30:04.394789+0000","last_fullsized":"2024-06-06T04:30:04.394789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:57:22.653052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905675+0000","last_change":"2024-06-06T04:27:32.617934+0000","last_active":"2024-06-06T04:29:05.905675+0000","last_peered":"2024-06-06T04:29:05.905675+0000","last_clean":"2024-06-06T04:29:05.905675+0000","last_became_active":"2024-06-06T04:27:32.617537+0000","last_became_peered":"2024-06-06T04:27:32.617537+0000","last_unstale":"2024-06-06T04:29:05.905675+0000","last_undegraded":"2024-06-06T04:29:05.905675+0000","last_fullsized":"2024-06-06T04:29:05.905675+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:21:48.319434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'5","reported_seq":149,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.864491+0000","last_change":"2024-06-06T04:27:19.306236+0000","last_active":"2024-06-06T04:30:04.864491+0000","last_peered":"2024-06-06T04:30:04.864491+0000","last_clean":"2024-06-06T04:30:04.864491+0000","last_became_active":"2024-06-06T04:27:19.305850+0000","last_became_peered":"2024-06-06T04:27:19.305850+0000","last_unstale":"2024-06-06T04:30:04.864491+0000","last_undegraded":"2024-06-06T04:30:04.864491+0000","last_fullsized":"2024-06-06T04:30:04.864491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:58:47.310040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.365431+0000","last_change":"2024-06-06T04:27:22.567440+0000","last_active":"2024-06-06T04:30:04.365431+0000","last_peered":"2024-06-06T04:30:04.365431+0000","last_clean":"2024-06-06T04:30:04.365431+0000","last_became_active":"2024-06-06T04:27:22.566732+0000","last_became_peered":"2024-06-06T04:27:22.566732+0000","last_unstale":"2024-06-06T04:30:04.365431+0000","last_undegraded":"2024-06-06T04:30:04.365431+0000","last_fullsized":"2024-06-06T04:30:04.365431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:48:46.270103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905710+0000","last_change":"2024-06-06T04:27:32.626554+0000","last_active":"2024-06-06T04:29:05.905710+0000","last_peered":"2024-06-06T04:29:05.905710+0000","last_clean":"2024-06-06T04:29:05.905710+0000","last_became_active":"2024-06-06T04:27:32.626108+0000","last_became_peered":"2024-06-06T04:27:32.626108+0000","last_unstale":"2024-06-06T04:29:05.905710+0000","last_undegraded":"2024-06-06T04:29:05.905710+0000","last_fullsized":"2024-06-06T04:29:05.905710+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:37:43.811005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.262695+0000","last_change":"2024-06-06T04:27:19.306821+0000","last_active":"2024-06-06T04:29:06.262695+0000","last_peered":"2024-06-06T04:29:06.262695+0000","last_clean":"2024-06-06T04:29:06.262695+0000","last_became_active":"2024-06-06T04:27:19.306489+0000","last_became_peered":"2024-06-06T04:27:19.306489+0000","last_unstale":"2024-06-06T04:29:06.262695+0000","last_undegraded":"2024-06-06T04:29:06.262695+0000","last_fullsized":"2024-06-06T04:29:06.262695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:31:28.973112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.378929+0000","last_change":"2024-06-06T04:27:22.570617+0000","last_active":"2024-06-06T04:30:04.378929+0000","last_peered":"2024-06-06T04:30:04.378929+0000","last_clean":"2024-06-06T04:30:04.378929+0000","last_became_active":"2024-06-06T04:27:22.570271+0000","last_became_peered":"2024-06-06T04:27:22.570271+0000","last_unstale":"2024-06-06T04:30:04.378929+0000","last_undegraded":"2024-06-06T04:30:04.378929+0000","last_fullsized":"2024-06-06T04:30:04.378929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:45:33.890164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.909010+0000","last_change":"2024-06-06T04:27:32.628132+0000","last_active":"2024-06-06T04:29:05.909010+0000","last_peered":"2024-06-06T04:29:05.909010+0000","last_clean":"2024-06-06T04:29:05.909010+0000","last_became_active":"2024-06-06T04:27:32.627658+0000","last_became_peered":"2024-06-06T04:27:32.627658+0000","last_unstale":"2024-06-06T04:29:05.909010+0000","last_undegraded":"2024-06-06T04:29:05.909010+0000","last_fullsized":"2024-06-06T04:29:05.909010+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:24.817079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904125+0000","last_change":"2024-06-06T04:27:19.300510+0000","last_active":"2024-06-06T04:29:05.904125+0000","last_peered":"2024-06-06T04:29:05.904125+0000","last_clean":"2024-06-06T04:29:05.904125+0000","last_became_active":"2024-06-06T04:27:19.300150+0000","last_became_peered":"2024-06-06T04:27:19.300150+0000","last_unstale":"2024-06-06T04:29:05.904125+0000","last_undegraded":"2024-06-06T04:29:05.904125+0000","last_fullsized":"2024-06-06T04:29:05.904125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:38:25.084690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381098+0000","last_change":"2024-06-06T04:27:22.563864+0000","last_active":"2024-06-06T04:30:04.381098+0000","last_peered":"2024-06-06T04:30:04.381098+0000","last_clean":"2024-06-06T04:30:04.381098+0000","last_became_active":"2024-06-06T04:27:22.563555+0000","last_became_peered":"2024-06-06T04:27:22.563555+0000","last_unstale":"2024-06-06T04:30:04.381098+0000","last_undegraded":"2024-06-06T04:30:04.381098+0000","last_fullsized":"2024-06-06T04:30:04.381098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:39:33.707404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.165015+0000","last_change":"2024-06-06T04:27:32.617627+0000","last_active":"2024-06-06T04:29:06.165015+0000","last_peered":"2024-06-06T04:29:06.165015+0000","last_clean":"2024-06-06T04:29:06.165015+0000","last_became_active":"2024-06-06T04:27:32.617187+0000","last_became_peered":"2024-06-06T04:27:32.617187+0000","last_unstale":"2024-06-06T04:29:06.165015+0000","last_undegraded":"2024-06-06T04:29:06.165015+0000","last_fullsized":"2024-06-06T04:29:06.165015+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:24:52.554176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790234+0000","last_change":"2024-06-06T04:27:19.304316+0000","last_active":"2024-06-06T04:29:06.790234+0000","last_peered":"2024-06-06T04:29:06.790234+0000","last_clean":"2024-06-06T04:29:06.790234+0000","last_became_active":"2024-06-06T04:27:19.303952+0000","last_became_peered":"2024-06-06T04:27:19.303952+0000","last_unstale":"2024-06-06T04:29:06.790234+0000","last_undegraded":"2024-06-06T04:29:06.790234+0000","last_fullsized":"2024-06-06T04:29:06.790234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:26.886916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.371820+0000","last_change":"2024-06-06T04:27:22.561824+0000","last_active":"2024-06-06T04:30:04.371820+0000","last_peered":"2024-06-06T04:30:04.371820+0000","last_clean":"2024-06-06T04:30:04.371820+0000","last_became_active":"2024-06-06T04:27:22.561495+0000","last_became_peered":"2024-06-06T04:27:22.561495+0000","last_unstale":"2024-06-06T04:30:04.371820+0000","last_undegraded":"2024-06-06T04:30:04.371820+0000","last_fullsized":"2024-06-06T04:30:04.371820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:40:31.314627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560380+0000","last_change":"2024-06-06T04:27:32.624540+0000","last_active":"2024-06-06T04:29:06.560380+0000","last_peered":"2024-06-06T04:29:06.560380+0000","last_clean":"2024-06-06T04:29:06.560380+0000","last_became_active":"2024-06-06T04:27:32.624036+0000","last_became_peered":"2024-06-06T04:27:32.624036+0000","last_unstale":"2024-06-06T04:29:06.560380+0000","last_undegraded":"2024-06-06T04:29:06.560380+0000","last_fullsized":"2024-06-06T04:29:06.560380+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:16.983576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":143,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.864030+0000","last_change":"2024-06-06T04:27:19.304614+0000","last_active":"2024-06-06T04:30:04.864030+0000","last_peered":"2024-06-06T04:30:04.864030+0000","last_clean":"2024-06-06T04:30:04.864030+0000","last_became_active":"2024-06-06T04:27:19.304126+0000","last_became_peered":"2024-06-06T04:27:19.304126+0000","last_unstale":"2024-06-06T04:30:04.864030+0000","last_undegraded":"2024-06-06T04:30:04.864030+0000","last_fullsized":"2024-06-06T04:30:04.864030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:07:39.908630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.396136+0000","last_change":"2024-06-06T04:27:22.572534+0000","last_active":"2024-06-06T04:30:04.396136+0000","last_peered":"2024-06-06T04:30:04.396136+0000","last_clean":"2024-06-06T04:30:04.396136+0000","last_became_active":"2024-06-06T04:27:22.572214+0000","last_became_peered":"2024-06-06T04:27:22.572214+0000","last_unstale":"2024-06-06T04:30:04.396136+0000","last_undegraded":"2024-06-06T04:30:04.396136+0000","last_fullsized":"2024-06-06T04:30:04.396136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:13.176045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.262294+0000","last_change":"2024-06-06T04:28:05.723450+0000","last_active":"2024-06-06T04:29:06.262294+0000","last_peered":"2024-06-06T04:29:06.262294+0000","last_clean":"2024-06-06T04:29:06.262294+0000","last_became_active":"2024-06-06T04:28:05.722928+0000","last_became_peered":"2024-06-06T04:28:05.722928+0000","last_unstale":"2024-06-06T04:29:06.262294+0000","last_undegraded":"2024-06-06T04:29:06.262294+0000","last_fullsized":"2024-06-06T04:29:06.262294+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:53.093276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905918+0000","last_change":"2024-06-06T04:27:19.308465+0000","last_active":"2024-06-06T04:29:05.905918+0000","last_peered":"2024-06-06T04:29:05.905918+0000","last_clean":"2024-06-06T04:29:05.905918+0000","last_became_active":"2024-06-06T04:27:19.308077+0000","last_became_peered":"2024-06-06T04:27:19.308077+0000","last_unstale":"2024-06-06T04:29:05.905918+0000","last_undegraded":"2024-06-06T04:29:05.905918+0000","last_fullsized":"2024-06-06T04:29:05.905918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:37:48.282613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385739+0000","last_change":"2024-06-06T04:27:22.574141+0000","last_active":"2024-06-06T04:30:04.385739+0000","last_peered":"2024-06-06T04:30:04.385739+0000","last_clean":"2024-06-06T04:30:04.385739+0000","last_became_active":"2024-06-06T04:27:22.573679+0000","last_became_peered":"2024-06-06T04:27:22.573679+0000","last_unstale":"2024-06-06T04:30:04.385739+0000","last_undegraded":"2024-06-06T04:30:04.385739+0000","last_fullsized":"2024-06-06T04:30:04.385739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:27:25.177990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401856+0000","last_change":"2024-06-06T04:27:32.624148+0000","last_active":"2024-06-06T04:29:06.401856+0000","last_peered":"2024-06-06T04:29:06.401856+0000","last_clean":"2024-06-06T04:29:06.401856+0000","last_became_active":"2024-06-06T04:27:32.623448+0000","last_became_peered":"2024-06-06T04:27:32.623448+0000","last_unstale":"2024-06-06T04:29:06.401856+0000","last_undegraded":"2024-06-06T04:29:06.401856+0000","last_fullsized":"2024-06-06T04:29:06.401856+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:02.037296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790253+0000","last_change":"2024-06-06T04:27:19.516875+0000","last_active":"2024-06-06T04:29:06.790253+0000","last_peered":"2024-06-06T04:29:06.790253+0000","last_clean":"2024-06-06T04:29:06.790253+0000","last_became_active":"2024-06-06T04:27:19.516393+0000","last_became_peered":"2024-06-06T04:27:19.516393+0000","last_unstale":"2024-06-06T04:29:06.790253+0000","last_undegraded":"2024-06-06T04:29:06.790253+0000","last_fullsized":"2024-06-06T04:29:06.790253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:20.079218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392596+0000","last_change":"2024-06-06T04:28:05.724043+0000","last_active":"2024-06-06T04:30:04.392596+0000","last_peered":"2024-06-06T04:30:04.392596+0000","last_clean":"2024-06-06T04:30:04.392596+0000","last_became_active":"2024-06-06T04:28:05.723622+0000","last_became_peered":"2024-06-06T04:28:05.723622+0000","last_unstale":"2024-06-06T04:30:04.392596+0000","last_undegraded":"2024-06-06T04:30:04.392596+0000","last_fullsized":"2024-06-06T04:30:04.392596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:42.162400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.907660+0000","last_change":"2024-06-06T04:27:32.626366+0000","last_active":"2024-06-06T04:29:05.907660+0000","last_peered":"2024-06-06T04:29:05.907660+0000","last_clean":"2024-06-06T04:29:05.907660+0000","last_became_active":"2024-06-06T04:27:32.625847+0000","last_became_peered":"2024-06-06T04:27:32.625847+0000","last_unstale":"2024-06-06T04:29:05.907660+0000","last_undegraded":"2024-06-06T04:29:05.907660+0000","last_fullsized":"2024-06-06T04:29:05.907660+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:47:44.167531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'38","reported_seq":285,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:05.865651+0000","last_change":"2024-06-06T04:27:19.303581+0000","last_active":"2024-06-06T04:30:05.865651+0000","last_peered":"2024-06-06T04:30:05.865651+0000","last_clean":"2024-06-06T04:30:05.865651+0000","last_became_active":"2024-06-06T04:27:19.303234+0000","last_became_peered":"2024-06-06T04:27:19.303234+0000","last_unstale":"2024-06-06T04:30:05.865651+0000","last_undegraded":"2024-06-06T04:30:05.865651+0000","last_fullsized":"2024-06-06T04:30:05.865651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:37.721588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64865,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":204,"num_read_kb":166,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.391077+0000","last_change":"2024-06-06T04:27:22.567177+0000","last_active":"2024-06-06T04:30:04.391077+0000","last_peered":"2024-06-06T04:30:04.391077+0000","last_clean":"2024-06-06T04:30:04.391077+0000","last_became_active":"2024-06-06T04:27:22.566751+0000","last_became_peered":"2024-06-06T04:27:22.566751+0000","last_unstale":"2024-06-06T04:30:04.391077+0000","last_undegraded":"2024-06-06T04:30:04.391077+0000","last_fullsized":"2024-06-06T04:30:04.391077+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:37:15.914505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790613+0000","last_change":"2024-06-06T04:27:32.619319+0000","last_active":"2024-06-06T04:29:06.790613+0000","last_peered":"2024-06-06T04:29:06.790613+0000","last_clean":"2024-06-06T04:29:06.790613+0000","last_became_active":"2024-06-06T04:27:32.618843+0000","last_became_peered":"2024-06-06T04:27:32.618843+0000","last_unstale":"2024-06-06T04:29:06.790613+0000","last_undegraded":"2024-06-06T04:29:06.790613+0000","last_fullsized":"2024-06-06T04:29:06.790613+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:32:20.071512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905924+0000","last_change":"2024-06-06T04:27:19.304731+0000","last_active":"2024-06-06T04:29:05.905924+0000","last_peered":"2024-06-06T04:29:05.905924+0000","last_clean":"2024-06-06T04:29:05.905924+0000","last_became_active":"2024-06-06T04:27:19.304351+0000","last_became_peered":"2024-06-06T04:27:19.304351+0000","last_unstale":"2024-06-06T04:29:05.905924+0000","last_undegraded":"2024-06-06T04:29:05.905924+0000","last_fullsized":"2024-06-06T04:29:05.905924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:16:14.136633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.391311+0000","last_change":"2024-06-06T04:29:05.909041+0000","last_active":"2024-06-06T04:30:04.391311+0000","last_peered":"2024-06-06T04:30:04.391311+0000","last_clean":"2024-06-06T04:30:04.391311+0000","last_became_active":"2024-06-06T04:29:05.908666+0000","last_became_peered":"2024-06-06T04:29:05.908666+0000","last_unstale":"2024-06-06T04:30:04.391311+0000","last_undegraded":"2024-06-06T04:30:04.391311+0000","last_fullsized":"2024-06-06T04:30:04.391311+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:47:55.225572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908861+0000","last_change":"2024-06-06T04:27:32.628645+0000","last_active":"2024-06-06T04:29:05.908861+0000","last_peered":"2024-06-06T04:29:05.908861+0000","last_clean":"2024-06-06T04:29:05.908861+0000","last_became_active":"2024-06-06T04:27:32.627586+0000","last_became_peered":"2024-06-06T04:27:32.627586+0000","last_unstale":"2024-06-06T04:29:05.908861+0000","last_undegraded":"2024-06-06T04:29:05.908861+0000","last_fullsized":"2024-06-06T04:29:05.908861+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:46:53.094511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790029+0000","last_change":"2024-06-06T04:27:19.303805+0000","last_active":"2024-06-06T04:29:06.790029+0000","last_peered":"2024-06-06T04:29:06.790029+0000","last_clean":"2024-06-06T04:29:06.790029+0000","last_became_active":"2024-06-06T04:27:19.303391+0000","last_became_peered":"2024-06-06T04:27:19.303391+0000","last_unstale":"2024-06-06T04:29:06.790029+0000","last_undegraded":"2024-06-06T04:29:06.790029+0000","last_fullsized":"2024-06-06T04:29:06.790029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:30:27.336567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.387613+0000","last_change":"2024-06-06T04:27:22.575148+0000","last_active":"2024-06-06T04:30:04.387613+0000","last_peered":"2024-06-06T04:30:04.387613+0000","last_clean":"2024-06-06T04:30:04.387613+0000","last_became_active":"2024-06-06T04:27:22.574591+0000","last_became_peered":"2024-06-06T04:27:22.574591+0000","last_unstale":"2024-06-06T04:30:04.387613+0000","last_undegraded":"2024-06-06T04:30:04.387613+0000","last_fullsized":"2024-06-06T04:30:04.387613+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:35:23.404644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214398+0000","last_change":"2024-06-06T04:28:05.726734+0000","last_active":"2024-06-06T04:29:06.214398+0000","last_peered":"2024-06-06T04:29:06.214398+0000","last_clean":"2024-06-06T04:29:06.214398+0000","last_became_active":"2024-06-06T04:28:05.726238+0000","last_became_peered":"2024-06-06T04:28:05.726238+0000","last_unstale":"2024-06-06T04:29:06.214398+0000","last_undegraded":"2024-06-06T04:29:06.214398+0000","last_fullsized":"2024-06-06T04:29:06.214398+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:47.841475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"100'5","reported_seq":151,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:05.866998+0000","last_change":"2024-06-06T04:27:19.305571+0000","last_active":"2024-06-06T04:30:05.866998+0000","last_peered":"2024-06-06T04:30:05.866998+0000","last_clean":"2024-06-06T04:30:05.866998+0000","last_became_active":"2024-06-06T04:27:19.305183+0000","last_became_peered":"2024-06-06T04:27:19.305183+0000","last_unstale":"2024-06-06T04:30:05.866998+0000","last_undegraded":"2024-06-06T04:30:05.866998+0000","last_fullsized":"2024-06-06T04:30:05.866998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:51.205209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392588+0000","last_change":"2024-06-06T04:29:06.263872+0000","last_active":"2024-06-06T04:30:04.392588+0000","last_peered":"2024-06-06T04:30:04.392588+0000","last_clean":"2024-06-06T04:30:04.392588+0000","last_became_active":"2024-06-06T04:29:06.263487+0000","last_became_peered":"2024-06-06T04:29:06.263487+0000","last_unstale":"2024-06-06T04:30:04.392588+0000","last_undegraded":"2024-06-06T04:30:04.392588+0000","last_fullsized":"2024-06-06T04:30:04.392588+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:15.965926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904639+0000","last_change":"2024-06-06T04:28:05.724892+0000","last_active":"2024-06-06T04:29:05.904639+0000","last_peered":"2024-06-06T04:29:05.904639+0000","last_clean":"2024-06-06T04:29:05.904639+0000","last_became_active":"2024-06-06T04:28:05.724420+0000","last_became_peered":"2024-06-06T04:28:05.724420+0000","last_unstale":"2024-06-06T04:29:05.904639+0000","last_undegraded":"2024-06-06T04:29:05.904639+0000","last_fullsized":"2024-06-06T04:29:05.904639+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:39.879481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":60,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.213846+0000","last_change":"2024-06-06T04:27:19.516687+0000","last_active":"2024-06-06T04:29:06.213846+0000","last_peered":"2024-06-06T04:29:06.213846+0000","last_clean":"2024-06-06T04:29:06.213846+0000","last_became_active":"2024-06-06T04:27:19.516271+0000","last_became_peered":"2024-06-06T04:27:19.516271+0000","last_unstale":"2024-06-06T04:29:06.213846+0000","last_undegraded":"2024-06-06T04:29:06.213846+0000","last_fullsized":"2024-06-06T04:29:06.213846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:33:46.817727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394832+0000","last_change":"2024-06-06T04:27:22.572671+0000","last_active":"2024-06-06T04:30:04.394832+0000","last_peered":"2024-06-06T04:30:04.394832+0000","last_clean":"2024-06-06T04:30:04.394832+0000","last_became_active":"2024-06-06T04:27:22.572230+0000","last_became_peered":"2024-06-06T04:27:22.572230+0000","last_unstale":"2024-06-06T04:30:04.394832+0000","last_undegraded":"2024-06-06T04:30:04.394832+0000","last_fullsized":"2024-06-06T04:30:04.394832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:36:48.393092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.789710+0000","last_change":"2024-06-06T04:27:32.620131+0000","last_active":"2024-06-06T04:29:06.789710+0000","last_peered":"2024-06-06T04:29:06.789710+0000","last_clean":"2024-06-06T04:29:06.789710+0000","last_became_active":"2024-06-06T04:27:32.619686+0000","last_became_peered":"2024-06-06T04:27:32.619686+0000","last_unstale":"2024-06-06T04:29:06.789710+0000","last_undegraded":"2024-06-06T04:29:06.789710+0000","last_fullsized":"2024-06-06T04:29:06.789710+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:19:17.130410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214545+0000","last_change":"2024-06-06T04:27:19.304682+0000","last_active":"2024-06-06T04:29:06.214545+0000","last_peered":"2024-06-06T04:29:06.214545+0000","last_clean":"2024-06-06T04:29:06.214545+0000","last_became_active":"2024-06-06T04:27:19.304262+0000","last_became_peered":"2024-06-06T04:27:19.304262+0000","last_unstale":"2024-06-06T04:29:06.214545+0000","last_undegraded":"2024-06-06T04:29:06.214545+0000","last_fullsized":"2024-06-06T04:29:06.214545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:45.819520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.377096+0000","last_change":"2024-06-06T04:29:06.791555+0000","last_active":"2024-06-06T04:30:04.377096+0000","last_peered":"2024-06-06T04:30:04.377096+0000","last_clean":"2024-06-06T04:30:04.377096+0000","last_became_active":"2024-06-06T04:29:06.790925+0000","last_became_peered":"2024-06-06T04:29:06.790925+0000","last_unstale":"2024-06-06T04:30:04.377096+0000","last_undegraded":"2024-06-06T04:30:04.377096+0000","last_fullsized":"2024-06-06T04:30:04.377096+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:23:39.549238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.213893+0000","last_change":"2024-06-06T04:27:32.622092+0000","last_active":"2024-06-06T04:29:06.213893+0000","last_peered":"2024-06-06T04:29:06.213893+0000","last_clean":"2024-06-06T04:29:06.213893+0000","last_became_active":"2024-06-06T04:27:32.621618+0000","last_became_peered":"2024-06-06T04:27:32.621618+0000","last_unstale":"2024-06-06T04:29:06.213893+0000","last_undegraded":"2024-06-06T04:29:06.213893+0000","last_fullsized":"2024-06-06T04:29:06.213893+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:03.067292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.907896+0000","last_change":"2024-06-06T04:27:19.516803+0000","last_active":"2024-06-06T04:29:05.907896+0000","last_peered":"2024-06-06T04:29:05.907896+0000","last_clean":"2024-06-06T04:29:05.907896+0000","last_became_active":"2024-06-06T04:27:19.516458+0000","last_became_peered":"2024-06-06T04:27:19.516458+0000","last_unstale":"2024-06-06T04:29:05.907896+0000","last_undegraded":"2024-06-06T04:29:05.907896+0000","last_fullsized":"2024-06-06T04:29:05.907896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:34:53.593825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380334+0000","last_change":"2024-06-06T04:27:22.564306+0000","last_active":"2024-06-06T04:30:04.380334+0000","last_peered":"2024-06-06T04:30:04.380334+0000","last_clean":"2024-06-06T04:30:04.380334+0000","last_became_active":"2024-06-06T04:27:22.563922+0000","last_became_peered":"2024-06-06T04:27:22.563922+0000","last_unstale":"2024-06-06T04:30:04.380334+0000","last_undegraded":"2024-06-06T04:30:04.380334+0000","last_fullsized":"2024-06-06T04:30:04.380334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:42:23.189047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904530+0000","last_change":"2024-06-06T04:27:32.617014+0000","last_active":"2024-06-06T04:29:05.904530+0000","last_peered":"2024-06-06T04:29:05.904530+0000","last_clean":"2024-06-06T04:29:05.904530+0000","last_became_active":"2024-06-06T04:27:32.616639+0000","last_became_peered":"2024-06-06T04:27:32.616639+0000","last_unstale":"2024-06-06T04:29:05.904530+0000","last_undegraded":"2024-06-06T04:29:05.904530+0000","last_fullsized":"2024-06-06T04:29:05.904530+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:54:52.034711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905627+0000","last_change":"2024-06-06T04:27:19.305000+0000","last_active":"2024-06-06T04:29:05.905627+0000","last_peered":"2024-06-06T04:29:05.905627+0000","last_clean":"2024-06-06T04:29:05.905627+0000","last_became_active":"2024-06-06T04:27:19.304676+0000","last_became_peered":"2024-06-06T04:27:19.304676+0000","last_unstale":"2024-06-06T04:29:05.905627+0000","last_undegraded":"2024-06-06T04:29:05.905627+0000","last_fullsized":"2024-06-06T04:29:05.905627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:19:34.054455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.382734+0000","last_change":"2024-06-06T04:27:22.572680+0000","last_active":"2024-06-06T04:30:04.382734+0000","last_peered":"2024-06-06T04:30:04.382734+0000","last_clean":"2024-06-06T04:30:04.382734+0000","last_became_active":"2024-06-06T04:27:22.572348+0000","last_became_peered":"2024-06-06T04:27:22.572348+0000","last_unstale":"2024-06-06T04:30:04.382734+0000","last_undegraded":"2024-06-06T04:30:04.382734+0000","last_fullsized":"2024-06-06T04:30:04.382734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:05:26.215771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904503+0000","last_change":"2024-06-06T04:27:32.611587+0000","last_active":"2024-06-06T04:29:05.904503+0000","last_peered":"2024-06-06T04:29:05.904503+0000","last_clean":"2024-06-06T04:29:05.904503+0000","last_became_active":"2024-06-06T04:27:32.611183+0000","last_became_peered":"2024-06-06T04:27:32.611183+0000","last_unstale":"2024-06-06T04:29:05.904503+0000","last_undegraded":"2024-06-06T04:29:05.904503+0000","last_fullsized":"2024-06-06T04:29:05.904503+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:36.773254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.371612+0000","last_change":"2024-06-06T04:29:06.403321+0000","last_active":"2024-06-06T04:30:04.371612+0000","last_peered":"2024-06-06T04:30:04.371612+0000","last_clean":"2024-06-06T04:30:04.371612+0000","last_became_active":"2024-06-06T04:29:06.402909+0000","last_became_peered":"2024-06-06T04:29:06.402909+0000","last_unstale":"2024-06-06T04:30:04.371612+0000","last_undegraded":"2024-06-06T04:30:04.371612+0000","last_fullsized":"2024-06-06T04:30:04.371612+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:29:38.963372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"100'5","reported_seq":149,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:06.867607+0000","last_change":"2024-06-06T04:27:19.516361+0000","last_active":"2024-06-06T04:30:06.867607+0000","last_peered":"2024-06-06T04:30:06.867607+0000","last_clean":"2024-06-06T04:30:06.867607+0000","last_became_active":"2024-06-06T04:27:19.515973+0000","last_became_peered":"2024-06-06T04:27:19.515973+0000","last_unstale":"2024-06-06T04:30:06.867607+0000","last_undegraded":"2024-06-06T04:30:06.867607+0000","last_fullsized":"2024-06-06T04:30:06.867607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:37.298819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905945+0000","last_change":"2024-06-06T04:27:32.630528+0000","last_active":"2024-06-06T04:29:05.905945+0000","last_peered":"2024-06-06T04:29:05.905945+0000","last_clean":"2024-06-06T04:29:05.905945+0000","last_became_active":"2024-06-06T04:27:32.630215+0000","last_became_peered":"2024-06-06T04:27:32.630215+0000","last_unstale":"2024-06-06T04:29:05.905945+0000","last_undegraded":"2024-06-06T04:29:05.905945+0000","last_fullsized":"2024-06-06T04:29:05.905945+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:34:29.980193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392719+0000","last_change":"2024-06-06T04:27:22.567466+0000","last_active":"2024-06-06T04:30:04.392719+0000","last_peered":"2024-06-06T04:30:04.392719+0000","last_clean":"2024-06-06T04:30:04.392719+0000","last_became_active":"2024-06-06T04:27:22.567145+0000","last_became_peered":"2024-06-06T04:27:22.567145+0000","last_unstale":"2024-06-06T04:30:04.392719+0000","last_undegraded":"2024-06-06T04:30:04.392719+0000","last_fullsized":"2024-06-06T04:30:04.392719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:56:56.203027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401783+0000","last_change":"2024-06-06T04:27:19.302726+0000","last_active":"2024-06-06T04:29:06.401783+0000","last_peered":"2024-06-06T04:29:06.401783+0000","last_clean":"2024-06-06T04:29:06.401783+0000","last_became_active":"2024-06-06T04:27:19.302329+0000","last_became_peered":"2024-06-06T04:27:19.302329+0000","last_unstale":"2024-06-06T04:29:06.401783+0000","last_undegraded":"2024-06-06T04:29:06.401783+0000","last_fullsized":"2024-06-06T04:29:06.401783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:55:15.010632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":366,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905176+0000","last_change":"2024-06-06T04:25:53.854188+0000","last_active":"2024-06-06T04:29:05.905176+0000","last_peered":"2024-06-06T04:29:05.905176+0000","last_clean":"2024-06-06T04:29:05.905176+0000","last_became_active":"2024-06-06T04:25:53.853700+0000","last_became_peered":"2024-06-06T04:25:53.853700+0000","last_unstale":"2024-06-06T04:29:05.905176+0000","last_undegraded":"2024-06-06T04:29:05.905176+0000","last_fullsized":"2024-06-06T04:29:05.905176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:22:28.847333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:22:28.847333+0000","last_clean_scrub_stamp":"2024-06-06T04:22:28.847333+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:59:10.246373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790675+0000","last_change":"2024-06-06T04:27:32.623300+0000","last_active":"2024-06-06T04:29:06.790675+0000","last_peered":"2024-06-06T04:29:06.790675+0000","last_clean":"2024-06-06T04:29:06.790675+0000","last_became_active":"2024-06-06T04:27:32.622811+0000","last_became_peered":"2024-06-06T04:27:32.622811+0000","last_unstale":"2024-06-06T04:29:06.790675+0000","last_undegraded":"2024-06-06T04:29:06.790675+0000","last_fullsized":"2024-06-06T04:29:06.790675+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:28.383079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.367134+0000","last_change":"2024-06-06T04:27:22.566617+0000","last_active":"2024-06-06T04:30:04.367134+0000","last_peered":"2024-06-06T04:30:04.367134+0000","last_clean":"2024-06-06T04:30:04.367134+0000","last_became_active":"2024-06-06T04:27:22.566256+0000","last_became_peered":"2024-06-06T04:27:22.566256+0000","last_unstale":"2024-06-06T04:30:04.367134+0000","last_undegraded":"2024-06-06T04:30:04.367134+0000","last_fullsized":"2024-06-06T04:30:04.367134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:50:41.318475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905457+0000","last_change":"2024-06-06T04:27:19.305941+0000","last_active":"2024-06-06T04:29:05.905457+0000","last_peered":"2024-06-06T04:29:05.905457+0000","last_clean":"2024-06-06T04:29:05.905457+0000","last_became_active":"2024-06-06T04:27:19.305037+0000","last_became_peered":"2024-06-06T04:27:19.305037+0000","last_unstale":"2024-06-06T04:29:05.905457+0000","last_undegraded":"2024-06-06T04:29:05.905457+0000","last_fullsized":"2024-06-06T04:29:05.905457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:54.084427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908935+0000","last_change":"2024-06-06T04:27:32.621182+0000","last_active":"2024-06-06T04:29:05.908935+0000","last_peered":"2024-06-06T04:29:05.908935+0000","last_clean":"2024-06-06T04:29:05.908935+0000","last_became_active":"2024-06-06T04:27:32.620710+0000","last_became_peered":"2024-06-06T04:27:32.620710+0000","last_unstale":"2024-06-06T04:29:05.908935+0000","last_undegraded":"2024-06-06T04:29:05.908935+0000","last_fullsized":"2024-06-06T04:29:05.908935+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:05.512309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214246+0000","last_change":"2024-06-06T04:27:19.517033+0000","last_active":"2024-06-06T04:29:06.214246+0000","last_peered":"2024-06-06T04:29:06.214246+0000","last_clean":"2024-06-06T04:29:06.214246+0000","last_became_active":"2024-06-06T04:27:19.516278+0000","last_became_peered":"2024-06-06T04:27:19.516278+0000","last_unstale":"2024-06-06T04:29:06.214246+0000","last_undegraded":"2024-06-06T04:29:06.214246+0000","last_fullsized":"2024-06-06T04:29:06.214246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:11:20.336818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.368634+0000","last_change":"2024-06-06T04:27:22.566261+0000","last_active":"2024-06-06T04:30:04.368634+0000","last_peered":"2024-06-06T04:30:04.368634+0000","last_clean":"2024-06-06T04:30:04.368634+0000","last_became_active":"2024-06-06T04:27:22.565875+0000","last_became_peered":"2024-06-06T04:27:22.565875+0000","last_unstale":"2024-06-06T04:30:04.368634+0000","last_undegraded":"2024-06-06T04:30:04.368634+0000","last_fullsized":"2024-06-06T04:30:04.368634+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:49:55.923950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560350+0000","last_change":"2024-06-06T04:27:32.619854+0000","last_active":"2024-06-06T04:29:06.560350+0000","last_peered":"2024-06-06T04:29:06.560350+0000","last_clean":"2024-06-06T04:29:06.560350+0000","last_became_active":"2024-06-06T04:27:32.618765+0000","last_became_peered":"2024-06-06T04:27:32.618765+0000","last_unstale":"2024-06-06T04:29:06.560350+0000","last_undegraded":"2024-06-06T04:29:06.560350+0000","last_fullsized":"2024-06-06T04:29:06.560350+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:47:05.283751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'12","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:31.134232+0000","last_change":"2024-06-06T04:27:19.303057+0000","last_active":"2024-06-06T04:29:31.134232+0000","last_peered":"2024-06-06T04:29:31.134232+0000","last_clean":"2024-06-06T04:29:31.134232+0000","last_became_active":"2024-06-06T04:27:19.302716+0000","last_became_peered":"2024-06-06T04:27:19.302716+0000","last_unstale":"2024-06-06T04:29:31.134232+0000","last_undegraded":"2024-06-06T04:29:31.134232+0000","last_fullsized":"2024-06-06T04:29:31.134232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:35.273663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381463+0000","last_change":"2024-06-06T04:27:22.575274+0000","last_active":"2024-06-06T04:30:04.381463+0000","last_peered":"2024-06-06T04:30:04.381463+0000","last_clean":"2024-06-06T04:30:04.381463+0000","last_became_active":"2024-06-06T04:27:22.574979+0000","last_became_peered":"2024-06-06T04:27:22.574979+0000","last_unstale":"2024-06-06T04:30:04.381463+0000","last_undegraded":"2024-06-06T04:30:04.381463+0000","last_fullsized":"2024-06-06T04:30:04.381463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:30.718398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905413+0000","last_change":"2024-06-06T04:27:32.619939+0000","last_active":"2024-06-06T04:29:05.905413+0000","last_peered":"2024-06-06T04:29:05.905413+0000","last_clean":"2024-06-06T04:29:05.905413+0000","last_became_active":"2024-06-06T04:27:32.619499+0000","last_became_peered":"2024-06-06T04:27:32.619499+0000","last_unstale":"2024-06-06T04:29:05.905413+0000","last_undegraded":"2024-06-06T04:29:05.905413+0000","last_fullsized":"2024-06-06T04:29:05.905413+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:41:13.933546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402340+0000","last_change":"2024-06-06T04:27:19.397552+0000","last_active":"2024-06-06T04:29:06.402340+0000","last_peered":"2024-06-06T04:29:06.402340+0000","last_clean":"2024-06-06T04:29:06.402340+0000","last_became_active":"2024-06-06T04:27:19.397186+0000","last_became_peered":"2024-06-06T04:27:19.397186+0000","last_unstale":"2024-06-06T04:29:06.402340+0000","last_undegraded":"2024-06-06T04:29:06.402340+0000","last_fullsized":"2024-06-06T04:29:06.402340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:24:13.403860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.384537+0000","last_change":"2024-06-06T04:28:05.728212+0000","last_active":"2024-06-06T04:30:04.384537+0000","last_peered":"2024-06-06T04:30:04.384537+0000","last_clean":"2024-06-06T04:30:04.384537+0000","last_became_active":"2024-06-06T04:28:05.727748+0000","last_became_peered":"2024-06-06T04:28:05.727748+0000","last_unstale":"2024-06-06T04:30:04.384537+0000","last_undegraded":"2024-06-06T04:30:04.384537+0000","last_fullsized":"2024-06-06T04:30:04.384537+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:50.583716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402083+0000","last_change":"2024-06-06T04:28:05.896480+0000","last_active":"2024-06-06T04:29:06.402083+0000","last_peered":"2024-06-06T04:29:06.402083+0000","last_clean":"2024-06-06T04:29:06.402083+0000","last_became_active":"2024-06-06T04:28:05.895967+0000","last_became_peered":"2024-06-06T04:28:05.895967+0000","last_unstale":"2024-06-06T04:29:06.402083+0000","last_undegraded":"2024-06-06T04:29:06.402083+0000","last_fullsized":"2024-06-06T04:29:06.402083+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:26:19.846884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.907843+0000","last_change":"2024-06-06T04:27:19.516515+0000","last_active":"2024-06-06T04:29:06.907843+0000","last_peered":"2024-06-06T04:29:06.907843+0000","last_clean":"2024-06-06T04:29:06.907843+0000","last_became_active":"2024-06-06T04:27:19.516087+0000","last_became_peered":"2024-06-06T04:27:19.516087+0000","last_unstale":"2024-06-06T04:29:06.907843+0000","last_undegraded":"2024-06-06T04:29:06.907843+0000","last_fullsized":"2024-06-06T04:29:06.907843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:58.440226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":40,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.342173+0000","last_change":"2024-06-06T04:27:22.562367+0000","last_active":"2024-06-06T04:30:04.342173+0000","last_peered":"2024-06-06T04:30:04.342173+0000","last_clean":"2024-06-06T04:30:04.342173+0000","last_became_active":"2024-06-06T04:27:22.561947+0000","last_became_peered":"2024-06-06T04:27:22.561947+0000","last_unstale":"2024-06-06T04:30:04.342173+0000","last_undegraded":"2024-06-06T04:30:04.342173+0000","last_fullsized":"2024-06-06T04:30:04.342173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:28:06.200748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908183+0000","last_change":"2024-06-06T04:27:32.631577+0000","last_active":"2024-06-06T04:29:05.908183+0000","last_peered":"2024-06-06T04:29:05.908183+0000","last_clean":"2024-06-06T04:29:05.908183+0000","last_became_active":"2024-06-06T04:27:32.629555+0000","last_became_peered":"2024-06-06T04:27:32.629555+0000","last_unstale":"2024-06-06T04:29:05.908183+0000","last_undegraded":"2024-06-06T04:29:05.908183+0000","last_fullsized":"2024-06-06T04:29:05.908183+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:52:56.197281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560477+0000","last_change":"2024-06-06T04:27:19.298880+0000","last_active":"2024-06-06T04:29:06.560477+0000","last_peered":"2024-06-06T04:29:06.560477+0000","last_clean":"2024-06-06T04:29:06.560477+0000","last_became_active":"2024-06-06T04:27:19.298551+0000","last_became_peered":"2024-06-06T04:27:19.298551+0000","last_unstale":"2024-06-06T04:29:06.560477+0000","last_undegraded":"2024-06-06T04:29:06.560477+0000","last_fullsized":"2024-06-06T04:29:06.560477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:11:32.145401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.383001+0000","last_change":"2024-06-06T04:27:22.575065+0000","last_active":"2024-06-06T04:30:04.383001+0000","last_peered":"2024-06-06T04:30:04.383001+0000","last_clean":"2024-06-06T04:30:04.383001+0000","last_became_active":"2024-06-06T04:27:22.574469+0000","last_became_peered":"2024-06-06T04:27:22.574469+0000","last_unstale":"2024-06-06T04:30:04.383001+0000","last_undegraded":"2024-06-06T04:30:04.383001+0000","last_fullsized":"2024-06-06T04:30:04.383001+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:29:44.943833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402041+0000","last_change":"2024-06-06T04:27:32.626678+0000","last_active":"2024-06-06T04:29:06.402041+0000","last_peered":"2024-06-06T04:29:06.402041+0000","last_clean":"2024-06-06T04:29:06.402041+0000","last_became_active":"2024-06-06T04:27:32.626167+0000","last_became_peered":"2024-06-06T04:27:32.626167+0000","last_unstale":"2024-06-06T04:29:06.402041+0000","last_undegraded":"2024-06-06T04:29:06.402041+0000","last_fullsized":"2024-06-06T04:29:06.402041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:56:26.363195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905544+0000","last_change":"2024-06-06T04:27:19.306466+0000","last_active":"2024-06-06T04:29:05.905544+0000","last_peered":"2024-06-06T04:29:05.905544+0000","last_clean":"2024-06-06T04:29:05.905544+0000","last_became_active":"2024-06-06T04:27:19.303821+0000","last_became_peered":"2024-06-06T04:27:19.303821+0000","last_unstale":"2024-06-06T04:29:05.905544+0000","last_undegraded":"2024-06-06T04:29:05.905544+0000","last_fullsized":"2024-06-06T04:29:05.905544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:23:53.593998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.386917+0000","last_change":"2024-06-06T04:27:22.562662+0000","last_active":"2024-06-06T04:30:04.386917+0000","last_peered":"2024-06-06T04:30:04.386917+0000","last_clean":"2024-06-06T04:30:04.386917+0000","last_became_active":"2024-06-06T04:27:22.562318+0000","last_became_peered":"2024-06-06T04:27:22.562318+0000","last_unstale":"2024-06-06T04:30:04.386917+0000","last_undegraded":"2024-06-06T04:30:04.386917+0000","last_fullsized":"2024-06-06T04:30:04.386917+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:31:26.639275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402218+0000","last_change":"2024-06-06T04:27:32.619970+0000","last_active":"2024-06-06T04:29:06.402218+0000","last_peered":"2024-06-06T04:29:06.402218+0000","last_clean":"2024-06-06T04:29:06.402218+0000","last_became_active":"2024-06-06T04:27:32.619464+0000","last_became_peered":"2024-06-06T04:27:32.619464+0000","last_unstale":"2024-06-06T04:29:06.402218+0000","last_undegraded":"2024-06-06T04:29:06.402218+0000","last_fullsized":"2024-06-06T04:29:06.402218+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:44.844018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214056+0000","last_change":"2024-06-06T04:27:19.308172+0000","last_active":"2024-06-06T04:29:06.214056+0000","last_peered":"2024-06-06T04:29:06.214056+0000","last_clean":"2024-06-06T04:29:06.214056+0000","last_became_active":"2024-06-06T04:27:19.307780+0000","last_became_peered":"2024-06-06T04:27:19.307780+0000","last_unstale":"2024-06-06T04:29:06.214056+0000","last_undegraded":"2024-06-06T04:29:06.214056+0000","last_fullsized":"2024-06-06T04:29:06.214056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:39:11.653649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394812+0000","last_change":"2024-06-06T04:27:22.568881+0000","last_active":"2024-06-06T04:30:04.394812+0000","last_peered":"2024-06-06T04:30:04.394812+0000","last_clean":"2024-06-06T04:30:04.394812+0000","last_became_active":"2024-06-06T04:27:22.568489+0000","last_became_peered":"2024-06-06T04:27:22.568489+0000","last_unstale":"2024-06-06T04:30:04.394812+0000","last_undegraded":"2024-06-06T04:30:04.394812+0000","last_fullsized":"2024-06-06T04:30:04.394812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:00:22.339313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.907971+0000","last_change":"2024-06-06T04:27:32.622357+0000","last_active":"2024-06-06T04:29:06.907971+0000","last_peered":"2024-06-06T04:29:06.907971+0000","last_clean":"2024-06-06T04:29:06.907971+0000","last_became_active":"2024-06-06T04:27:32.621680+0000","last_became_peered":"2024-06-06T04:27:32.621680+0000","last_unstale":"2024-06-06T04:29:06.907971+0000","last_undegraded":"2024-06-06T04:29:06.907971+0000","last_fullsized":"2024-06-06T04:29:06.907971+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:00.677721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.906141+0000","last_change":"2024-06-06T04:27:19.516890+0000","last_active":"2024-06-06T04:29:05.906141+0000","last_peered":"2024-06-06T04:29:05.906141+0000","last_clean":"2024-06-06T04:29:05.906141+0000","last_became_active":"2024-06-06T04:27:19.516490+0000","last_became_peered":"2024-06-06T04:27:19.516490+0000","last_unstale":"2024-06-06T04:29:05.906141+0000","last_undegraded":"2024-06-06T04:29:05.906141+0000","last_fullsized":"2024-06-06T04:29:05.906141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:11:59.336250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.395341+0000","last_change":"2024-06-06T04:27:22.571003+0000","last_active":"2024-06-06T04:30:04.395341+0000","last_peered":"2024-06-06T04:30:04.395341+0000","last_clean":"2024-06-06T04:30:04.395341+0000","last_became_active":"2024-06-06T04:27:22.570442+0000","last_became_peered":"2024-06-06T04:27:22.570442+0000","last_unstale":"2024-06-06T04:30:04.395341+0000","last_undegraded":"2024-06-06T04:30:04.395341+0000","last_fullsized":"2024-06-06T04:30:04.395341+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:28:26.870429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.164730+0000","last_change":"2024-06-06T04:27:32.624424+0000","last_active":"2024-06-06T04:29:06.164730+0000","last_peered":"2024-06-06T04:29:06.164730+0000","last_clean":"2024-06-06T04:29:06.164730+0000","last_became_active":"2024-06-06T04:27:32.624026+0000","last_became_peered":"2024-06-06T04:27:32.624026+0000","last_unstale":"2024-06-06T04:29:06.164730+0000","last_undegraded":"2024-06-06T04:29:06.164730+0000","last_fullsized":"2024-06-06T04:29:06.164730+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:30:08.814284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790341+0000","last_change":"2024-06-06T04:27:19.301382+0000","last_active":"2024-06-06T04:29:06.790341+0000","last_peered":"2024-06-06T04:29:06.790341+0000","last_clean":"2024-06-06T04:29:06.790341+0000","last_became_active":"2024-06-06T04:27:19.300882+0000","last_became_peered":"2024-06-06T04:27:19.300882+0000","last_unstale":"2024-06-06T04:29:06.790341+0000","last_undegraded":"2024-06-06T04:29:06.790341+0000","last_fullsized":"2024-06-06T04:29:06.790341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:50.514103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392499+0000","last_change":"2024-06-06T04:27:22.570661+0000","last_active":"2024-06-06T04:30:04.392499+0000","last_peered":"2024-06-06T04:30:04.392499+0000","last_clean":"2024-06-06T04:30:04.392499+0000","last_became_active":"2024-06-06T04:27:22.570287+0000","last_became_peered":"2024-06-06T04:27:22.570287+0000","last_unstale":"2024-06-06T04:30:04.392499+0000","last_undegraded":"2024-06-06T04:30:04.392499+0000","last_fullsized":"2024-06-06T04:30:04.392499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:48.790492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.789984+0000","last_change":"2024-06-06T04:27:32.618385+0000","last_active":"2024-06-06T04:29:06.789984+0000","last_peered":"2024-06-06T04:29:06.789984+0000","last_clean":"2024-06-06T04:29:06.789984+0000","last_became_active":"2024-06-06T04:27:32.617956+0000","last_became_peered":"2024-06-06T04:27:32.617956+0000","last_unstale":"2024-06-06T04:29:06.789984+0000","last_undegraded":"2024-06-06T04:29:06.789984+0000","last_fullsized":"2024-06-06T04:29:06.789984+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:26.308657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904541+0000","last_change":"2024-06-06T04:28:05.728633+0000","last_active":"2024-06-06T04:29:05.904541+0000","last_peered":"2024-06-06T04:29:05.904541+0000","last_clean":"2024-06-06T04:29:05.904541+0000","last_became_active":"2024-06-06T04:28:05.722427+0000","last_became_peered":"2024-06-06T04:28:05.722427+0000","last_unstale":"2024-06-06T04:29:05.904541+0000","last_undegraded":"2024-06-06T04:29:05.904541+0000","last_fullsized":"2024-06-06T04:29:05.904541+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:13:22.061798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388201+0000","last_change":"2024-06-06T04:27:22.574950+0000","last_active":"2024-06-06T04:30:04.388201+0000","last_peered":"2024-06-06T04:30:04.388201+0000","last_clean":"2024-06-06T04:30:04.388201+0000","last_became_active":"2024-06-06T04:27:22.574604+0000","last_became_peered":"2024-06-06T04:27:22.574604+0000","last_unstale":"2024-06-06T04:30:04.388201+0000","last_undegraded":"2024-06-06T04:30:04.388201+0000","last_fullsized":"2024-06-06T04:30:04.388201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:17:29.395137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380448+0000","last_change":"2024-06-06T04:27:22.568554+0000","last_active":"2024-06-06T04:30:04.380448+0000","last_peered":"2024-06-06T04:30:04.380448+0000","last_clean":"2024-06-06T04:30:04.380448+0000","last_became_active":"2024-06-06T04:27:22.568196+0000","last_became_peered":"2024-06-06T04:27:22.568196+0000","last_unstale":"2024-06-06T04:30:04.380448+0000","last_undegraded":"2024-06-06T04:30:04.380448+0000","last_fullsized":"2024-06-06T04:30:04.380448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:11.564067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.372439+0000","last_change":"2024-06-06T04:27:22.565472+0000","last_active":"2024-06-06T04:30:04.372439+0000","last_peered":"2024-06-06T04:30:04.372439+0000","last_clean":"2024-06-06T04:30:04.372439+0000","last_became_active":"2024-06-06T04:27:22.565116+0000","last_became_peered":"2024-06-06T04:27:22.565116+0000","last_unstale":"2024-06-06T04:30:04.372439+0000","last_undegraded":"2024-06-06T04:30:04.372439+0000","last_fullsized":"2024-06-06T04:30:04.372439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:03:58.232707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.377900+0000","last_change":"2024-06-06T04:27:22.560210+0000","last_active":"2024-06-06T04:30:04.377900+0000","last_peered":"2024-06-06T04:30:04.377900+0000","last_clean":"2024-06-06T04:30:04.377900+0000","last_became_active":"2024-06-06T04:27:22.559853+0000","last_became_peered":"2024-06-06T04:27:22.559853+0000","last_unstale":"2024-06-06T04:30:04.377900+0000","last_undegraded":"2024-06-06T04:30:04.377900+0000","last_fullsized":"2024-06-06T04:30:04.377900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:56:18.650427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385809+0000","last_change":"2024-06-06T04:27:22.565071+0000","last_active":"2024-06-06T04:30:04.385809+0000","last_peered":"2024-06-06T04:30:04.385809+0000","last_clean":"2024-06-06T04:30:04.385809+0000","last_became_active":"2024-06-06T04:27:22.561411+0000","last_became_peered":"2024-06-06T04:27:22.561411+0000","last_unstale":"2024-06-06T04:30:04.385809+0000","last_undegraded":"2024-06-06T04:30:04.385809+0000","last_fullsized":"2024-06-06T04:30:04.385809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:41.948481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.382771+0000","last_change":"2024-06-06T04:27:22.563280+0000","last_active":"2024-06-06T04:30:04.382771+0000","last_peered":"2024-06-06T04:30:04.382771+0000","last_clean":"2024-06-06T04:30:04.382771+0000","last_became_active":"2024-06-06T04:27:22.563049+0000","last_became_peered":"2024-06-06T04:27:22.563049+0000","last_unstale":"2024-06-06T04:30:04.382771+0000","last_undegraded":"2024-06-06T04:30:04.382771+0000","last_fullsized":"2024-06-06T04:30:04.382771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:41:48.287969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.377210+0000","last_change":"2024-06-06T04:27:22.567691+0000","last_active":"2024-06-06T04:30:04.377210+0000","last_peered":"2024-06-06T04:30:04.377210+0000","last_clean":"2024-06-06T04:30:04.377210+0000","last_became_active":"2024-06-06T04:27:22.567331+0000","last_became_peered":"2024-06-06T04:27:22.567331+0000","last_unstale":"2024-06-06T04:30:04.377210+0000","last_undegraded":"2024-06-06T04:30:04.377210+0000","last_fullsized":"2024-06-06T04:30:04.377210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:03.886084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.393172+0000","last_change":"2024-06-06T04:27:22.563853+0000","last_active":"2024-06-06T04:30:04.393172+0000","last_peered":"2024-06-06T04:30:04.393172+0000","last_clean":"2024-06-06T04:30:04.393172+0000","last_became_active":"2024-06-06T04:27:22.563436+0000","last_became_peered":"2024-06-06T04:27:22.563436+0000","last_unstale":"2024-06-06T04:30:04.393172+0000","last_undegraded":"2024-06-06T04:30:04.393172+0000","last_fullsized":"2024-06-06T04:30:04.393172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:26.201308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.387060+0000","last_change":"2024-06-06T04:27:22.568189+0000","last_active":"2024-06-06T04:30:04.387060+0000","last_peered":"2024-06-06T04:30:04.387060+0000","last_clean":"2024-06-06T04:30:04.387060+0000","last_became_active":"2024-06-06T04:27:22.567815+0000","last_became_peered":"2024-06-06T04:27:22.567815+0000","last_unstale":"2024-06-06T04:30:04.387060+0000","last_undegraded":"2024-06-06T04:30:04.387060+0000","last_fullsized":"2024-06-06T04:30:04.387060+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:13:39.259012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.359906+0000","last_change":"2024-06-06T04:27:22.569698+0000","last_active":"2024-06-06T04:30:04.359906+0000","last_peered":"2024-06-06T04:30:04.359906+0000","last_clean":"2024-06-06T04:30:04.359906+0000","last_became_active":"2024-06-06T04:27:22.569347+0000","last_became_peered":"2024-06-06T04:27:22.569347+0000","last_unstale":"2024-06-06T04:30:04.359906+0000","last_undegraded":"2024-06-06T04:30:04.359906+0000","last_fullsized":"2024-06-06T04:30:04.359906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:06:34.722798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381456+0000","last_change":"2024-06-06T04:27:22.561149+0000","last_active":"2024-06-06T04:30:04.381456+0000","last_peered":"2024-06-06T04:30:04.381456+0000","last_clean":"2024-06-06T04:30:04.381456+0000","last_became_active":"2024-06-06T04:27:22.560793+0000","last_became_peered":"2024-06-06T04:27:22.560793+0000","last_unstale":"2024-06-06T04:30:04.381456+0000","last_undegraded":"2024-06-06T04:30:04.381456+0000","last_fullsized":"2024-06-06T04:30:04.381456+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:00:41.563443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394791+0000","last_change":"2024-06-06T04:27:22.569331+0000","last_active":"2024-06-06T04:30:04.394791+0000","last_peered":"2024-06-06T04:30:04.394791+0000","last_clean":"2024-06-06T04:30:04.394791+0000","last_became_active":"2024-06-06T04:27:22.569021+0000","last_became_peered":"2024-06-06T04:27:22.569021+0000","last_unstale":"2024-06-06T04:30:04.394791+0000","last_undegraded":"2024-06-06T04:30:04.394791+0000","last_fullsized":"2024-06-06T04:30:04.394791+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:52:37.064317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.391532+0000","last_change":"2024-06-06T04:27:22.574558+0000","last_active":"2024-06-06T04:30:04.391532+0000","last_peered":"2024-06-06T04:30:04.391532+0000","last_clean":"2024-06-06T04:30:04.391532+0000","last_became_active":"2024-06-06T04:27:22.574215+0000","last_became_peered":"2024-06-06T04:27:22.574215+0000","last_unstale":"2024-06-06T04:30:04.391532+0000","last_undegraded":"2024-06-06T04:30:04.391532+0000","last_fullsized":"2024-06-06T04:30:04.391532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:40:52.496322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385148+0000","last_change":"2024-06-06T04:27:22.565320+0000","last_active":"2024-06-06T04:30:04.385148+0000","last_peered":"2024-06-06T04:30:04.385148+0000","last_clean":"2024-06-06T04:30:04.385148+0000","last_became_active":"2024-06-06T04:27:22.564786+0000","last_became_peered":"2024-06-06T04:27:22.564786+0000","last_unstale":"2024-06-06T04:30:04.385148+0000","last_undegraded":"2024-06-06T04:30:04.385148+0000","last_fullsized":"2024-06-06T04:30:04.385148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:19:42.057666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249851,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69006,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":615,"num_read_kb":382,"num_write":176,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":207018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351026,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":572909,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612663,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28236,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716916,"statfs":{"total":95995035648,"available":95966121984,"internally_reserved":0,"allocated":1249280,"data_stored":1225573,"data_compressed":68587,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776188,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":577241,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037825,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573065,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299462,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":1169114,"data_compressed":68587,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462988,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":568321,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":633096,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":569709,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":634446,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149786,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":568515,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888149,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":573502,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724624,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1175552,"data_stored":1164171,"data_compressed":68587,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34523,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34523,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T04:30:12.583 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-06T04:30:12.583 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-06T04:30:12.583 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-06T04:30:12.583 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-06T04:30:12.838 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:12.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:12 smithi086 ceph-mon[36053]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:30:12.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:12 smithi154 ceph-mon[36428]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:30:13.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:13 smithi086 ceph-mon[36053]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-06T04:30:13.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:13 smithi154 ceph-mon[36428]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-06T04:30:13.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:13 smithi045 ceph-mon[32299]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-06T04:30:14.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.645+0000 7f011368d700 1 -- 172.21.15.45:0/1775682521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c1030f0 msgr2=0x7f010c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:14.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.645+0000 7f011368d700 1 --2- 172.21.15.45:0/1775682521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c1030f0 0x7f010c107110 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0099c0 tx=0x7f00fc009cd0 comp rx=0 tx=0).stop 2024-06-06T04:30:14.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.645+0000 7f011368d700 1 -- 172.21.15.45:0/1775682521 shutdown_connections 2024-06-06T04:30:14.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.645+0000 7f011368d700 1 --2- 172.21.15.45:0/1775682521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f010c107840 0x7f010c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:14.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.645+0000 7f011368d700 1 --2- 172.21.15.45:0/1775682521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f010c102740 0x7f010c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:14.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.645+0000 7f011368d700 1 --2- 172.21.15.45:0/1775682521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c1030f0 0x7f010c107110 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:14.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.645+0000 7f011368d700 1 -- 172.21.15.45:0/1775682521 >> 172.21.15.45:0/1775682521 conn(0x7f010c0795f0 msgr2=0x7f010c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:14.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.645+0000 7f011368d700 1 -- 172.21.15.45:0/1775682521 shutdown_connections 2024-06-06T04:30:14.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.645+0000 7f011368d700 1 -- 172.21.15.45:0/1775682521 wait complete. 2024-06-06T04:30:14.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.645+0000 7f011368d700 1 Processor -- start 2024-06-06T04:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f011368d700 1 -- start start 2024-06-06T04:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f011368d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f010c102740 0x7f010c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f011368d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c1030f0 0x7f010c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f011368d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f010c107840 0x7f010c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f011368d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f010c112680 con 0x7f010c1030f0 2024-06-06T04:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f011368d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f010c112500 con 0x7f010c102740 2024-06-06T04:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f011368d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f010c112800 con 0x7f010c107840 2024-06-06T04:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f0110c28700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c1030f0 0x7f010c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f0110c28700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c1030f0 0x7f010c19ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54190/0 (socket says 172.21.15.45:54190) 2024-06-06T04:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f0110c28700 1 -- 172.21.15.45:0/2425535305 learned_addr learned my addr 172.21.15.45:0/2425535305 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f0111429700 1 --2- 172.21.15.45:0/2425535305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f010c102740 0x7f010c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f0111c2a700 1 --2- 172.21.15.45:0/2425535305 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f010c107840 0x7f010c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f0110c28700 1 -- 172.21.15.45:0/2425535305 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f010c107840 msgr2=0x7f010c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f0110c28700 1 --2- 172.21.15.45:0/2425535305 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f010c107840 0x7f010c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f0110c28700 1 -- 172.21.15.45:0/2425535305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f010c102740 msgr2=0x7f010c19e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f0110c28700 1 --2- 172.21.15.45:0/2425535305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f010c102740 0x7f010c19e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f0110c28700 1 -- 172.21.15.45:0/2425535305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c199040 con 0x7f010c1030f0 2024-06-06T04:30:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.646+0000 7f0111429700 1 --2- 172.21.15.45:0/2425535305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f010c102740 0x7f010c19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:30:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.647+0000 7f0110c28700 1 --2- 172.21.15.45:0/2425535305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c1030f0 0x7f010c19ec60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009fd0 tx=0x7f00fc0037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.647+0000 7f01027fc700 1 -- 172.21.15.45:0/2425535305 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc027070 con 0x7f010c1030f0 2024-06-06T04:30:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.647+0000 7f01027fc700 1 -- 172.21.15.45:0/2425535305 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00fc004140 con 0x7f010c1030f0 2024-06-06T04:30:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.647+0000 7f011368d700 1 -- 172.21.15.45:0/2425535305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010c1992d0 con 0x7f010c1030f0 2024-06-06T04:30:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.647+0000 7f01027fc700 1 -- 172.21.15.45:0/2425535305 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc022e60 con 0x7f010c1030f0 2024-06-06T04:30:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.647+0000 7f011368d700 1 -- 172.21.15.45:0/2425535305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f010c1a55f0 con 0x7f010c1030f0 2024-06-06T04:30:14.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.648+0000 7f011368d700 1 -- 172.21.15.45:0/2425535305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f010c069180 con 0x7f010c1030f0 2024-06-06T04:30:14.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.649+0000 7f01027fc700 1 -- 172.21.15.45:0/2425535305 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f00fc02a070 con 0x7f010c1030f0 2024-06-06T04:30:14.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.650+0000 7f01027fc700 1 --2- 172.21.15.45:0/2425535305 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f00f80a2c20 0x7f00f80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:14.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.650+0000 7f0111429700 1 --2- 172.21.15.45:0/2425535305 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f00f80a2c20 0x7f00f80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:14.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.650+0000 7f01027fc700 1 -- 172.21.15.45:0/2425535305 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f00fc0d0240 con 0x7f010c1030f0 2024-06-06T04:30:14.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.650+0000 7f0111429700 1 --2- 172.21.15.45:0/2425535305 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f00f80a2c20 0x7f00f80a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f010c199a20 tx=0x7f010800a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:14.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.654+0000 7f01027fc700 1 -- 172.21.15.45:0/2425535305 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f00fc013070 con 0x7f010c1030f0 2024-06-06T04:30:14.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.907+0000 7f011368d700 1 -- 172.21.15.45:0/2425535305 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f010c02dca0 con 0x7f010c1030f0 2024-06-06T04:30:14.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.908+0000 7f01027fc700 1 -- 172.21.15.45:0/2425535305 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f00fc09ce70 con 0x7f010c1030f0 2024-06-06T04:30:14.909 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:30:14.909 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-06T04:30:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.911+0000 7f011368d700 1 -- 172.21.15.45:0/2425535305 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f00f80a2c20 msgr2=0x7f00f80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.911+0000 7f011368d700 1 --2- 172.21.15.45:0/2425535305 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f00f80a2c20 0x7f00f80a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f010c199a20 tx=0x7f010800a680 comp rx=0 tx=0).stop 2024-06-06T04:30:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.911+0000 7f011368d700 1 -- 172.21.15.45:0/2425535305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c1030f0 msgr2=0x7f010c19ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.911+0000 7f011368d700 1 --2- 172.21.15.45:0/2425535305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c1030f0 0x7f010c19ec60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009fd0 tx=0x7f00fc0037a0 comp rx=0 tx=0).stop 2024-06-06T04:30:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.911+0000 7f011368d700 1 -- 172.21.15.45:0/2425535305 shutdown_connections 2024-06-06T04:30:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.911+0000 7f011368d700 1 --2- 172.21.15.45:0/2425535305 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f010c107840 0x7f010c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.911+0000 7f011368d700 1 --2- 172.21.15.45:0/2425535305 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f00f80a2c20 0x7f00f80a50e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.911+0000 7f011368d700 1 --2- 172.21.15.45:0/2425535305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f010c102740 0x7f010c19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:14.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.911+0000 7f011368d700 1 --2- 172.21.15.45:0/2425535305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c1030f0 0x7f010c19ec60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:14.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.911+0000 7f011368d700 1 -- 172.21.15.45:0/2425535305 >> 172.21.15.45:0/2425535305 conn(0x7f010c0795f0 msgr2=0x7f010c101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:14.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.911+0000 7f011368d700 1 -- 172.21.15.45:0/2425535305 shutdown_connections 2024-06-06T04:30:14.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:14.911+0000 7f011368d700 1 -- 172.21.15.45:0/2425535305 wait complete. 2024-06-06T04:30:15.695 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-06T04:30:15.696 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:30:15.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:15 smithi086 ceph-mon[36053]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-06T04:30:15.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:15 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2425535305' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-06T04:30:15.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:15 smithi154 ceph-mon[36428]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-06T04:30:15.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:15 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2425535305' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-06T04:30:15.949 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:15.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:15 smithi045 ceph-mon[32299]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-06T04:30:15.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:15 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2425535305' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-06T04:30:17.671 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:17 smithi045 ceph-mon[32299]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-06T04:30:17.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.780+0000 7f7f6a377700 1 -- 172.21.15.45:0/2613393976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f641073f0 msgr2=0x7f7f641077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:17.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.780+0000 7f7f6a377700 1 --2- 172.21.15.45:0/2613393976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f641073f0 0x7f7f641077d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009990 tx=0x7f7f54009ca0 comp rx=0 tx=0).stop 2024-06-06T04:30:17.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.780+0000 7f7f6a377700 1 -- 172.21.15.45:0/2613393976 shutdown_connections 2024-06-06T04:30:17.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.780+0000 7f7f6a377700 1 --2- 172.21.15.45:0/2613393976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f64101d40 0x7f7f6410e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:17.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.780+0000 7f7f6a377700 1 --2- 172.21.15.45:0/2613393976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f641073f0 0x7f7f641077d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:17.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.780+0000 7f7f6a377700 1 --2- 172.21.15.45:0/2613393976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f64101380 0x7f7f64101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:17.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.780+0000 7f7f6a377700 1 -- 172.21.15.45:0/2613393976 >> 172.21.15.45:0/2613393976 conn(0x7f7f64078300 msgr2=0x7f7f64078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:17.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.781+0000 7f7f6a377700 1 -- 172.21.15.45:0/2613393976 shutdown_connections 2024-06-06T04:30:17.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.781+0000 7f7f6a377700 1 -- 172.21.15.45:0/2613393976 wait complete. 2024-06-06T04:30:17.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.781+0000 7f7f6a377700 1 Processor -- start 2024-06-06T04:30:17.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.781+0000 7f7f6a377700 1 -- start start 2024-06-06T04:30:17.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.781+0000 7f7f6a377700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f64101380 0x7f7f64103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:17.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.781+0000 7f7f6a377700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f64101d40 0x7f7f641035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:17.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.781+0000 7f7f6a377700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f641073f0 0x7f7f64103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:17.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.781+0000 7f7f6a377700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7f64111510 con 0x7f7f641073f0 2024-06-06T04:30:17.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f6a377700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f64111390 con 0x7f7f64101d40 2024-06-06T04:30:17.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f6a377700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f64111690 con 0x7f7f64101380 2024-06-06T04:30:17.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f68914700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f641073f0 0x7f7f64103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:17.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f5bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f64101d40 0x7f7f641035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:17.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f68914700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f641073f0 0x7f7f64103ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52532/0 (socket says 172.21.15.45:52532) 2024-06-06T04:30:17.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f68914700 1 -- 172.21.15.45:0/2128635114 learned_addr learned my addr 172.21.15.45:0/2128635114 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:17.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f63fff700 1 --2- 172.21.15.45:0/2128635114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f64101380 0x7f7f64103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f68914700 1 -- 172.21.15.45:0/2128635114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f64101380 msgr2=0x7f7f64103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f68914700 1 --2- 172.21.15.45:0/2128635114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f64101380 0x7f7f64103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f68914700 1 -- 172.21.15.45:0/2128635114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f64101d40 msgr2=0x7f7f641035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f68914700 1 --2- 172.21.15.45:0/2128635114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f64101d40 0x7f7f641035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f68914700 1 -- 172.21.15.45:0/2128635114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f641a7010 con 0x7f7f641073f0 2024-06-06T04:30:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f68914700 1 --2- 172.21.15.45:0/2128635114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f641073f0 0x7f7f64103ae0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c007d70 tx=0x7f7f5c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f61ffb700 1 -- 172.21.15.45:0/2128635114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c016020 con 0x7f7f641073f0 2024-06-06T04:30:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f6a377700 1 -- 172.21.15.45:0/2128635114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f641a7300 con 0x7f7f641073f0 2024-06-06T04:30:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f61ffb700 1 -- 172.21.15.45:0/2128635114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f5c00ac90 con 0x7f7f641073f0 2024-06-06T04:30:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.782+0000 7f7f61ffb700 1 -- 172.21.15.45:0/2128635114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c0115e0 con 0x7f7f641073f0 2024-06-06T04:30:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.783+0000 7f7f6a377700 1 -- 172.21.15.45:0/2128635114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f641a77f0 con 0x7f7f641073f0 2024-06-06T04:30:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.783+0000 7f7f6a377700 1 -- 172.21.15.45:0/2128635114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f64069180 con 0x7f7f641073f0 2024-06-06T04:30:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.785+0000 7f7f61ffb700 1 -- 172.21.15.45:0/2128635114 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7f5c019020 con 0x7f7f641073f0 2024-06-06T04:30:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.785+0000 7f7f61ffb700 1 --2- 172.21.15.45:0/2128635114 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7f500a2c20 0x7f7f500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.785+0000 7f7f63fff700 1 --2- 172.21.15.45:0/2128635114 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7f500a2c20 0x7f7f500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.785+0000 7f7f61ffb700 1 -- 172.21.15.45:0/2128635114 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f7f5c0cf320 con 0x7f7f641073f0 2024-06-06T04:30:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.786+0000 7f7f63fff700 1 --2- 172.21.15.45:0/2128635114 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7f500a2c20 0x7f7f500a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400b5c0 tx=0x7f7f54005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:17.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:17.790+0000 7f7f61ffb700 1 -- 172.21.15.45:0/2128635114 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7f5c020020 con 0x7f7f641073f0 2024-06-06T04:30:17.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:17 smithi086 ceph-mon[36053]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-06T04:30:17.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:17 smithi154 ceph-mon[36428]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-06T04:30:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:18.043+0000 7f7f6a377700 1 -- 172.21.15.45:0/2128635114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f641a7ab0 con 0x7f7f641073f0 2024-06-06T04:30:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:18.044+0000 7f7f61ffb700 1 -- 172.21.15.45:0/2128635114 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11858 (secure 0 0 0) 0x7f7f5c09bec0 con 0x7f7f641073f0 2024-06-06T04:30:18.046 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:30:18.046 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T04:27:24.462897+0000","modified":"2024-06-06T04:28:32.515629+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24463,24473,24451]},"id":1}]} 2024-06-06T04:30:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:18.046+0000 7f7f6a377700 1 -- 172.21.15.45:0/2128635114 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7f500a2c20 msgr2=0x7f7f500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:18.046+0000 7f7f6a377700 1 --2- 172.21.15.45:0/2128635114 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7f500a2c20 0x7f7f500a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400b5c0 tx=0x7f7f54005960 comp rx=0 tx=0).stop 2024-06-06T04:30:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:18.046+0000 7f7f6a377700 1 -- 172.21.15.45:0/2128635114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f641073f0 msgr2=0x7f7f64103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:18.046+0000 7f7f6a377700 1 --2- 172.21.15.45:0/2128635114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f641073f0 0x7f7f64103ae0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c007d70 tx=0x7f7f5c00a340 comp rx=0 tx=0).stop 2024-06-06T04:30:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:18.046+0000 7f7f6a377700 1 -- 172.21.15.45:0/2128635114 shutdown_connections 2024-06-06T04:30:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:18.046+0000 7f7f6a377700 1 --2- 172.21.15.45:0/2128635114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f641073f0 0x7f7f64103ae0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:18.046+0000 7f7f6a377700 1 --2- 172.21.15.45:0/2128635114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f64101380 0x7f7f64103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:18.046+0000 7f7f6a377700 1 --2- 172.21.15.45:0/2128635114 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7f500a2c20 0x7f7f500a50e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:18.046+0000 7f7f6a377700 1 --2- 172.21.15.45:0/2128635114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f64101d40 0x7f7f641035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:18.046+0000 7f7f6a377700 1 -- 172.21.15.45:0/2128635114 >> 172.21.15.45:0/2128635114 conn(0x7f7f64078300 msgr2=0x7f7f641006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:18.047+0000 7f7f6a377700 1 -- 172.21.15.45:0/2128635114 shutdown_connections 2024-06-06T04:30:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:18.047+0000 7f7f6a377700 1 -- 172.21.15.45:0/2128635114 wait complete. 2024-06-06T04:30:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-06-06T04:30:18.603 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:18 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2128635114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:30:18.631 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:30:18.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:18 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2128635114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:30:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:18.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:18 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2128635114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:30:19.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:19 smithi045 ceph-mon[32299]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-06T04:30:19.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:19 smithi086 ceph-mon[36053]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-06T04:30:19.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:19 smithi154 ceph-mon[36428]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-06T04:30:20.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.684+0000 7f1f5c6bd700 1 -- 172.21.15.45:0/253922463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f54102e80 msgr2=0x7f1f5410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:20.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.684+0000 7f1f5c6bd700 1 --2- 172.21.15.45:0/253922463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f54102e80 0x7f1f5410f330 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00b0a0 tx=0x7f1f4c00b3b0 comp rx=0 tx=0).stop 2024-06-06T04:30:20.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.685+0000 7f1f5c6bd700 1 -- 172.21.15.45:0/253922463 shutdown_connections 2024-06-06T04:30:20.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.685+0000 7f1f5c6bd700 1 --2- 172.21.15.45:0/253922463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f54102e80 0x7f1f5410f330 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:20.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.685+0000 7f1f5c6bd700 1 --2- 172.21.15.45:0/253922463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f541084f0 0x7f1f541088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:20.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.685+0000 7f1f5c6bd700 1 --2- 172.21.15.45:0/253922463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f541024e0 0x7f1f54102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:20.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.685+0000 7f1f5c6bd700 1 -- 172.21.15.45:0/253922463 >> 172.21.15.45:0/253922463 conn(0x7f1f54079530 msgr2=0x7f1f54079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:20.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.685+0000 7f1f5c6bd700 1 -- 172.21.15.45:0/253922463 shutdown_connections 2024-06-06T04:30:20.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.685+0000 7f1f5c6bd700 1 -- 172.21.15.45:0/253922463 wait complete. 2024-06-06T04:30:20.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.686+0000 7f1f5c6bd700 1 Processor -- start 2024-06-06T04:30:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.686+0000 7f1f5c6bd700 1 -- start start 2024-06-06T04:30:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.686+0000 7f1f5c6bd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f541024e0 0x7f1f54074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.686+0000 7f1f5c6bd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f54102e80 0x7f1f54074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.686+0000 7f1f5c6bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f541084f0 0x7f1f54075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.686+0000 7f1f5c6bd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54112610 con 0x7f1f541084f0 2024-06-06T04:30:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.686+0000 7f1f5c6bd700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54112490 con 0x7f1f541024e0 2024-06-06T04:30:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.686+0000 7f1f5c6bd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54112790 con 0x7f1f54102e80 2024-06-06T04:30:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.686+0000 7f1f5ac5a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f541084f0 0x7f1f54075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.686+0000 7f1f5ac5a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f541084f0 0x7f1f54075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52544/0 (socket says 172.21.15.45:52544) 2024-06-06T04:30:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.686+0000 7f1f5ac5a700 1 -- 172.21.15.45:0/3628484569 learned_addr learned my addr 172.21.15.45:0/3628484569 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.686+0000 7f1f5a459700 1 --2- 172.21.15.45:0/3628484569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f541024e0 0x7f1f54074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.686+0000 7f1f59c58700 1 --2- 172.21.15.45:0/3628484569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f54102e80 0x7f1f54074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.687+0000 7f1f5ac5a700 1 -- 172.21.15.45:0/3628484569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f54102e80 msgr2=0x7f1f54074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.687+0000 7f1f5ac5a700 1 --2- 172.21.15.45:0/3628484569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f54102e80 0x7f1f54074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.687+0000 7f1f5ac5a700 1 -- 172.21.15.45:0/3628484569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f541024e0 msgr2=0x7f1f54074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.687+0000 7f1f5ac5a700 1 --2- 172.21.15.45:0/3628484569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f541024e0 0x7f1f54074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.687+0000 7f1f5ac5a700 1 -- 172.21.15.45:0/3628484569 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f541a7030 con 0x7f1f541084f0 2024-06-06T04:30:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.687+0000 7f1f5ac5a700 1 --2- 172.21.15.45:0/3628484569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f541084f0 0x7f1f54075210 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0062a0 tx=0x7f1f4c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.687+0000 7f1f4b7fe700 1 -- 172.21.15.45:0/3628484569 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c021070 con 0x7f1f541084f0 2024-06-06T04:30:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.687+0000 7f1f4b7fe700 1 -- 172.21.15.45:0/3628484569 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f4c007e30 con 0x7f1f541084f0 2024-06-06T04:30:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.687+0000 7f1f5c6bd700 1 -- 172.21.15.45:0/3628484569 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f541a72c0 con 0x7f1f541084f0 2024-06-06T04:30:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.687+0000 7f1f4b7fe700 1 -- 172.21.15.45:0/3628484569 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c0046d0 con 0x7f1f541084f0 2024-06-06T04:30:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.687+0000 7f1f5c6bd700 1 -- 172.21.15.45:0/3628484569 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f541a77c0 con 0x7f1f541084f0 2024-06-06T04:30:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.688+0000 7f1f5c6bd700 1 -- 172.21.15.45:0/3628484569 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f54069180 con 0x7f1f541084f0 2024-06-06T04:30:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.689+0000 7f1f4b7fe700 1 -- 172.21.15.45:0/3628484569 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1f4c007980 con 0x7f1f541084f0 2024-06-06T04:30:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.690+0000 7f1f4b7fe700 1 --2- 172.21.15.45:0/3628484569 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1f400a2ca0 0x7f1f400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.690+0000 7f1f4b7fe700 1 -- 172.21.15.45:0/3628484569 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f1f4c00f020 con 0x7f1f541084f0 2024-06-06T04:30:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.690+0000 7f1f5a459700 1 --2- 172.21.15.45:0/3628484569 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1f400a2ca0 0x7f1f400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.690+0000 7f1f5a459700 1 --2- 172.21.15.45:0/3628484569 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1f400a2ca0 0x7f1f400a5160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009be0 tx=0x7f1f50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.694+0000 7f1f4b7fe700 1 -- 172.21.15.45:0/3628484569 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1f4c02a020 con 0x7f1f541084f0 2024-06-06T04:30:20.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.945+0000 7f1f5c6bd700 1 -- 172.21.15.45:0/3628484569 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f5402de60 con 0x7f1f541084f0 2024-06-06T04:30:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.946+0000 7f1f4b7fe700 1 -- 172.21.15.45:0/3628484569 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11858 (secure 0 0 0) 0x7f1f4c09d380 con 0x7f1f541084f0 2024-06-06T04:30:20.947 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:30:20.947 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T04:27:24.462897+0000","modified":"2024-06-06T04:28:32.515629+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24463,24473,24451]},"id":1}]} 2024-06-06T04:30:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.948+0000 7f1f5c6bd700 1 -- 172.21.15.45:0/3628484569 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1f400a2ca0 msgr2=0x7f1f400a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.948+0000 7f1f5c6bd700 1 --2- 172.21.15.45:0/3628484569 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1f400a2ca0 0x7f1f400a5160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009be0 tx=0x7f1f50009260 comp rx=0 tx=0).stop 2024-06-06T04:30:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.948+0000 7f1f5c6bd700 1 -- 172.21.15.45:0/3628484569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f541084f0 msgr2=0x7f1f54075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.948+0000 7f1f5c6bd700 1 --2- 172.21.15.45:0/3628484569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f541084f0 0x7f1f54075210 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0062a0 tx=0x7f1f4c004c80 comp rx=0 tx=0).stop 2024-06-06T04:30:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.948+0000 7f1f5c6bd700 1 -- 172.21.15.45:0/3628484569 shutdown_connections 2024-06-06T04:30:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.948+0000 7f1f5c6bd700 1 --2- 172.21.15.45:0/3628484569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f541084f0 0x7f1f54075210 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.948+0000 7f1f5c6bd700 1 --2- 172.21.15.45:0/3628484569 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1f400a2ca0 0x7f1f400a5160 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.948+0000 7f1f5c6bd700 1 --2- 172.21.15.45:0/3628484569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f541024e0 0x7f1f54074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.948+0000 7f1f5c6bd700 1 --2- 172.21.15.45:0/3628484569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f54102e80 0x7f1f54074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.948+0000 7f1f5c6bd700 1 -- 172.21.15.45:0/3628484569 >> 172.21.15.45:0/3628484569 conn(0x7f1f54079530 msgr2=0x7f1f54100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:20.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.948+0000 7f1f5c6bd700 1 -- 172.21.15.45:0/3628484569 shutdown_connections 2024-06-06T04:30:20.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:20.949+0000 7f1f5c6bd700 1 -- 172.21.15.45:0/3628484569 wait complete. 2024-06-06T04:30:20.950 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-06-06T04:30:21.700 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T04:30:21.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:21 smithi086 ceph-mon[36053]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-06T04:30:21.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:21 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3628484569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:30:21.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:21 smithi154 ceph-mon[36428]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-06T04:30:21.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:21 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3628484569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:30:21.956 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:21.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:21 smithi045 ceph-mon[32299]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-06T04:30:21.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:21 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3628484569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:30:23.700 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:23 smithi045 ceph-mon[32299]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:30:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.699+0000 7f2492b82700 1 -- 172.21.15.45:0/2570226077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248c107840 msgr2=0x7f248c10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.699+0000 7f2492b82700 1 --2- 172.21.15.45:0/2570226077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248c107840 0x7f248c10f3c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f24840099c0 tx=0x7f2484009cd0 comp rx=0 tx=0).stop 2024-06-06T04:30:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.699+0000 7f2492b82700 1 -- 172.21.15.45:0/2570226077 shutdown_connections 2024-06-06T04:30:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.699+0000 7f2492b82700 1 --2- 172.21.15.45:0/2570226077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248c107840 0x7f248c10f3c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.699+0000 7f2492b82700 1 --2- 172.21.15.45:0/2570226077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f248c102740 0x7f248c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.699+0000 7f2492b82700 1 --2- 172.21.15.45:0/2570226077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f248c1030f0 0x7f248c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.699+0000 7f2492b82700 1 -- 172.21.15.45:0/2570226077 >> 172.21.15.45:0/2570226077 conn(0x7f248c0795f0 msgr2=0x7f248c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.699+0000 7f2492b82700 1 -- 172.21.15.45:0/2570226077 shutdown_connections 2024-06-06T04:30:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.699+0000 7f2492b82700 1 -- 172.21.15.45:0/2570226077 wait complete. 2024-06-06T04:30:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f2492b82700 1 Processor -- start 2024-06-06T04:30:23.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f2492b82700 1 -- start start 2024-06-06T04:30:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f2492b82700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f248c102740 0x7f248c19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f2492b82700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248c1030f0 0x7f248c19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f2492b82700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f248c107840 0x7f248c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f2492b82700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f248c112610 con 0x7f248c1030f0 2024-06-06T04:30:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f2492b82700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f248c112490 con 0x7f248c102740 2024-06-06T04:30:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f2492b82700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f248c112790 con 0x7f248c107840 2024-06-06T04:30:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f248bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248c1030f0 0x7f248c19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f248bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248c1030f0 0x7f248c19ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52560/0 (socket says 172.21.15.45:52560) 2024-06-06T04:30:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f248bfff700 1 -- 172.21.15.45:0/1008475637 learned_addr learned my addr 172.21.15.45:0/1008475637 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f249091e700 1 --2- 172.21.15.45:0/1008475637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f248c102740 0x7f248c19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f249111f700 1 --2- 172.21.15.45:0/1008475637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f248c107840 0x7f248c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f249091e700 1 -- 172.21.15.45:0/1008475637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f248c107840 msgr2=0x7f248c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f249091e700 1 --2- 172.21.15.45:0/1008475637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f248c107840 0x7f248c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:23.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f249091e700 1 -- 172.21.15.45:0/1008475637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248c1030f0 msgr2=0x7f248c19ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f249091e700 1 --2- 172.21.15.45:0/1008475637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248c1030f0 0x7f248c19ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f249091e700 1 -- 172.21.15.45:0/1008475637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c199030 con 0x7f248c102740 2024-06-06T04:30:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.700+0000 7f248bfff700 1 --2- 172.21.15.45:0/1008475637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248c1030f0 0x7f248c19ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:30:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.701+0000 7f249091e700 1 --2- 172.21.15.45:0/1008475637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f248c102740 0x7f248c19e710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2474009c50 tx=0x7f2474009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.701+0000 7f2489ffb700 1 -- 172.21.15.45:0/1008475637 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247401e070 con 0x7f248c102740 2024-06-06T04:30:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.701+0000 7f2489ffb700 1 -- 172.21.15.45:0/1008475637 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2474019410 con 0x7f248c102740 2024-06-06T04:30:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.701+0000 7f2492b82700 1 -- 172.21.15.45:0/1008475637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248c199320 con 0x7f248c102740 2024-06-06T04:30:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.701+0000 7f2489ffb700 1 -- 172.21.15.45:0/1008475637 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474018610 con 0x7f248c102740 2024-06-06T04:30:23.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.701+0000 7f2492b82700 1 -- 172.21.15.45:0/1008475637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f248c0746c0 con 0x7f248c102740 2024-06-06T04:30:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.702+0000 7f2492b82700 1 -- 172.21.15.45:0/1008475637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f248c069180 con 0x7f248c102740 2024-06-06T04:30:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.703+0000 7f2489ffb700 1 -- 172.21.15.45:0/1008475637 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2474021070 con 0x7f248c102740 2024-06-06T04:30:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.704+0000 7f2489ffb700 1 --2- 172.21.15.45:0/1008475637 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f24780a2c20 0x7f24780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.704+0000 7f248bfff700 1 --2- 172.21.15.45:0/1008475637 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f24780a2c20 0x7f24780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.704+0000 7f2489ffb700 1 -- 172.21.15.45:0/1008475637 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f24740d29e0 con 0x7f248c102740 2024-06-06T04:30:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.704+0000 7f248bfff700 1 --2- 172.21.15.45:0/1008475637 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f24780a2c20 0x7f24780a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f248c1070c0 tx=0x7f247c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:23.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.708+0000 7f2489ffb700 1 -- 172.21.15.45:0/1008475637 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f247409f580 con 0x7f248c102740 2024-06-06T04:30:23.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:23 smithi086 ceph-mon[36053]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:30:23.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.914+0000 7f2492b82700 1 -- 172.21.15.45:0/1008475637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f248c199e00 con 0x7f248c102740 2024-06-06T04:30:23.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.915+0000 7f2489ffb700 1 -- 172.21.15.45:0/1008475637 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24789 (secure 0 0 0) 0x7f247409ee80 con 0x7f248c102740 2024-06-06T04:30:23.916 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:30:23.916 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":100,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","created":"2024-06-06T04:19:02.626916+0000","modified":"2024-06-06T04:29:05.901370+0000","last_up_change":"2024-06-06T04:26:17.390429+0000","last_in_change":"2024-06-06T04:25:56.938388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T04:22:28.797137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T04:27:17.750625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T04:27:21.486917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T04:27:31.029818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d3cee4a-5025-45b3-b217-082af37e79db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6803","nonce":2111759250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6805","nonce":2111759250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6809","nonce":2111759250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6807","nonce":2111759250}]},"public_addr":"172.21.15.45:6803/2111759250","cluster_addr":"172.21.15.45:6805/2111759250","heartbeat_back_addr":"172.21.15.45:6809/2111759250","heartbeat_front_addr":"172.21.15.45:6807/2111759250","state":["exists","up"]},{"osd":1,"uuid":"75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6801","nonce":3732225287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6803","nonce":3732225287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6807","nonce":3732225287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6805","nonce":3732225287}]},"public_addr":"172.21.15.86:6801/3732225287","cluster_addr":"172.21.15.86:6803/3732225287","heartbeat_back_addr":"172.21.15.86:6807/3732225287","heartbeat_front_addr":"172.21.15.86:6805/3732225287","state":["exists","up"]},{"osd":2,"uuid":"bb20565f-f9ae-47e3-9604-c69f737cff4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6801","nonce":2898783039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6803","nonce":2898783039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6807","nonce":2898783039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6805","nonce":2898783039}]},"public_addr":"172.21.15.154:6801/2898783039","cluster_addr":"172.21.15.154:6803/2898783039","heartbeat_back_addr":"172.21.15.154:6807/2898783039","heartbeat_front_addr":"172.21.15.154:6805/2898783039","state":["exists","up"]},{"osd":3,"uuid":"e1160799-7811-4326-9792-7e6c7d189c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6811","nonce":3436753135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6813","nonce":3436753135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6817","nonce":3436753135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6815","nonce":3436753135}]},"public_addr":"172.21.15.45:6811/3436753135","cluster_addr":"172.21.15.45:6813/3436753135","heartbeat_back_addr":"172.21.15.45:6817/3436753135","heartbeat_front_addr":"172.21.15.45:6815/3436753135","state":["exists","up"]},{"osd":4,"uuid":"6b389bc8-e19a-47f3-83ff-7b782d146d0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6809","nonce":3030235796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6811","nonce":3030235796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6815","nonce":3030235796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6813","nonce":3030235796}]},"public_addr":"172.21.15.86:6809/3030235796","cluster_addr":"172.21.15.86:6811/3030235796","heartbeat_back_addr":"172.21.15.86:6815/3030235796","heartbeat_front_addr":"172.21.15.86:6813/3030235796","state":["exists","up"]},{"osd":5,"uuid":"069de7b2-89bf-4150-bed7-7b8832573dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6809","nonce":2222268646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6811","nonce":2222268646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6815","nonce":2222268646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6813","nonce":2222268646}]},"public_addr":"172.21.15.154:6809/2222268646","cluster_addr":"172.21.15.154:6811/2222268646","heartbeat_back_addr":"172.21.15.154:6815/2222268646","heartbeat_front_addr":"172.21.15.154:6813/2222268646","state":["exists","up"]},{"osd":6,"uuid":"4c70b627-eafa-4e27-ba2a-4d8e1e335962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6819","nonce":384734939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6821","nonce":384734939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6825","nonce":384734939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6823","nonce":384734939}]},"public_addr":"172.21.15.45:6819/384734939","cluster_addr":"172.21.15.45:6821/384734939","heartbeat_back_addr":"172.21.15.45:6825/384734939","heartbeat_front_addr":"172.21.15.45:6823/384734939","state":["exists","up"]},{"osd":7,"uuid":"3b09b875-be52-4be0-b116-7fd9b0c26705","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6817","nonce":1119791005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6819","nonce":1119791005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6823","nonce":1119791005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6821","nonce":1119791005}]},"public_addr":"172.21.15.86:6817/1119791005","cluster_addr":"172.21.15.86:6819/1119791005","heartbeat_back_addr":"172.21.15.86:6823/1119791005","heartbeat_front_addr":"172.21.15.86:6821/1119791005","state":["exists","up"]},{"osd":8,"uuid":"cdf1dd1b-5392-4836-8e65-4c7c5b688d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6817","nonce":3054812496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6819","nonce":3054812496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6823","nonce":3054812496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6821","nonce":3054812496}]},"public_addr":"172.21.15.154:6817/3054812496","cluster_addr":"172.21.15.154:6819/3054812496","heartbeat_back_addr":"172.21.15.154:6823/3054812496","heartbeat_front_addr":"172.21.15.154:6821/3054812496","state":["exists","up"]},{"osd":9,"uuid":"acede92d-2cc1-4967-a357-7e5e201e99d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6827","nonce":1360655510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6829","nonce":1360655510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6833","nonce":1360655510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6831","nonce":1360655510}]},"public_addr":"172.21.15.45:6827/1360655510","cluster_addr":"172.21.15.45:6829/1360655510","heartbeat_back_addr":"172.21.15.45:6833/1360655510","heartbeat_front_addr":"172.21.15.45:6831/1360655510","state":["exists","up"]},{"osd":10,"uuid":"a2d0c488-1892-4da4-adbf-952c22df0ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6825","nonce":1701691877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6827","nonce":1701691877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6831","nonce":1701691877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6829","nonce":1701691877}]},"public_addr":"172.21.15.86:6825/1701691877","cluster_addr":"172.21.15.86:6827/1701691877","heartbeat_back_addr":"172.21.15.86:6831/1701691877","heartbeat_front_addr":"172.21.15.86:6829/1701691877","state":["exists","up"]},{"osd":11,"uuid":"6b349c4c-b672-480b-ac6a-8594dab0eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6825","nonce":2065648007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6827","nonce":2065648007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6831","nonce":2065648007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6829","nonce":2065648007}]},"public_addr":"172.21.15.154:6825/2065648007","cluster_addr":"172.21.15.154:6827/2065648007","heartbeat_back_addr":"172.21.15.154:6831/2065648007","heartbeat_front_addr":"172.21.15.154:6829/2065648007","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:21:37.147394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:00.776617+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:24.786849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:50.487455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:14.869272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:39.815158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:05.972619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:31.410063+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:56.795119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:23.114884+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:49.256504+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:26:15.251967+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/36732849":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/1500318775":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6801/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6800/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/2041380391":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/2504003341":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/3798552599":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/498952822":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6800/1562511056":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6801/1562511056":"2024-06-07T04:19:37.026199+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T04:30:23.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.918+0000 7f2492b82700 1 -- 172.21.15.45:0/1008475637 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f24780a2c20 msgr2=0x7f24780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.918+0000 7f2492b82700 1 --2- 172.21.15.45:0/1008475637 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f24780a2c20 0x7f24780a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f248c1070c0 tx=0x7f247c009040 comp rx=0 tx=0).stop 2024-06-06T04:30:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.918+0000 7f2492b82700 1 -- 172.21.15.45:0/1008475637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f248c102740 msgr2=0x7f248c19e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.918+0000 7f2492b82700 1 --2- 172.21.15.45:0/1008475637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f248c102740 0x7f248c19e710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2474009c50 tx=0x7f2474009f60 comp rx=0 tx=0).stop 2024-06-06T04:30:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.918+0000 7f2492b82700 1 -- 172.21.15.45:0/1008475637 shutdown_connections 2024-06-06T04:30:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.918+0000 7f2492b82700 1 --2- 172.21.15.45:0/1008475637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f248c107840 0x7f248c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.918+0000 7f2492b82700 1 --2- 172.21.15.45:0/1008475637 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f24780a2c20 0x7f24780a50e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.918+0000 7f2492b82700 1 --2- 172.21.15.45:0/1008475637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f248c102740 0x7f248c19e710 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.918+0000 7f2492b82700 1 --2- 172.21.15.45:0/1008475637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f248c1030f0 0x7f248c19ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.918+0000 7f2492b82700 1 -- 172.21.15.45:0/1008475637 >> 172.21.15.45:0/1008475637 conn(0x7f248c0795f0 msgr2=0x7f248c100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.918+0000 7f2492b82700 1 -- 172.21.15.45:0/1008475637 shutdown_connections 2024-06-06T04:30:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:23.918+0000 7f2492b82700 1 -- 172.21.15.45:0/1008475637 wait complete. 2024-06-06T04:30:23.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:23 smithi154 ceph-mon[36428]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:30:24.683 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:30:24.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:24 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1008475637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:30:24.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:24 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1008475637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:30:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:24.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:24 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1008475637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:30:25.677 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:25 smithi045 ceph-mon[32299]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:25.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:25 smithi086 ceph-mon[36053]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:25.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:25 smithi154 ceph-mon[36428]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:26.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.775+0000 7f0e5cdd2700 1 -- 172.21.15.45:0/1017493867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e58102740 msgr2=0x7f0e58102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:26.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.775+0000 7f0e5cdd2700 1 --2- 172.21.15.45:0/1017493867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e58102740 0x7f0e58102b20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0e480099c0 tx=0x7f0e48009cd0 comp rx=0 tx=0).stop 2024-06-06T04:30:26.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.775+0000 7f0e5cdd2700 1 -- 172.21.15.45:0/1017493867 shutdown_connections 2024-06-06T04:30:26.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.775+0000 7f0e5cdd2700 1 --2- 172.21.15.45:0/1017493867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e58107840 0x7f0e5810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:26.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.775+0000 7f0e5cdd2700 1 --2- 172.21.15.45:0/1017493867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e58102740 0x7f0e58102b20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:26.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.775+0000 7f0e5cdd2700 1 --2- 172.21.15.45:0/1017493867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e581030f0 0x7f0e58107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:26.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.775+0000 7f0e5cdd2700 1 -- 172.21.15.45:0/1017493867 >> 172.21.15.45:0/1017493867 conn(0x7f0e580795f0 msgr2=0x7f0e58079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:26.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.775+0000 7f0e5cdd2700 1 -- 172.21.15.45:0/1017493867 shutdown_connections 2024-06-06T04:30:26.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.775+0000 7f0e5cdd2700 1 -- 172.21.15.45:0/1017493867 wait complete. 2024-06-06T04:30:26.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.776+0000 7f0e5cdd2700 1 Processor -- start 2024-06-06T04:30:26.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.776+0000 7f0e5cdd2700 1 -- start start 2024-06-06T04:30:26.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.776+0000 7f0e5cdd2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e58102740 0x7f0e58074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:26.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.776+0000 7f0e5cdd2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e581030f0 0x7f0e58074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:26.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.776+0000 7f0e5cdd2700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e58107840 0x7f0e58075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:26.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.776+0000 7f0e5cdd2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0e58112690 con 0x7f0e581030f0 2024-06-06T04:30:26.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.776+0000 7f0e5cdd2700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e58112510 con 0x7f0e58107840 2024-06-06T04:30:26.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.776+0000 7f0e5cdd2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0e58112810 con 0x7f0e58102740 2024-06-06T04:30:26.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.776+0000 7f0e55d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e581030f0 0x7f0e58074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:26.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.776+0000 7f0e55d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e581030f0 0x7f0e58074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52574/0 (socket says 172.21.15.45:52574) 2024-06-06T04:30:26.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.776+0000 7f0e55d9b700 1 -- 172.21.15.45:0/2970262166 learned_addr learned my addr 172.21.15.45:0/2970262166 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:26.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.776+0000 7f0e5659c700 1 --2- 172.21.15.45:0/2970262166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e58102740 0x7f0e58074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:26.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.776+0000 7f0e56d9d700 1 --2- 172.21.15.45:0/2970262166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e58107840 0x7f0e58075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:26.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.777+0000 7f0e55d9b700 1 -- 172.21.15.45:0/2970262166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e58102740 msgr2=0x7f0e58074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:26.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.777+0000 7f0e55d9b700 1 --2- 172.21.15.45:0/2970262166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e58102740 0x7f0e58074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:26.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.777+0000 7f0e55d9b700 1 -- 172.21.15.45:0/2970262166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e58107840 msgr2=0x7f0e58075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:26.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.777+0000 7f0e55d9b700 1 --2- 172.21.15.45:0/2970262166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e58107840 0x7f0e58075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:26.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.777+0000 7f0e55d9b700 1 -- 172.21.15.45:0/2970262166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e581a7070 con 0x7f0e581030f0 2024-06-06T04:30:26.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.777+0000 7f0e56d9d700 1 --2- 172.21.15.45:0/2970262166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e58107840 0x7f0e58075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:30:26.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.777+0000 7f0e55d9b700 1 --2- 172.21.15.45:0/2970262166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e581030f0 0x7f0e58074cd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00d950 tx=0x7f0e4c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:26.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.777+0000 7f0e477fe700 1 -- 172.21.15.45:0/2970262166 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c016070 con 0x7f0e581030f0 2024-06-06T04:30:26.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.777+0000 7f0e5cdd2700 1 -- 172.21.15.45:0/2970262166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e581a7360 con 0x7f0e581030f0 2024-06-06T04:30:26.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.777+0000 7f0e477fe700 1 -- 172.21.15.45:0/2970262166 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e4c010460 con 0x7f0e581030f0 2024-06-06T04:30:26.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.777+0000 7f0e5cdd2700 1 -- 172.21.15.45:0/2970262166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e581a78a0 con 0x7f0e581030f0 2024-06-06T04:30:26.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.777+0000 7f0e477fe700 1 -- 172.21.15.45:0/2970262166 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c00f620 con 0x7f0e581030f0 2024-06-06T04:30:26.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.778+0000 7f0e5cdd2700 1 -- 172.21.15.45:0/2970262166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e5810aa80 con 0x7f0e581030f0 2024-06-06T04:30:26.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.779+0000 7f0e477fe700 1 -- 172.21.15.45:0/2970262166 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0e4c019070 con 0x7f0e581030f0 2024-06-06T04:30:26.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.780+0000 7f0e477fe700 1 --2- 172.21.15.45:0/2970262166 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0e400a2c20 0x7f0e400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:26.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.780+0000 7f0e5659c700 1 --2- 172.21.15.45:0/2970262166 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0e400a2c20 0x7f0e400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:26.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.780+0000 7f0e477fe700 1 -- 172.21.15.45:0/2970262166 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f0e4c0cf0b0 con 0x7f0e581030f0 2024-06-06T04:30:26.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.780+0000 7f0e5659c700 1 --2- 172.21.15.45:0/2970262166 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0e400a2c20 0x7f0e400a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0e48000c00 tx=0x7f0e48005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:26.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:26.783+0000 7f0e477fe700 1 -- 172.21.15.45:0/2970262166 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0e4c09bbf0 con 0x7f0e581030f0 2024-06-06T04:30:27.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:27.033+0000 7f0e5cdd2700 1 -- 172.21.15.45:0/2970262166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e58069180 con 0x7f0e581030f0 2024-06-06T04:30:27.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:27.033+0000 7f0e477fe700 1 -- 172.21.15.45:0/2970262166 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11858 (secure 0 0 0) 0x7f0e4c09b4f0 con 0x7f0e581030f0 2024-06-06T04:30:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:30:27.034 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T04:27:24.462897+0000","modified":"2024-06-06T04:28:32.515629+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24463,24473,24451]},"id":1}]} 2024-06-06T04:30:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:27.035+0000 7f0e5cdd2700 1 -- 172.21.15.45:0/2970262166 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0e400a2c20 msgr2=0x7f0e400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:27.035+0000 7f0e5cdd2700 1 --2- 172.21.15.45:0/2970262166 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0e400a2c20 0x7f0e400a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0e48000c00 tx=0x7f0e48005fd0 comp rx=0 tx=0).stop 2024-06-06T04:30:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:27.035+0000 7f0e5cdd2700 1 -- 172.21.15.45:0/2970262166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e581030f0 msgr2=0x7f0e58074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:27.035+0000 7f0e5cdd2700 1 --2- 172.21.15.45:0/2970262166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e581030f0 0x7f0e58074cd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00d950 tx=0x7f0e4c00dd10 comp rx=0 tx=0).stop 2024-06-06T04:30:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:27.035+0000 7f0e5cdd2700 1 -- 172.21.15.45:0/2970262166 shutdown_connections 2024-06-06T04:30:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:27.035+0000 7f0e5cdd2700 1 --2- 172.21.15.45:0/2970262166 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0e400a2c20 0x7f0e400a50e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:27.036+0000 7f0e5cdd2700 1 --2- 172.21.15.45:0/2970262166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e58107840 0x7f0e58075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:27.036+0000 7f0e5cdd2700 1 --2- 172.21.15.45:0/2970262166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e58102740 0x7f0e58074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:27.036+0000 7f0e5cdd2700 1 --2- 172.21.15.45:0/2970262166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e581030f0 0x7f0e58074cd0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:27.036+0000 7f0e5cdd2700 1 -- 172.21.15.45:0/2970262166 >> 172.21.15.45:0/2970262166 conn(0x7f0e580795f0 msgr2=0x7f0e581007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:27.036+0000 7f0e5cdd2700 1 -- 172.21.15.45:0/2970262166 shutdown_connections 2024-06-06T04:30:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:27.036+0000 7f0e5cdd2700 1 -- 172.21.15.45:0/2970262166 wait complete. 2024-06-06T04:30:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-06-06T04:30:27.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:27 smithi045 ceph-mon[32299]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:30:27.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:27 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2970262166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:30:27.681 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-06T04:27:24.462897+0000', 'modified': '2024-06-06T04:28:32.515629+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24473, 'mds_1': 24463, 'mds_2': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6835/4268654313', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 4268654313}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 4268654313}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.154:6835/954391764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 954391764}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 954391764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6837/414874753', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 414874753}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 414874753}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.154:6837/1140071744', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 1140071744}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 1140071744}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.154:6839/982197202', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6838', 'nonce': 982197202}, {'type': 'v1', 'addr': '172.21.15.154:6839', 'nonce': 982197202}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.86:6839/816791193', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 816791193}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 816791193}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24473, 'qdb_cluster': [24463, 24473, 24451]} 2024-06-06T04:30:27.682 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-06T04:30:27.682 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-06T04:30:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:27 smithi086 ceph-mon[36053]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:30:27.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:27 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2970262166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:30:27.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:27 smithi154 ceph-mon[36428]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:30:27.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:27 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2970262166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:30:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:29.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.525+0000 7fb108f38700 1 -- 172.21.15.45:0/372798219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb10406a170 msgr2=0x7fb10406a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:29.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.525+0000 7fb108f38700 1 --2- 172.21.15.45:0/372798219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb10406a170 0x7fb10406a5f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009990 tx=0x7fb0ec009ca0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.525+0000 7fb108f38700 1 -- 172.21.15.45:0/372798219 shutdown_connections 2024-06-06T04:30:29.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.525+0000 7fb108f38700 1 --2- 172.21.15.45:0/372798219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb10406ab30 0x7fb10410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.525+0000 7fb108f38700 1 --2- 172.21.15.45:0/372798219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb104108500 0x7fb1041088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.525+0000 7fb108f38700 1 --2- 172.21.15.45:0/372798219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb10406a170 0x7fb10406a5f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.525+0000 7fb108f38700 1 -- 172.21.15.45:0/372798219 >> 172.21.15.45:0/372798219 conn(0x7fb104078340 msgr2=0x7fb104078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:29.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.525+0000 7fb108f38700 1 -- 172.21.15.45:0/372798219 shutdown_connections 2024-06-06T04:30:29.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.525+0000 7fb108f38700 1 -- 172.21.15.45:0/372798219 wait complete. 2024-06-06T04:30:29.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.526+0000 7fb108f38700 1 Processor -- start 2024-06-06T04:30:29.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.526+0000 7fb108f38700 1 -- start start 2024-06-06T04:30:29.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.527+0000 7fb108f38700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb10406a170 0x7fb10419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.527+0000 7fb108f38700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10406ab30 0x7fb10419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.527+0000 7fb108f38700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb104108500 0x7fb104198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.527+0000 7fb108f38700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb104112630 con 0x7fb104108500 2024-06-06T04:30:29.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.527+0000 7fb108f38700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041124b0 con 0x7fb10406ab30 2024-06-06T04:30:29.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.527+0000 7fb108f38700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041127b0 con 0x7fb10406a170 2024-06-06T04:30:29.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.527+0000 7fb102d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb104108500 0x7fb104198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.527+0000 7fb102d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb104108500 0x7fb104198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37716/0 (socket says 172.21.15.45:37716) 2024-06-06T04:30:29.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.527+0000 7fb102d9d700 1 -- 172.21.15.45:0/1343517826 learned_addr learned my addr 172.21.15.45:0/1343517826 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:29.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.527+0000 7fb10259c700 1 --2- 172.21.15.45:0/1343517826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb10406a170 0x7fb10419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.527+0000 7fb0f9d9b700 1 --2- 172.21.15.45:0/1343517826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10406ab30 0x7fb10419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.527+0000 7fb102d9d700 1 -- 172.21.15.45:0/1343517826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb10406a170 msgr2=0x7fb10419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:29.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.527+0000 7fb102d9d700 1 --2- 172.21.15.45:0/1343517826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb10406a170 0x7fb10419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.527+0000 7fb102d9d700 1 -- 172.21.15.45:0/1343517826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10406ab30 msgr2=0x7fb10419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:29.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.527+0000 7fb102d9d700 1 --2- 172.21.15.45:0/1343517826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10406ab30 0x7fb10419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.527+0000 7fb102d9d700 1 -- 172.21.15.45:0/1343517826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1041990f0 con 0x7fb104108500 2024-06-06T04:30:29.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.528+0000 7fb102d9d700 1 --2- 172.21.15.45:0/1343517826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb104108500 0x7fb104198990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc007ee0 tx=0x7fb0fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:29.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.528+0000 7fb0fbfff700 1 -- 172.21.15.45:0/1343517826 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc004bb0 con 0x7fb104108500 2024-06-06T04:30:29.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.528+0000 7fb0fbfff700 1 -- 172.21.15.45:0/1343517826 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0fc004d50 con 0x7fb104108500 2024-06-06T04:30:29.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.528+0000 7fb108f38700 1 -- 172.21.15.45:0/1343517826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1041993e0 con 0x7fb104108500 2024-06-06T04:30:29.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.528+0000 7fb0fbfff700 1 -- 172.21.15.45:0/1343517826 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0058e0 con 0x7fb104108500 2024-06-06T04:30:29.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.528+0000 7fb108f38700 1 -- 172.21.15.45:0/1343517826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1041a5650 con 0x7fb104108500 2024-06-06T04:30:29.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.531+0000 7fb0fbfff700 1 -- 172.21.15.45:0/1343517826 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb0fc01a020 con 0x7fb104108500 2024-06-06T04:30:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.532+0000 7fb0fbfff700 1 --2- 172.21.15.45:0/1343517826 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb0f00a2c20 0x7fb0f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.532+0000 7fb10259c700 1 --2- 172.21.15.45:0/1343517826 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb0f00a2c20 0x7fb0f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.532+0000 7fb0fbfff700 1 -- 172.21.15.45:0/1343517826 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fb0fc0ceb00 con 0x7fb104108500 2024-06-06T04:30:29.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.532+0000 7fb10259c700 1 --2- 172.21.15.45:0/1343517826 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb0f00a2c20 0x7fb0f00a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb1040ffa60 tx=0x7fb0f400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:29.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.532+0000 7fb1137bbb80 1 Processor -- start 2024-06-06T04:30:29.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb1137bbb80 1 -- start start 2024-06-06T04:30:29.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb1137bbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b52f80 0x55afe0b53360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb1137bbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b538a0 0x55afe0b576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb1137bbb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b57be0 0x55afe0b58040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb1137bbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55afe0aa07c0 con 0x55afe0b52f80 2024-06-06T04:30:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb1137bbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55afe0a486d0 con 0x55afe0b538a0 2024-06-06T04:30:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb1137bbb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55afe0a9c780 con 0x55afe0b57be0 2024-06-06T04:30:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb10259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b52f80 0x55afe0b53360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb10259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b52f80 0x55afe0b53360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37724/0 (socket says 172.21.15.45:37724) 2024-06-06T04:30:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb10259c700 1 -- 172.21.15.45:0/1595063177 learned_addr learned my addr 172.21.15.45:0/1595063177 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb102d9d700 1 --2- 172.21.15.45:0/1595063177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b57be0 0x55afe0b58040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb10259c700 1 -- 172.21.15.45:0/1595063177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b57be0 msgr2=0x55afe0b58040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb10259c700 1 --2- 172.21.15.45:0/1595063177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b57be0 0x55afe0b58040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb0f9d9b700 1 --2- 172.21.15.45:0/1595063177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b538a0 0x55afe0b576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb10259c700 1 -- 172.21.15.45:0/1595063177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b538a0 msgr2=0x55afe0b576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb10259c700 1 --2- 172.21.15.45:0/1595063177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b538a0 0x55afe0b576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb10259c700 1 -- 172.21.15.45:0/1595063177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afe0a3ef00 con 0x55afe0b52f80 2024-06-06T04:30:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb0f9d9b700 1 --2- 172.21.15.45:0/1595063177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b538a0 0x55afe0b576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:30:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.533+0000 7fb10259c700 1 --2- 172.21.15.45:0/1595063177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b52f80 0x55afe0b53360 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4010e00 tx=0x7fb0f4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdd76e0a9dbfffa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.534+0000 7fb0f8d99700 1 -- 172.21.15.45:0/1595063177 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f401e420 con 0x55afe0b52f80 2024-06-06T04:30:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.534+0000 7fb0f8d99700 1 -- 172.21.15.45:0/1595063177 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0f401ea70 con 0x55afe0b52f80 2024-06-06T04:30:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.534+0000 7fb0f8d99700 1 -- 172.21.15.45:0/1595063177 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f401cac0 con 0x55afe0b52f80 2024-06-06T04:30:29.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.534+0000 7fb1137bbb80 1 -- 172.21.15.45:0/1595063177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b52f80 msgr2=0x55afe0b53360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:29.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.534+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/1595063177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b52f80 0x55afe0b53360 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4010e00 tx=0x7fb0f4013000 comp rx=0 tx=0).stop 2024-06-06T04:30:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.534+0000 7fb1137bbb80 1 -- 172.21.15.45:0/1595063177 shutdown_connections 2024-06-06T04:30:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.535+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/1595063177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b57be0 0x55afe0b58040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.535+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/1595063177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b52f80 0x55afe0b53360 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.535+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/1595063177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b538a0 0x55afe0b576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.535+0000 7fb1137bbb80 1 -- 172.21.15.45:0/1595063177 >> 172.21.15.45:0/1595063177 conn(0x55afe0a79c70 msgr2=0x55afe0b49800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.535+0000 7fb1137bbb80 1 -- 172.21.15.45:0/1595063177 shutdown_connections 2024-06-06T04:30:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.535+0000 7fb1137bbb80 1 -- 172.21.15.45:0/1595063177 wait complete. 2024-06-06T04:30:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.535+0000 7fb1137bbb80 1 -- start start 2024-06-06T04:30:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.535+0000 7fb1137bbb80 1 Processor -- start 2024-06-06T04:30:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.535+0000 7fb1137bbb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b52f80 0x55afe0b5f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.535+0000 7fb1137bbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b5c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.535+0000 7fb1137bbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b57be0 0x55afe0b5ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.535+0000 7fb1137bbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55afe0b5fd80 con 0x55afe0b538a0 2024-06-06T04:30:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.535+0000 7fb1137bbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55afe0a40ba0 con 0x55afe0b57be0 2024-06-06T04:30:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.535+0000 7fb1137bbb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55afe0b5ff00 con 0x55afe0b52f80 2024-06-06T04:30:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.535+0000 7fb1137bbb80 10 client.0 _ll_register_callbacks cb 0x55afe0a3a8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:30:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb0f9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b5c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb0f9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b5c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37730/0 (socket says 172.21.15.45:37730) 2024-06-06T04:30:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb0f9d9b700 1 -- 172.21.15.45:0/2577074444 learned_addr learned my addr 172.21.15.45:0/2577074444 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb102d9d700 1 --2- 172.21.15.45:0/2577074444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b57be0 0x55afe0b5ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb10259c700 1 --2- 172.21.15.45:0/2577074444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b52f80 0x55afe0b5f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb0f9d9b700 1 -- 172.21.15.45:0/2577074444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b52f80 msgr2=0x55afe0b5f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb0f9d9b700 1 --2- 172.21.15.45:0/2577074444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b52f80 0x55afe0b5f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb0f9d9b700 1 -- 172.21.15.45:0/2577074444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b57be0 msgr2=0x55afe0b5ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb0f9d9b700 1 --2- 172.21.15.45:0/2577074444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b57be0 0x55afe0b5ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb0f9d9b700 1 -- 172.21.15.45:0/2577074444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afe09ef880 con 0x55afe0b538a0 2024-06-06T04:30:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb0f9d9b700 1 --2- 172.21.15.45:0/2577074444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b5c540 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009990 tx=0x7fb0ec009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7814ede0ebc8a975 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb0e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:30:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb0e9ffb700 1 -- 172.21.15.45:0/2577074444 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec003680 con 0x55afe0b538a0 2024-06-06T04:30:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb1137bbb80 1 -- 172.21.15.45:0/2577074444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0f401ea70 con 0x55afe0b538a0 2024-06-06T04:30:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb0e9ffb700 1 -- 172.21.15.45:0/2577074444 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fb0ec0054e0 con 0x55afe0b538a0 2024-06-06T04:30:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb0e9ffb700 1 -- 172.21.15.45:0/2577074444 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb0ec02f050 con 0x55afe0b538a0 2024-06-06T04:30:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb1137bbb80 10 client.15048 fetch_fsmap learned FSMap version 26 2024-06-06T04:30:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.536+0000 7fb1137bbb80 1 -- 172.21.15.45:0/2577074444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55afe0b6a730 con 0x55afe0b538a0 2024-06-06T04:30:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.537+0000 7fb0e9ffb700 1 -- 172.21.15.45:0/2577074444 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0f401ea70 con 0x55afe0b538a0 2024-06-06T04:30:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.537+0000 7fb0e9ffb700 1 -- 172.21.15.45:0/2577074444 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb0ec02a030 con 0x55afe0b538a0 2024-06-06T04:30:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.537+0000 7fb1137bbb80 10 client.15048 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T04:30:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.537+0000 7fb1137bbb80 10 client.15048 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-06T04:30:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.537+0000 7fb1137bbb80 20 client.15048 populate_metadata read hostname 'smithi045' 2024-06-06T04:30:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.537+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/2577074444 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55afe0b6cdb0 0x55afe0b6e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.537+0000 7fb1137bbb80 4 client.15048 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:30:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.537+0000 7fb1137bbb80 1 -- 172.21.15.45:0/2577074444 --> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afe0aadcc0 con 0x55afe0b6cdb0 2024-06-06T04:30:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.537+0000 7fb10259c700 1 --2- 172.21.15.45:0/2577074444 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55afe0b6cdb0 0x55afe0b6e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.538+0000 7fb10259c700 1 --2- 172.21.15.45:0/2577074444 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55afe0b6cdb0 0x55afe0b6e220 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=afe22d9694b7f29b server_cookie=a11f44c3933c9d79 in_seq=0 out_seq=0 2024-06-06T04:30:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.538+0000 7fb0e9ffb700 10 client.15048 ms_handle_connect on v2:172.21.15.154:6836/1140071744 2024-06-06T04:30:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.538+0000 7fb0e9ffb700 1 -- 172.21.15.45:0/2577074444 <== mds.2 v2:172.21.15.154:6836/1140071744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55afe0aadcc0 con 0x55afe0b6cdb0 2024-06-06T04:30:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.538+0000 7fb0e9ffb700 10 client.15048 handle_command_reply: tid=0 2024-06-06T04:30:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.539+0000 7fb1137bbb80 1 client.15048 shutdown 2024-06-06T04:30:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.539+0000 7fb1137bbb80 1 -- 172.21.15.45:0/2577074444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 msgr2=0x55afe0b5c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.539+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/2577074444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b5c540 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009990 tx=0x7fb0ec009ee0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.539+0000 7fb1137bbb80 1 -- 172.21.15.45:0/2577074444 shutdown_connections 2024-06-06T04:30:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.539+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/2577074444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b57be0 0x55afe0b5ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.539+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/2577074444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b52f80 0x55afe0b5f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.539+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/2577074444 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55afe0b6cdb0 0x55afe0b6e220 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.539+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/2577074444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b5c540 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.539+0000 7fb0e9ffb700 0 client.15048 ms_handle_reset on v2:172.21.15.154:6836/1140071744 2024-06-06T04:30:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.539+0000 7fb1137bbb80 1 -- 172.21.15.45:0/2577074444 >> 172.21.15.45:0/2577074444 conn(0x55afe0a79c70 msgr2=0x55afe0b4d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.539+0000 7fb1137bbb80 1 -- 172.21.15.45:0/2577074444 shutdown_connections 2024-06-06T04:30:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.539+0000 7fb1137bbb80 1 -- 172.21.15.45:0/2577074444 wait complete. 2024-06-06T04:30:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.539+0000 7fb1137bbb80 20 client.15048 trim_cache size 0 max 16384 2024-06-06T04:30:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.561+0000 7fb1137bbb80 1 Processor -- start 2024-06-06T04:30:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.561+0000 7fb1137bbb80 1 -- start start 2024-06-06T04:30:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.561+0000 7fb1137bbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b52f80 0x55afe0b5f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.561+0000 7fb1137bbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b77680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.561+0000 7fb1137bbb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b57be0 0x55afe0b77bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.561+0000 7fb1137bbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55afe0b6c560 con 0x55afe0b538a0 2024-06-06T04:30:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.561+0000 7fb1137bbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55afe0b6cc30 con 0x55afe0b52f80 2024-06-06T04:30:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.561+0000 7fb1137bbb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55afe0b6d300 con 0x55afe0b57be0 2024-06-06T04:30:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.561+0000 7fb0f9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b77680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.561+0000 7fb0f9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b77680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37742/0 (socket says 172.21.15.45:37742) 2024-06-06T04:30:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.561+0000 7fb0f9d9b700 1 -- 172.21.15.45:0/670852733 learned_addr learned my addr 172.21.15.45:0/670852733 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.561+0000 7fb10259c700 1 --2- 172.21.15.45:0/670852733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b52f80 0x55afe0b5f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.562+0000 7fb102d9d700 1 --2- 172.21.15.45:0/670852733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b57be0 0x55afe0b77bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.562+0000 7fb0f9d9b700 1 -- 172.21.15.45:0/670852733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b57be0 msgr2=0x55afe0b77bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.562+0000 7fb0f9d9b700 1 --2- 172.21.15.45:0/670852733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b57be0 0x55afe0b77bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.562+0000 7fb0f9d9b700 1 -- 172.21.15.45:0/670852733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b52f80 msgr2=0x55afe0b5f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:29.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.562+0000 7fb0f9d9b700 1 --2- 172.21.15.45:0/670852733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b52f80 0x55afe0b5f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.562+0000 7fb0f9d9b700 1 -- 172.21.15.45:0/670852733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afe0b5aab0 con 0x55afe0b538a0 2024-06-06T04:30:29.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.562+0000 7fb0f9d9b700 1 --2- 172.21.15.45:0/670852733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b77680 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00c010 tx=0x7fb0ec009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28d96f91132d9e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:29.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.562+0000 7fb0ebfff700 1 -- 172.21.15.45:0/670852733 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec037020 con 0x55afe0b538a0 2024-06-06T04:30:29.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.562+0000 7fb0ebfff700 1 -- 172.21.15.45:0/670852733 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0ec005bc0 con 0x55afe0b538a0 2024-06-06T04:30:29.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.562+0000 7fb0ebfff700 1 -- 172.21.15.45:0/670852733 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec003a70 con 0x55afe0b538a0 2024-06-06T04:30:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 -- 172.21.15.45:0/670852733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 msgr2=0x55afe0b77680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/670852733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b77680 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00c010 tx=0x7fb0ec009310 comp rx=0 tx=0).stop 2024-06-06T04:30:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 -- 172.21.15.45:0/670852733 shutdown_connections 2024-06-06T04:30:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/670852733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b57be0 0x55afe0b77bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/670852733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b52f80 0x55afe0b5f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/670852733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b77680 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 -- 172.21.15.45:0/670852733 >> 172.21.15.45:0/670852733 conn(0x55afe0a79c70 msgr2=0x55afe0b5ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 -- 172.21.15.45:0/670852733 shutdown_connections 2024-06-06T04:30:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 -- 172.21.15.45:0/670852733 wait complete. 2024-06-06T04:30:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 -- start start 2024-06-06T04:30:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 Processor -- start 2024-06-06T04:30:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b52f80 0x55afe0b776a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b77be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b57be0 0x55afe0b72650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55afe0b6e0a0 con 0x55afe0b538a0 2024-06-06T04:30:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55afe0b6d9d0 con 0x55afe0b57be0 2024-06-06T04:30:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.563+0000 7fb1137bbb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55afe0b6e770 con 0x55afe0b52f80 2024-06-06T04:30:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb0f9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b77be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb1137bbb80 10 client.0 _ll_register_callbacks cb 0x55afe0a3a8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:30:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb0f9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b77be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37744/0 (socket says 172.21.15.45:37744) 2024-06-06T04:30:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb0f9d9b700 1 -- 172.21.15.45:0/1521016029 learned_addr learned my addr 172.21.15.45:0/1521016029 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb102d9d700 1 --2- 172.21.15.45:0/1521016029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b57be0 0x55afe0b72650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb10259c700 1 --2- 172.21.15.45:0/1521016029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b52f80 0x55afe0b776a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb0f9d9b700 1 -- 172.21.15.45:0/1521016029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b52f80 msgr2=0x55afe0b776a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb0f9d9b700 1 --2- 172.21.15.45:0/1521016029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b52f80 0x55afe0b776a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb0f9d9b700 1 -- 172.21.15.45:0/1521016029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b57be0 msgr2=0x55afe0b72650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb0f9d9b700 1 --2- 172.21.15.45:0/1521016029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b57be0 0x55afe0b72650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb0f9d9b700 1 -- 172.21.15.45:0/1521016029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afe0b72b90 con 0x55afe0b538a0 2024-06-06T04:30:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb0f9d9b700 1 --2- 172.21.15.45:0/1521016029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b77be0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec006010 tx=0x7fb0ec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a645a60e341de1ca server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb0e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:30:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb0e9ffb700 1 -- 172.21.15.45:0/1521016029 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec037070 con 0x55afe0b538a0 2024-06-06T04:30:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb0e9ffb700 1 -- 172.21.15.45:0/1521016029 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fb0ec0054e0 con 0x55afe0b538a0 2024-06-06T04:30:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb1137bbb80 1 -- 172.21.15.45:0/1521016029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0ec005bc0 con 0x55afe0b538a0 2024-06-06T04:30:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb0e9ffb700 1 -- 172.21.15.45:0/1521016029 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb0ec04c050 con 0x55afe0b538a0 2024-06-06T04:30:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb1137bbb80 10 client.15054 fetch_fsmap learned FSMap version 26 2024-06-06T04:30:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.564+0000 7fb1137bbb80 1 -- 172.21.15.45:0/1521016029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55afe0bd8d20 con 0x55afe0b538a0 2024-06-06T04:30:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.565+0000 7fb0e9ffb700 1 -- 172.21.15.45:0/1521016029 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0ec005bc0 con 0x55afe0b538a0 2024-06-06T04:30:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.565+0000 7fb0e9ffb700 1 -- 172.21.15.45:0/1521016029 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb0ec02a040 con 0x55afe0b538a0 2024-06-06T04:30:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.565+0000 7fb1137bbb80 10 client.15054 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T04:30:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.565+0000 7fb1137bbb80 10 client.15054 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-06T04:30:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.565+0000 7fb1137bbb80 20 client.15054 populate_metadata read hostname 'smithi045' 2024-06-06T04:30:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.565+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/1521016029 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55afe0bdd5b0 0x55afe0bdc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.565+0000 7fb1137bbb80 4 client.15054 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T04:30:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.565+0000 7fb1137bbb80 1 -- 172.21.15.45:0/1521016029 --> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] -- command(tid 0: {"prefix": "status"}) -- 0x55afe0aa8240 con 0x55afe0bdd5b0 2024-06-06T04:30:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.565+0000 7fb10259c700 1 --2- 172.21.15.45:0/1521016029 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55afe0bdd5b0 0x55afe0bdc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb10259c700 1 --2- 172.21.15.45:0/1521016029 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55afe0bdd5b0 0x55afe0bdc2c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dbb3fb039b7d0d02 server_cookie=4a127ef6e0526afa in_seq=0 out_seq=0 2024-06-06T04:30:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb0e9ffb700 10 client.15054 ms_handle_connect on v2:172.21.15.154:6836/1140071744 2024-06-06T04:30:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb0e9ffb700 1 -- 172.21.15.45:0/1521016029 <== mds.2 v2:172.21.15.154:6836/1140071744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55afe0aa8240 con 0x55afe0bdd5b0 2024-06-06T04:30:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb0e9ffb700 10 client.15054 handle_command_reply: tid=0 2024-06-06T04:30:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb1137bbb80 1 client.15054 shutdown 2024-06-06T04:30:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb1137bbb80 1 -- 172.21.15.45:0/1521016029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 msgr2=0x55afe0b77be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/1521016029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b77be0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec006010 tx=0x7fb0ec000bc0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb1137bbb80 1 -- 172.21.15.45:0/1521016029 shutdown_connections 2024-06-06T04:30:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/1521016029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afe0b57be0 0x55afe0b72650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/1521016029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55afe0b52f80 0x55afe0b776a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/1521016029 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55afe0bdd5b0 0x55afe0bdc2c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb1137bbb80 1 --2- 172.21.15.45:0/1521016029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afe0b538a0 0x55afe0b77be0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb1137bbb80 1 -- 172.21.15.45:0/1521016029 >> 172.21.15.45:0/1521016029 conn(0x55afe0a79c70 msgr2=0x55afe0b42db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb0e9ffb700 0 client.15054 ms_handle_reset on v2:172.21.15.154:6836/1140071744 2024-06-06T04:30:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb1137bbb80 1 -- 172.21.15.45:0/1521016029 shutdown_connections 2024-06-06T04:30:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb1137bbb80 1 -- 172.21.15.45:0/1521016029 wait complete. 2024-06-06T04:30:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.566+0000 7fb1137bbb80 20 client.15054 trim_cache size 0 max 16384 2024-06-06T04:30:29.573 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:30:29.573 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "c4f6b938-23bb-11ef-bc9b-c7b262605968", 2024-06-06T04:30:29.573 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 2, 2024-06-06T04:30:29.573 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24451, 2024-06-06T04:30:29.573 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-06-06T04:30:29.573 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-06-06T04:30:29.573 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-06-06T04:30:29.573 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 121.856383251, 2024-06-06T04:30:29.573 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2024-06-06T04:30:29.573 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 100, 2024-06-06T04:30:29.573 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 98, 2024-06-06T04:30:29.574 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 216.34464288500001 2024-06-06T04:30:29.574 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:30:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.567+0000 7fb108f38700 1 -- 172.21.15.45:0/1343517826 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb0f00a2c20 msgr2=0x7fb0f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.567+0000 7fb108f38700 1 --2- 172.21.15.45:0/1343517826 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb0f00a2c20 0x7fb0f00a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb1040ffa60 tx=0x7fb0f400b410 comp rx=0 tx=0).stop 2024-06-06T04:30:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.567+0000 7fb108f38700 1 -- 172.21.15.45:0/1343517826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb104108500 msgr2=0x7fb104198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.567+0000 7fb108f38700 1 --2- 172.21.15.45:0/1343517826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb104108500 0x7fb104198990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc007ee0 tx=0x7fb0fc00d3b0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.567+0000 7fb108f38700 1 -- 172.21.15.45:0/1343517826 shutdown_connections 2024-06-06T04:30:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.567+0000 7fb108f38700 1 --2- 172.21.15.45:0/1343517826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb104108500 0x7fb104198990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.567+0000 7fb108f38700 1 --2- 172.21.15.45:0/1343517826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb10406a170 0x7fb10419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.567+0000 7fb108f38700 1 --2- 172.21.15.45:0/1343517826 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb0f00a2c20 0x7fb0f00a50e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.567+0000 7fb108f38700 1 --2- 172.21.15.45:0/1343517826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10406ab30 0x7fb10419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.567+0000 7fb108f38700 1 -- 172.21.15.45:0/1343517826 >> 172.21.15.45:0/1343517826 conn(0x7fb104078340 msgr2=0x7fb104100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.567+0000 7fb108f38700 1 -- 172.21.15.45:0/1343517826 shutdown_connections 2024-06-06T04:30:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:29.567+0000 7fb108f38700 1 -- 172.21.15.45:0/1343517826 wait complete. 2024-06-06T04:30:29.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:29 smithi086 ceph-mon[36053]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:30:29.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:29 smithi045 ceph-mon[32299]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:30:29.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:29 smithi154 ceph-mon[36428]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:30:30.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-06T04:30:30.515 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:31.810 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:31 smithi045 ceph-mon[32299]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:31.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:31 smithi086 ceph-mon[36053]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:31.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:31 smithi154 ceph-mon[36428]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:32.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.154+0000 7effb4766700 1 -- 172.21.15.45:0/2119260804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effac1073f0 msgr2=0x7effac1077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:32.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.154+0000 7effb4766700 1 --2- 172.21.15.45:0/2119260804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effac1073f0 0x7effac1077d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7effa8009990 tx=0x7effa8009ca0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.154+0000 7effb4766700 1 -- 172.21.15.45:0/2119260804 shutdown_connections 2024-06-06T04:30:32.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.154+0000 7effb4766700 1 --2- 172.21.15.45:0/2119260804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effac101d40 0x7effac10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.154+0000 7effb4766700 1 --2- 172.21.15.45:0/2119260804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effac1073f0 0x7effac1077d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.154+0000 7effb4766700 1 --2- 172.21.15.45:0/2119260804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effac101380 0x7effac101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.154+0000 7effb4766700 1 -- 172.21.15.45:0/2119260804 >> 172.21.15.45:0/2119260804 conn(0x7effac078300 msgr2=0x7effac078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:32.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.154+0000 7effb4766700 1 -- 172.21.15.45:0/2119260804 shutdown_connections 2024-06-06T04:30:32.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.154+0000 7effb4766700 1 -- 172.21.15.45:0/2119260804 wait complete. 2024-06-06T04:30:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.155+0000 7effb4766700 1 Processor -- start 2024-06-06T04:30:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.155+0000 7effb4766700 1 -- start start 2024-06-06T04:30:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.155+0000 7effb4766700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effac101380 0x7effac19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.155+0000 7effb4766700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effac101d40 0x7effac19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.155+0000 7effb4766700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effac1073f0 0x7effac197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.155+0000 7effb4766700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7effac111510 con 0x7effac1073f0 2024-06-06T04:30:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.155+0000 7effb4766700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7effac111390 con 0x7effac101d40 2024-06-06T04:30:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.155+0000 7effb4766700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7effac111690 con 0x7effac101380 2024-06-06T04:30:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.155+0000 7effb2502700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effac101380 0x7effac19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.155+0000 7effb2502700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effac101380 0x7effac19d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:47598/0 (socket says 172.21.15.45:47598) 2024-06-06T04:30:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.155+0000 7effb2502700 1 -- 172.21.15.45:0/237328121 learned_addr learned my addr 172.21.15.45:0/237328121 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.155+0000 7effb2d03700 1 --2- 172.21.15.45:0/237328121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effac1073f0 0x7effac197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.155+0000 7effb1d01700 1 --2- 172.21.15.45:0/237328121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effac101d40 0x7effac19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.156+0000 7effb2502700 1 -- 172.21.15.45:0/237328121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effac101d40 msgr2=0x7effac19dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.156+0000 7effb2502700 1 --2- 172.21.15.45:0/237328121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effac101d40 0x7effac19dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.156+0000 7effb2502700 1 -- 172.21.15.45:0/237328121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effac1073f0 msgr2=0x7effac197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.156+0000 7effb2502700 1 --2- 172.21.15.45:0/237328121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effac1073f0 0x7effac197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.156+0000 7effb2502700 1 -- 172.21.15.45:0/237328121 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effac198010 con 0x7effac101380 2024-06-06T04:30:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.156+0000 7effb2d03700 1 --2- 172.21.15.45:0/237328121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effac1073f0 0x7effac197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:30:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.156+0000 7effb2502700 1 --2- 172.21.15.45:0/237328121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effac101380 0x7effac19d660 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7effa800ba90 tx=0x7effa800bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.156+0000 7effa37fe700 1 -- 172.21.15.45:0/237328121 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8027070 con 0x7effac101380 2024-06-06T04:30:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.156+0000 7effb4766700 1 -- 172.21.15.45:0/237328121 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effac1982a0 con 0x7effac101380 2024-06-06T04:30:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.156+0000 7effa37fe700 1 -- 172.21.15.45:0/237328121 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effa8019cb0 con 0x7effac101380 2024-06-06T04:30:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.156+0000 7effa37fe700 1 -- 172.21.15.45:0/237328121 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8003dd0 con 0x7effac101380 2024-06-06T04:30:32.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.156+0000 7effb4766700 1 -- 172.21.15.45:0/237328121 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effac06aa00 con 0x7effac101380 2024-06-06T04:30:32.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.159+0000 7effa37fe700 1 -- 172.21.15.45:0/237328121 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7effa802a070 con 0x7effac101380 2024-06-06T04:30:32.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.159+0000 7effa37fe700 1 --2- 172.21.15.45:0/237328121 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7eff980a2c20 0x7eff980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.159+0000 7effb1d01700 1 --2- 172.21.15.45:0/237328121 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7eff980a2c20 0x7eff980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.159+0000 7effa37fe700 1 -- 172.21.15.45:0/237328121 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7effa80cfe20 con 0x7effac101380 2024-06-06T04:30:32.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.160+0000 7effb1d01700 1 --2- 172.21.15.45:0/237328121 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7eff980a2c20 0x7eff980a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7effac078900 tx=0x7eff9c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:32.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.160+0000 7effbefe9b80 1 Processor -- start 2024-06-06T04:30:32.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.160+0000 7effbefe9b80 1 -- start start 2024-06-06T04:30:32.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.160+0000 7effbefe9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d49765f0 0x5577d49769d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.160+0000 7effbefe9b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d4976f10 0x5577d4970990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.160+0000 7effbefe9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4970ed0 0x5577d49712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.160+0000 7effbefe9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5577d48c27c0 con 0x5577d4970ed0 2024-06-06T04:30:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.160+0000 7effbefe9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5577d486a6d0 con 0x5577d49765f0 2024-06-06T04:30:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.160+0000 7effbefe9b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5577d48be780 con 0x5577d4976f10 2024-06-06T04:30:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.161+0000 7effb2d03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4970ed0 0x5577d49712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.161+0000 7effb2d03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4970ed0 0x5577d49712f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37768/0 (socket says 172.21.15.45:37768) 2024-06-06T04:30:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.161+0000 7effb2d03700 1 -- 172.21.15.45:0/1869802238 learned_addr learned my addr 172.21.15.45:0/1869802238 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.161+0000 7effb1d01700 1 --2- 172.21.15.45:0/1869802238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d4976f10 0x5577d4970990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.161+0000 7effb2502700 1 --2- 172.21.15.45:0/1869802238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d49765f0 0x5577d49769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.161+0000 7effb2d03700 1 -- 172.21.15.45:0/1869802238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d4976f10 msgr2=0x5577d4970990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.161+0000 7effb2d03700 1 --2- 172.21.15.45:0/1869802238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d4976f10 0x5577d4970990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.161+0000 7effb2d03700 1 -- 172.21.15.45:0/1869802238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d49765f0 msgr2=0x5577d49769d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.161+0000 7effb2d03700 1 --2- 172.21.15.45:0/1869802238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d49765f0 0x5577d49769d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.161+0000 7effb2d03700 1 -- 172.21.15.45:0/1869802238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577d4860f00 con 0x5577d4970ed0 2024-06-06T04:30:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.161+0000 7effb2502700 1 --2- 172.21.15.45:0/1869802238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d49765f0 0x5577d49769d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:30:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.161+0000 7effb2d03700 1 --2- 172.21.15.45:0/1869802238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4970ed0 0x5577d49712f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7effa4007850 tx=0x7effa4007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a682c6850d86793 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.161+0000 7effa0ff9700 1 -- 172.21.15.45:0/1869802238 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa400e920 con 0x5577d4970ed0 2024-06-06T04:30:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.161+0000 7effa0ff9700 1 -- 172.21.15.45:0/1869802238 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effa401e070 con 0x5577d4970ed0 2024-06-06T04:30:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.161+0000 7effa0ff9700 1 -- 172.21.15.45:0/1869802238 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4019df0 con 0x5577d4970ed0 2024-06-06T04:30:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.162+0000 7effbefe9b80 1 -- 172.21.15.45:0/1869802238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4970ed0 msgr2=0x5577d49712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.162+0000 7effbefe9b80 1 --2- 172.21.15.45:0/1869802238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4970ed0 0x5577d49712f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7effa4007850 tx=0x7effa4007b60 comp rx=0 tx=0).stop 2024-06-06T04:30:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.162+0000 7effbefe9b80 1 -- 172.21.15.45:0/1869802238 shutdown_connections 2024-06-06T04:30:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.162+0000 7effbefe9b80 1 --2- 172.21.15.45:0/1869802238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4970ed0 0x5577d49712f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.162+0000 7effbefe9b80 1 --2- 172.21.15.45:0/1869802238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d49765f0 0x5577d49769d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.162+0000 7effbefe9b80 1 --2- 172.21.15.45:0/1869802238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d4976f10 0x5577d4970990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.162+0000 7effbefe9b80 1 -- 172.21.15.45:0/1869802238 >> 172.21.15.45:0/1869802238 conn(0x5577d489bc70 msgr2=0x5577d496b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.162+0000 7effbefe9b80 1 -- 172.21.15.45:0/1869802238 shutdown_connections 2024-06-06T04:30:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.162+0000 7effbefe9b80 1 -- 172.21.15.45:0/1869802238 wait complete. 2024-06-06T04:30:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.162+0000 7effbefe9b80 1 -- start start 2024-06-06T04:30:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effbefe9b80 1 Processor -- start 2024-06-06T04:30:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effbefe9b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d4970ed0 0x5577d497ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effbefe9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d49765f0 0x5577d4979fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effbefe9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4976f10 0x5577d497a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effbefe9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5577d497d980 con 0x5577d4976f10 2024-06-06T04:30:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effbefe9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5577d4862ba0 con 0x5577d49765f0 2024-06-06T04:30:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effbefe9b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5577d497db00 con 0x5577d4970ed0 2024-06-06T04:30:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effbefe9b80 10 client.0 _ll_register_callbacks cb 0x5577d485c8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:30:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effb2d03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4976f10 0x5577d497a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effb2d03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4976f10 0x5577d497a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37778/0 (socket says 172.21.15.45:37778) 2024-06-06T04:30:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effb2d03700 1 -- 172.21.15.45:0/1946714749 learned_addr learned my addr 172.21.15.45:0/1946714749 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effb2502700 1 --2- 172.21.15.45:0/1946714749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d4970ed0 0x5577d497ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effb1d01700 1 --2- 172.21.15.45:0/1946714749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d49765f0 0x5577d4979fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effb2d03700 1 -- 172.21.15.45:0/1946714749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d4970ed0 msgr2=0x5577d497ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effb2d03700 1 --2- 172.21.15.45:0/1946714749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d4970ed0 0x5577d497ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effb2d03700 1 -- 172.21.15.45:0/1946714749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d49765f0 msgr2=0x5577d4979fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effb2d03700 1 --2- 172.21.15.45:0/1946714749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d49765f0 0x5577d4979fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effb2d03700 1 -- 172.21.15.45:0/1946714749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577d4811880 con 0x5577d4976f10 2024-06-06T04:30:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.163+0000 7effb1d01700 1 --2- 172.21.15.45:0/1946714749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d49765f0 0x5577d4979fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:30:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.164+0000 7effb2d03700 1 --2- 172.21.15.45:0/1946714749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4976f10 0x5577d497a4f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7effa4000f80 tx=0x7effa4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420972256c253807 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.164+0000 7eff95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:30:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.164+0000 7eff95ffb700 1 -- 172.21.15.45:0/1946714749 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4019da0 con 0x5577d4976f10 2024-06-06T04:30:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.164+0000 7eff95ffb700 1 -- 172.21.15.45:0/1946714749 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7effa400e720 con 0x5577d4976f10 2024-06-06T04:30:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.164+0000 7effbefe9b80 1 -- 172.21.15.45:0/1946714749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa401e070 con 0x5577d4976f10 2024-06-06T04:30:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.164+0000 7eff95ffb700 1 -- 172.21.15.45:0/1946714749 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7effa4034050 con 0x5577d4976f10 2024-06-06T04:30:32.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.164+0000 7effbefe9b80 10 client.15066 fetch_fsmap learned FSMap version 26 2024-06-06T04:30:32.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.164+0000 7effbefe9b80 1 -- 172.21.15.45:0/1946714749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5577d4988220 con 0x5577d4976f10 2024-06-06T04:30:32.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.164+0000 7eff95ffb700 1 -- 172.21.15.45:0/1946714749 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effa401e070 con 0x5577d4976f10 2024-06-06T04:30:32.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.164+0000 7eff95ffb700 1 -- 172.21.15.45:0/1946714749 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7effa40055a0 con 0x5577d4976f10 2024-06-06T04:30:32.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.164+0000 7effbefe9b80 10 client.15066 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T04:30:32.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.164+0000 7effbefe9b80 10 client.15066 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T04:30:32.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.164+0000 7effbefe9b80 20 client.15066 populate_metadata read hostname 'smithi045' 2024-06-06T04:30:32.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.164+0000 7effbefe9b80 1 --2- 172.21.15.45:0/1946714749 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x5577d498a520 0x5577d498b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.165+0000 7effbefe9b80 4 client.15066 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:30:32.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.165+0000 7effbefe9b80 1 -- 172.21.15.45:0/1946714749 --> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577d48cfcc0 con 0x5577d498a520 2024-06-06T04:30:32.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.165+0000 7effb2502700 1 --2- 172.21.15.45:0/1946714749 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x5577d498a520 0x5577d498b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.165+0000 7effb2502700 1 --2- 172.21.15.45:0/1946714749 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x5577d498a520 0x5577d498b990 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa9abef2a0755e70 server_cookie=1870ac40b8b76883 in_seq=0 out_seq=0 2024-06-06T04:30:32.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.165+0000 7eff95ffb700 10 client.15066 ms_handle_connect on v2:172.21.15.154:6838/982197202 2024-06-06T04:30:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.166+0000 7eff95ffb700 1 -- 172.21.15.45:0/1946714749 <== mds.1 v2:172.21.15.154:6838/982197202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5577d48cfcc0 con 0x5577d498a520 2024-06-06T04:30:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.166+0000 7eff95ffb700 10 client.15066 handle_command_reply: tid=0 2024-06-06T04:30:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.166+0000 7effbefe9b80 1 client.15066 shutdown 2024-06-06T04:30:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.166+0000 7effbefe9b80 1 -- 172.21.15.45:0/1946714749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4976f10 msgr2=0x5577d497a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.166+0000 7effbefe9b80 1 --2- 172.21.15.45:0/1946714749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4976f10 0x5577d497a4f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7effa4000f80 tx=0x7effa4004d00 comp rx=0 tx=0).stop 2024-06-06T04:30:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.166+0000 7effbefe9b80 1 -- 172.21.15.45:0/1946714749 shutdown_connections 2024-06-06T04:30:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.166+0000 7effbefe9b80 1 --2- 172.21.15.45:0/1946714749 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x5577d498a520 0x5577d498b990 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.166+0000 7effbefe9b80 1 --2- 172.21.15.45:0/1946714749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4976f10 0x5577d497a4f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.166+0000 7effbefe9b80 1 --2- 172.21.15.45:0/1946714749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d4970ed0 0x5577d497ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.166+0000 7eff95ffb700 0 client.15066 ms_handle_reset on v2:172.21.15.154:6838/982197202 2024-06-06T04:30:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.166+0000 7effbefe9b80 1 --2- 172.21.15.45:0/1946714749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d49765f0 0x5577d4979fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.166+0000 7effbefe9b80 1 -- 172.21.15.45:0/1946714749 >> 172.21.15.45:0/1946714749 conn(0x5577d489bc70 msgr2=0x5577d496f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.166+0000 7effbefe9b80 1 -- 172.21.15.45:0/1946714749 shutdown_connections 2024-06-06T04:30:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.166+0000 7effbefe9b80 1 -- 172.21.15.45:0/1946714749 wait complete. 2024-06-06T04:30:32.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.166+0000 7effbefe9b80 20 client.15066 trim_cache size 0 max 16384 2024-06-06T04:30:32.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effbefe9b80 1 Processor -- start 2024-06-06T04:30:32.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effbefe9b80 1 -- start start 2024-06-06T04:30:32.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effbefe9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4970ed0 0x5577d4970990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effbefe9b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d49765f0 0x5577d4974780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effbefe9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d4976f10 0x5577d496ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effbefe9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5577d4989cd0 con 0x5577d4970ed0 2024-06-06T04:30:32.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effbefe9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5577d498a3a0 con 0x5577d4976f10 2024-06-06T04:30:32.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effbefe9b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5577d498aa70 con 0x5577d49765f0 2024-06-06T04:30:32.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effb2502700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4970ed0 0x5577d4970990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effb2502700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4970ed0 0x5577d4970990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37786/0 (socket says 172.21.15.45:37786) 2024-06-06T04:30:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effb2502700 1 -- 172.21.15.45:0/3353282260 learned_addr learned my addr 172.21.15.45:0/3353282260 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effb2d03700 1 --2- 172.21.15.45:0/3353282260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d4976f10 0x5577d496ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effb1d01700 1 --2- 172.21.15.45:0/3353282260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d49765f0 0x5577d4974780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effb2502700 1 -- 172.21.15.45:0/3353282260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d49765f0 msgr2=0x5577d4974780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effb2502700 1 --2- 172.21.15.45:0/3353282260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d49765f0 0x5577d4974780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effb2502700 1 -- 172.21.15.45:0/3353282260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d4976f10 msgr2=0x5577d496ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effb2502700 1 --2- 172.21.15.45:0/3353282260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d4976f10 0x5577d496ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effb2502700 1 -- 172.21.15.45:0/3353282260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577d496c4b0 con 0x5577d4970ed0 2024-06-06T04:30:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effb2d03700 1 --2- 172.21.15.45:0/3353282260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d4976f10 0x5577d496ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:30:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effb1d01700 1 --2- 172.21.15.45:0/3353282260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d49765f0 0x5577d4974780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:30:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.189+0000 7effb2502700 1 --2- 172.21.15.45:0/3353282260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4970ed0 0x5577d4970990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7effa8005cb0 tx=0x7effa80bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fab1bc496adf336 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.190+0000 7eff97fff700 1 -- 172.21.15.45:0/3353282260 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8096e60 con 0x5577d4970ed0 2024-06-06T04:30:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.190+0000 7eff97fff700 1 -- 172.21.15.45:0/3353282260 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effa809c9f0 con 0x5577d4970ed0 2024-06-06T04:30:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.190+0000 7eff97fff700 1 -- 172.21.15.45:0/3353282260 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa809a5f0 con 0x5577d4970ed0 2024-06-06T04:30:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.190+0000 7effbefe9b80 1 -- 172.21.15.45:0/3353282260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4970ed0 msgr2=0x5577d4970990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.190+0000 7effbefe9b80 1 --2- 172.21.15.45:0/3353282260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4970ed0 0x5577d4970990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7effa8005cb0 tx=0x7effa80bc040 comp rx=0 tx=0).stop 2024-06-06T04:30:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.190+0000 7effbefe9b80 1 -- 172.21.15.45:0/3353282260 shutdown_connections 2024-06-06T04:30:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.190+0000 7effbefe9b80 1 --2- 172.21.15.45:0/3353282260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d4976f10 0x5577d496ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.190+0000 7effbefe9b80 1 --2- 172.21.15.45:0/3353282260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d4970ed0 0x5577d4970990 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.190+0000 7effbefe9b80 1 --2- 172.21.15.45:0/3353282260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d49765f0 0x5577d4974780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.190+0000 7effbefe9b80 1 -- 172.21.15.45:0/3353282260 >> 172.21.15.45:0/3353282260 conn(0x5577d489bc70 msgr2=0x5577d4974070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.190+0000 7effbefe9b80 1 -- 172.21.15.45:0/3353282260 shutdown_connections 2024-06-06T04:30:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.190+0000 7effbefe9b80 1 -- 172.21.15.45:0/3353282260 wait complete. 2024-06-06T04:30:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.191+0000 7effbefe9b80 1 -- start start 2024-06-06T04:30:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.191+0000 7effbefe9b80 1 Processor -- start 2024-06-06T04:30:32.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.191+0000 7effbefe9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d4970ed0 0x5577d496fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.191+0000 7effbefe9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d49765f0 0x5577d49700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.191+0000 7effbefe9b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d4976f10 0x5577d4970600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.191+0000 7effbefe9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5577d498b810 con 0x5577d49765f0 2024-06-06T04:30:32.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.191+0000 7effbefe9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5577d498b140 con 0x5577d4970ed0 2024-06-06T04:30:32.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.191+0000 7effbefe9b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5577d498bee0 con 0x5577d4976f10 2024-06-06T04:30:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.191+0000 7effbefe9b80 10 client.0 _ll_register_callbacks cb 0x5577d485c8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:30:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.191+0000 7effb1d01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d49765f0 0x5577d49700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.191+0000 7effb1d01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d49765f0 0x5577d49700c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37796/0 (socket says 172.21.15.45:37796) 2024-06-06T04:30:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.191+0000 7effb1d01700 1 -- 172.21.15.45:0/2981789692 learned_addr learned my addr 172.21.15.45:0/2981789692 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.191+0000 7effb2d03700 1 --2- 172.21.15.45:0/2981789692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d4976f10 0x5577d4970600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.191+0000 7effb2502700 1 --2- 172.21.15.45:0/2981789692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d4970ed0 0x5577d496fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.192+0000 7effb1d01700 1 -- 172.21.15.45:0/2981789692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d4976f10 msgr2=0x5577d4970600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.192+0000 7effb1d01700 1 --2- 172.21.15.45:0/2981789692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d4976f10 0x5577d4970600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.192+0000 7effb1d01700 1 -- 172.21.15.45:0/2981789692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d4970ed0 msgr2=0x5577d496fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.192+0000 7effb1d01700 1 --2- 172.21.15.45:0/2981789692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d4970ed0 0x5577d496fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.192+0000 7effb1d01700 1 -- 172.21.15.45:0/2981789692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577d4993230 con 0x5577d49765f0 2024-06-06T04:30:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.192+0000 7effb1d01700 1 --2- 172.21.15.45:0/2981789692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d49765f0 0x5577d49700c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00aa30 tx=0x7eff9c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c270ca586c28dd5f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.192+0000 7eff95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:30:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.192+0000 7eff95ffb700 1 -- 172.21.15.45:0/2981789692 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c016070 con 0x5577d49765f0 2024-06-06T04:30:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.192+0000 7eff95ffb700 1 -- 172.21.15.45:0/2981789692 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7eff9c010840 con 0x5577d49765f0 2024-06-06T04:30:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.192+0000 7effbefe9b80 1 -- 172.21.15.45:0/2981789692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa809c9f0 con 0x5577d49765f0 2024-06-06T04:30:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.192+0000 7eff95ffb700 1 -- 172.21.15.45:0/2981789692 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7eff9c031050 con 0x5577d49765f0 2024-06-06T04:30:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.192+0000 7effbefe9b80 10 client.15072 fetch_fsmap learned FSMap version 26 2024-06-06T04:30:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.192+0000 7effbefe9b80 1 -- 172.21.15.45:0/2981789692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5577d49f5f60 con 0x5577d49765f0 2024-06-06T04:30:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.193+0000 7eff95ffb700 1 -- 172.21.15.45:0/2981789692 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effa809c9f0 con 0x5577d49765f0 2024-06-06T04:30:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.193+0000 7eff95ffb700 1 -- 172.21.15.45:0/2981789692 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7eff9c026020 con 0x5577d49765f0 2024-06-06T04:30:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.193+0000 7effbefe9b80 10 client.15072 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T04:30:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.193+0000 7effbefe9b80 10 client.15072 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T04:30:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.193+0000 7effbefe9b80 20 client.15072 populate_metadata read hostname 'smithi045' 2024-06-06T04:30:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.193+0000 7effbefe9b80 1 --2- 172.21.15.45:0/2981789692 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x5577d49fab70 0x5577d49f9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.193+0000 7effbefe9b80 4 client.15072 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T04:30:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.193+0000 7effbefe9b80 1 -- 172.21.15.45:0/2981789692 --> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] -- command(tid 0: {"prefix": "status"}) -- 0x5577d48ca240 con 0x5577d49fab70 2024-06-06T04:30:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.193+0000 7effb2502700 1 --2- 172.21.15.45:0/2981789692 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x5577d49fab70 0x5577d49f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7effb2502700 1 --2- 172.21.15.45:0/2981789692 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x5577d49fab70 0x5577d49f9880 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=473b69ff7e6bb205 server_cookie=b08818c8ac0885b9 in_seq=0 out_seq=0 2024-06-06T04:30:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7eff95ffb700 10 client.15072 ms_handle_connect on v2:172.21.15.154:6838/982197202 2024-06-06T04:30:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7eff95ffb700 1 -- 172.21.15.45:0/2981789692 <== mds.1 v2:172.21.15.154:6838/982197202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5577d48ca240 con 0x5577d49fab70 2024-06-06T04:30:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7eff95ffb700 10 client.15072 handle_command_reply: tid=0 2024-06-06T04:30:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7effbefe9b80 1 client.15072 shutdown 2024-06-06T04:30:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7effbefe9b80 1 -- 172.21.15.45:0/2981789692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d49765f0 msgr2=0x5577d49700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7effbefe9b80 1 --2- 172.21.15.45:0/2981789692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d49765f0 0x5577d49700c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00aa30 tx=0x7eff9c0047c0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7effbefe9b80 1 -- 172.21.15.45:0/2981789692 shutdown_connections 2024-06-06T04:30:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7effbefe9b80 1 --2- 172.21.15.45:0/2981789692 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x5577d49fab70 0x5577d49f9880 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7effbefe9b80 1 --2- 172.21.15.45:0/2981789692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5577d4976f10 0x5577d4970600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7effbefe9b80 1 --2- 172.21.15.45:0/2981789692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577d4970ed0 0x5577d496fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7effbefe9b80 1 --2- 172.21.15.45:0/2981789692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d49765f0 0x5577d49700c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7effbefe9b80 1 -- 172.21.15.45:0/2981789692 >> 172.21.15.45:0/2981789692 conn(0x5577d489bc70 msgr2=0x5577d4964db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7eff95ffb700 0 client.15072 ms_handle_reset on v2:172.21.15.154:6838/982197202 2024-06-06T04:30:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7effbefe9b80 1 -- 172.21.15.45:0/2981789692 shutdown_connections 2024-06-06T04:30:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7effbefe9b80 1 -- 172.21.15.45:0/2981789692 wait complete. 2024-06-06T04:30:32.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.194+0000 7effbefe9b80 20 client.15072 trim_cache size 0 max 16384 2024-06-06T04:30:32.200 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:30:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "c4f6b938-23bb-11ef-bc9b-c7b262605968", 2024-06-06T04:30:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 1, 2024-06-06T04:30:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24463, 2024-06-06T04:30:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-06-06T04:30:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-06-06T04:30:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-06-06T04:30:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 125.491153497, 2024-06-06T04:30:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2024-06-06T04:30:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 100, 2024-06-06T04:30:32.201 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 98, 2024-06-06T04:30:32.201 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 213.29861798499999 2024-06-06T04:30:32.201 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:30:32.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.195+0000 7effb4766700 1 -- 172.21.15.45:0/237328121 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7eff980a2c20 msgr2=0x7eff980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:32.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.195+0000 7effb4766700 1 --2- 172.21.15.45:0/237328121 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7eff980a2c20 0x7eff980a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7effac078900 tx=0x7eff9c00a380 comp rx=0 tx=0).stop 2024-06-06T04:30:32.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.195+0000 7effb4766700 1 -- 172.21.15.45:0/237328121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effac101380 msgr2=0x7effac19d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:32.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.195+0000 7effb4766700 1 --2- 172.21.15.45:0/237328121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effac101380 0x7effac19d660 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7effa800ba90 tx=0x7effa800bac0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.195+0000 7effb4766700 1 -- 172.21.15.45:0/237328121 shutdown_connections 2024-06-06T04:30:32.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.195+0000 7effb4766700 1 --2- 172.21.15.45:0/237328121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effac1073f0 0x7effac197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.195+0000 7effb4766700 1 --2- 172.21.15.45:0/237328121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effac101380 0x7effac19d660 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.195+0000 7effb4766700 1 --2- 172.21.15.45:0/237328121 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7eff980a2c20 0x7eff980a50e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.195+0000 7effb4766700 1 --2- 172.21.15.45:0/237328121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effac101d40 0x7effac19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.195+0000 7effb4766700 1 -- 172.21.15.45:0/237328121 >> 172.21.15.45:0/237328121 conn(0x7effac078300 msgr2=0x7effac1000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.195+0000 7effb4766700 1 -- 172.21.15.45:0/237328121 shutdown_connections 2024-06-06T04:30:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:32.195+0000 7effb4766700 1 -- 172.21.15.45:0/237328121 wait complete. 2024-06-06T04:30:32.874 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-06T04:30:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:33.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:33 smithi086 ceph-mon[36053]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:30:33.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:33 smithi154 ceph-mon[36428]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:30:33.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:33 smithi045 ceph-mon[32299]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:30:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.848+0000 7f31101f8700 1 -- 172.21.15.45:0/1891039920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f310806ac40 msgr2=0x7f310806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.848+0000 7f31101f8700 1 --2- 172.21.15.45:0/1891039920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f310806ac40 0x7f310806b020 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3104009990 tx=0x7f3104009ca0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.848+0000 7f31101f8700 1 -- 172.21.15.45:0/1891039920 shutdown_connections 2024-06-06T04:30:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.848+0000 7f31101f8700 1 --2- 172.21.15.45:0/1891039920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f310810f240 0x7f3108111630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.848+0000 7f31101f8700 1 --2- 172.21.15.45:0/1891039920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f310806ac40 0x7f310806b020 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.848+0000 7f31101f8700 1 --2- 172.21.15.45:0/1891039920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310806b560 0x7f310810ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.848+0000 7f31101f8700 1 -- 172.21.15.45:0/1891039920 >> 172.21.15.45:0/1891039920 conn(0x7f3108078190 msgr2=0x7f31080785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.848+0000 7f31101f8700 1 -- 172.21.15.45:0/1891039920 shutdown_connections 2024-06-06T04:30:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.848+0000 7f31101f8700 1 -- 172.21.15.45:0/1891039920 wait complete. 2024-06-06T04:30:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.849+0000 7f31101f8700 1 Processor -- start 2024-06-06T04:30:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.849+0000 7f31101f8700 1 -- start start 2024-06-06T04:30:34.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.849+0000 7f31101f8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f310806ac40 0x7f310819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.849+0000 7f31101f8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310806b560 0x7f310819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.849+0000 7f31101f8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f310810f240 0x7f310819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.849+0000 7f31101f8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3108114960 con 0x7f310806ac40 2024-06-06T04:30:34.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.849+0000 7f31101f8700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f31081147e0 con 0x7f310806b560 2024-06-06T04:30:34.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.849+0000 7f31101f8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3108114ae0 con 0x7f310810f240 2024-06-06T04:30:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.849+0000 7f310df94700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f310806ac40 0x7f310819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.849+0000 7f310df94700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f310806ac40 0x7f310819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37804/0 (socket says 172.21.15.45:37804) 2024-06-06T04:30:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.849+0000 7f310df94700 1 -- 172.21.15.45:0/4185434938 learned_addr learned my addr 172.21.15.45:0/4185434938 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.849+0000 7f310e795700 1 --2- 172.21.15.45:0/4185434938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f310810f240 0x7f310819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.850+0000 7f310d793700 1 --2- 172.21.15.45:0/4185434938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310806b560 0x7f310819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.849+0000 7f310df94700 1 -- 172.21.15.45:0/4185434938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f310810f240 msgr2=0x7f310819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.850+0000 7f310df94700 1 --2- 172.21.15.45:0/4185434938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f310810f240 0x7f310819f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.850+0000 7f310df94700 1 -- 172.21.15.45:0/4185434938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310806b560 msgr2=0x7f310819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.850+0000 7f310df94700 1 --2- 172.21.15.45:0/4185434938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310806b560 0x7f310819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.850+0000 7f310df94700 1 -- 172.21.15.45:0/4185434938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310819fdd0 con 0x7f310806ac40 2024-06-06T04:30:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.850+0000 7f310d793700 1 --2- 172.21.15.45:0/4185434938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310806b560 0x7f310819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:30:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.850+0000 7f310df94700 1 --2- 172.21.15.45:0/4185434938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f310806ac40 0x7f310819ae10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3104000c00 tx=0x7f3104003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:34.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.850+0000 7f30feffd700 1 -- 172.21.15.45:0/4185434938 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104027070 con 0x7f310806ac40 2024-06-06T04:30:34.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.850+0000 7f31101f8700 1 -- 172.21.15.45:0/4185434938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31081a0060 con 0x7f310806ac40 2024-06-06T04:30:34.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.850+0000 7f30feffd700 1 -- 172.21.15.45:0/4185434938 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f310400bdc0 con 0x7f310806ac40 2024-06-06T04:30:34.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.850+0000 7f30feffd700 1 -- 172.21.15.45:0/4185434938 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104022e40 con 0x7f310806ac40 2024-06-06T04:30:34.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.850+0000 7f31101f8700 1 -- 172.21.15.45:0/4185434938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31081a7940 con 0x7f310806ac40 2024-06-06T04:30:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.852+0000 7f30feffd700 1 -- 172.21.15.45:0/4185434938 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f310402a070 con 0x7f310806ac40 2024-06-06T04:30:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.853+0000 7f30feffd700 1 --2- 172.21.15.45:0/4185434938 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f30f40a2ca0 0x7f30f40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.853+0000 7f310d793700 1 --2- 172.21.15.45:0/4185434938 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f30f40a2ca0 0x7f30f40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.853+0000 7f30feffd700 1 -- 172.21.15.45:0/4185434938 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f31040d01b0 con 0x7f310806ac40 2024-06-06T04:30:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.853+0000 7f310d793700 1 --2- 172.21.15.45:0/4185434938 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f30f40a2ca0 0x7f30f40a5160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f30f8005fd0 tx=0x7f30f8005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.854+0000 7f311aa7bb80 1 Processor -- start 2024-06-06T04:30:34.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.854+0000 7f311aa7bb80 1 -- start start 2024-06-06T04:30:34.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.854+0000 7f311aa7bb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d1061f10 0x55f8d10622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.854+0000 7f311aa7bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1062830 0x55f8d105c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.854+0000 7f311aa7bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d105c750 0x55f8d105cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.854+0000 7f311aa7bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f8d0f4ef10 con 0x55f8d1062830 2024-06-06T04:30:34.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.854+0000 7f311aa7bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8d0fad800 con 0x55f8d105c750 2024-06-06T04:30:34.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.854+0000 7f311aa7bb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f8d0f15410 con 0x55f8d1061f10 2024-06-06T04:30:34.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.854+0000 7f310d793700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1062830 0x55f8d105c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.854+0000 7f310d793700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1062830 0x55f8d105c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37808/0 (socket says 172.21.15.45:37808) 2024-06-06T04:30:34.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.854+0000 7f310d793700 1 -- 172.21.15.45:0/2227354273 learned_addr learned my addr 172.21.15.45:0/2227354273 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:34.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.855+0000 7f310e795700 1 --2- 172.21.15.45:0/2227354273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d105c750 0x55f8d105cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.855+0000 7f310df94700 1 --2- 172.21.15.45:0/2227354273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d1061f10 0x55f8d10622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.855+0000 7f310d793700 1 -- 172.21.15.45:0/2227354273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d1061f10 msgr2=0x55f8d10622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:34.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.855+0000 7f310d793700 1 --2- 172.21.15.45:0/2227354273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d1061f10 0x55f8d10622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.855+0000 7f310d793700 1 -- 172.21.15.45:0/2227354273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d105c750 msgr2=0x55f8d105cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:34.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.855+0000 7f310d793700 1 --2- 172.21.15.45:0/2227354273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d105c750 0x55f8d105cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.855+0000 7f310d793700 1 -- 172.21.15.45:0/2227354273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8d0efc880 con 0x55f8d1062830 2024-06-06T04:30:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.855+0000 7f310d793700 1 --2- 172.21.15.45:0/2227354273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1062830 0x55f8d105c210 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f30f800eda0 tx=0x7f30f8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a368d353e08f792 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.855+0000 7f30f3fff700 1 -- 172.21.15.45:0/2227354273 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f800edd0 con 0x55f8d1062830 2024-06-06T04:30:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.855+0000 7f30f3fff700 1 -- 172.21.15.45:0/2227354273 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30f8017030 con 0x55f8d1062830 2024-06-06T04:30:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.855+0000 7f30f3fff700 1 -- 172.21.15.45:0/2227354273 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8005410 con 0x55f8d1062830 2024-06-06T04:30:34.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.856+0000 7f311aa7bb80 1 -- 172.21.15.45:0/2227354273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1062830 msgr2=0x55f8d105c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:34.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.856+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/2227354273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1062830 0x55f8d105c210 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f30f800eda0 tx=0x7f30f8009f90 comp rx=0 tx=0).stop 2024-06-06T04:30:34.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.856+0000 7f311aa7bb80 1 -- 172.21.15.45:0/2227354273 shutdown_connections 2024-06-06T04:30:34.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.856+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/2227354273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d105c750 0x55f8d105cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.856+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/2227354273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d1061f10 0x55f8d10622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.856+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/2227354273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1062830 0x55f8d105c210 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.856+0000 7f311aa7bb80 1 -- 172.21.15.45:0/2227354273 >> 172.21.15.45:0/2227354273 conn(0x55f8d0f89030 msgr2=0x55f8d10571d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:34.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.856+0000 7f311aa7bb80 1 -- 172.21.15.45:0/2227354273 shutdown_connections 2024-06-06T04:30:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.856+0000 7f311aa7bb80 1 -- 172.21.15.45:0/2227354273 wait complete. 2024-06-06T04:30:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.856+0000 7f311aa7bb80 1 -- start start 2024-06-06T04:30:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.856+0000 7f311aa7bb80 1 Processor -- start 2024-06-06T04:30:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.857+0000 7f311aa7bb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d105c750 0x55f8d10685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.857+0000 7f311aa7bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d1061f10 0x55f8d10658e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.857+0000 7f311aa7bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1062830 0x55f8d1065e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.857+0000 7f311aa7bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f8d0f4df70 con 0x55f8d1062830 2024-06-06T04:30:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.857+0000 7f311aa7bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8d0fa97c0 con 0x55f8d1061f10 2024-06-06T04:30:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.857+0000 7f311aa7bb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f8d1069210 con 0x55f8d105c750 2024-06-06T04:30:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.857+0000 7f311aa7bb80 10 client.0 _ll_register_callbacks cb 0x55f8d0f43560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:30:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.857+0000 7f310e795700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1062830 0x55f8d1065e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.857+0000 7f310e795700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1062830 0x55f8d1065e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37810/0 (socket says 172.21.15.45:37810) 2024-06-06T04:30:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.857+0000 7f310e795700 1 -- 172.21.15.45:0/1725313589 learned_addr learned my addr 172.21.15.45:0/1725313589 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.857+0000 7f310d793700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d1061f10 0x55f8d10658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.857+0000 7f310df94700 1 --2- 172.21.15.45:0/1725313589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d105c750 0x55f8d10685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.857+0000 7f310e795700 1 -- 172.21.15.45:0/1725313589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d105c750 msgr2=0x55f8d10685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.858+0000 7f310e795700 1 --2- 172.21.15.45:0/1725313589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d105c750 0x55f8d10685e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.858+0000 7f310e795700 1 -- 172.21.15.45:0/1725313589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d1061f10 msgr2=0x55f8d10658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.858+0000 7f310e795700 1 --2- 172.21.15.45:0/1725313589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d1061f10 0x55f8d10658e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.858+0000 7f310e795700 1 -- 172.21.15.45:0/1725313589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8d0f73700 con 0x55f8d1062830 2024-06-06T04:30:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.858+0000 7f310d793700 1 --2- 172.21.15.45:0/1725313589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d1061f10 0x55f8d10658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:30:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.858+0000 7f310e795700 1 --2- 172.21.15.45:0/1725313589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1062830 0x55f8d1065e20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3100007940 tx=0x7f3100007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7335230f7e4ecf66 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.858+0000 7f30f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:30:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.858+0000 7f30f17fa700 1 -- 172.21.15.45:0/1725313589 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100010670 con 0x55f8d1062830 2024-06-06T04:30:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.858+0000 7f311aa7bb80 1 -- 172.21.15.45:0/1725313589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f8017030 con 0x55f8d1062830 2024-06-06T04:30:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.858+0000 7f30f17fa700 1 -- 172.21.15.45:0/1725313589 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f3100010810 con 0x55f8d1062830 2024-06-06T04:30:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.858+0000 7f30f17fa700 1 -- 172.21.15.45:0/1725313589 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3100024050 con 0x55f8d1062830 2024-06-06T04:30:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.858+0000 7f311aa7bb80 10 client.15084 fetch_fsmap learned FSMap version 26 2024-06-06T04:30:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.858+0000 7f311aa7bb80 1 -- 172.21.15.45:0/1725313589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f8d1073a30 con 0x55f8d1062830 2024-06-06T04:30:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.858+0000 7f30f17fa700 1 -- 172.21.15.45:0/1725313589 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30f8017030 con 0x55f8d1062830 2024-06-06T04:30:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.858+0000 7f30f17fa700 1 -- 172.21.15.45:0/1725313589 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3100016bb0 con 0x55f8d1062830 2024-06-06T04:30:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.859+0000 7f311aa7bb80 10 client.15084 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T04:30:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.859+0000 7f311aa7bb80 10 client.15084 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T04:30:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.859+0000 7f311aa7bb80 20 client.15084 populate_metadata read hostname 'smithi045' 2024-06-06T04:30:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.859+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/1725313589 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55f8d1076120 0x55f8d1077590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.859+0000 7f311aa7bb80 4 client.15084 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:30:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.859+0000 7f311aa7bb80 1 -- 172.21.15.45:0/1725313589 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8d0fbad00 con 0x55f8d1076120 2024-06-06T04:30:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.859+0000 7f310df94700 1 --2- 172.21.15.45:0/1725313589 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55f8d1076120 0x55f8d1077590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.859+0000 7f310df94700 1 --2- 172.21.15.45:0/1725313589 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55f8d1076120 0x55f8d1077590 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=758b36b1373c64a0 server_cookie=88c0ca82af5da90a in_seq=0 out_seq=0 2024-06-06T04:30:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.859+0000 7f30f17fa700 10 client.15084 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:30:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.860+0000 7f30f17fa700 1 -- 172.21.15.45:0/1725313589 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f8d0fbad00 con 0x55f8d1076120 2024-06-06T04:30:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.860+0000 7f30f17fa700 10 client.15084 handle_command_reply: tid=0 2024-06-06T04:30:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.860+0000 7f311aa7bb80 1 client.15084 shutdown 2024-06-06T04:30:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.860+0000 7f311aa7bb80 1 -- 172.21.15.45:0/1725313589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1062830 msgr2=0x55f8d1065e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.860+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/1725313589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1062830 0x55f8d1065e20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3100007940 tx=0x7f3100007c50 comp rx=0 tx=0).stop 2024-06-06T04:30:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.860+0000 7f311aa7bb80 1 -- 172.21.15.45:0/1725313589 shutdown_connections 2024-06-06T04:30:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.860+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/1725313589 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55f8d1076120 0x55f8d1077590 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.860+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/1725313589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1062830 0x55f8d1065e20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.860+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/1725313589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d105c750 0x55f8d10685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.860+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/1725313589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d1061f10 0x55f8d10658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.860+0000 7f30f17fa700 0 client.15084 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:30:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.860+0000 7f311aa7bb80 1 -- 172.21.15.45:0/1725313589 >> 172.21.15.45:0/1725313589 conn(0x55f8d0f89030 msgr2=0x55f8d10566b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.860+0000 7f311aa7bb80 1 -- 172.21.15.45:0/1725313589 shutdown_connections 2024-06-06T04:30:34.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.861+0000 7f311aa7bb80 1 -- 172.21.15.45:0/1725313589 wait complete. 2024-06-06T04:30:34.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.861+0000 7f311aa7bb80 20 client.15084 trim_cache size 0 max 16384 2024-06-06T04:30:34.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.882+0000 7f311aa7bb80 1 Processor -- start 2024-06-06T04:30:34.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.882+0000 7f311aa7bb80 1 -- start start 2024-06-06T04:30:34.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.882+0000 7f311aa7bb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d105c750 0x55f8d105c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.882+0000 7f311aa7bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1061f10 0x55f8d1065e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.882+0000 7f311aa7bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d1062830 0x55f8d10633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.882+0000 7f311aa7bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f8d10758d0 con 0x55f8d1061f10 2024-06-06T04:30:34.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.882+0000 7f311aa7bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8d1075fa0 con 0x55f8d1062830 2024-06-06T04:30:34.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.882+0000 7f311aa7bb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f8d1076670 con 0x55f8d105c750 2024-06-06T04:30:34.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.882+0000 7f310d793700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1061f10 0x55f8d1065e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.883+0000 7f310d793700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1061f10 0x55f8d1065e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37826/0 (socket says 172.21.15.45:37826) 2024-06-06T04:30:34.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.883+0000 7f310d793700 1 -- 172.21.15.45:0/1605804344 learned_addr learned my addr 172.21.15.45:0/1605804344 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:34.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.883+0000 7f310df94700 1 --2- 172.21.15.45:0/1605804344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d105c750 0x55f8d105c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.883+0000 7f310e795700 1 --2- 172.21.15.45:0/1605804344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d1062830 0x55f8d10633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.883+0000 7f310d793700 1 -- 172.21.15.45:0/1605804344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d105c750 msgr2=0x55f8d105c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:34.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.883+0000 7f310d793700 1 --2- 172.21.15.45:0/1605804344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d105c750 0x55f8d105c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.883+0000 7f310d793700 1 -- 172.21.15.45:0/1605804344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d1062830 msgr2=0x55f8d10633e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:34.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.883+0000 7f310d793700 1 --2- 172.21.15.45:0/1605804344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d1062830 0x55f8d10633e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.883+0000 7f310d793700 1 -- 172.21.15.45:0/1605804344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8d1069210 con 0x55f8d1061f10 2024-06-06T04:30:34.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.883+0000 7f310df94700 1 --2- 172.21.15.45:0/1605804344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d105c750 0x55f8d105c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:30:34.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.883+0000 7f310e795700 1 --2- 172.21.15.45:0/1605804344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d1062830 0x55f8d10633e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:30:34.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.883+0000 7f310d793700 1 --2- 172.21.15.45:0/1605804344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1061f10 0x55f8d1065e50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f30f800eda0 tx=0x7f30f800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a7bf7c717a6340 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:34.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.883+0000 7f30f37fe700 1 -- 172.21.15.45:0/1605804344 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8011e50 con 0x55f8d1061f10 2024-06-06T04:30:34.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.883+0000 7f30f37fe700 1 -- 172.21.15.45:0/1605804344 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30f8017030 con 0x55f8d1061f10 2024-06-06T04:30:34.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.883+0000 7f30f37fe700 1 -- 172.21.15.45:0/1605804344 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f801e460 con 0x55f8d1061f10 2024-06-06T04:30:34.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.884+0000 7f311aa7bb80 1 -- 172.21.15.45:0/1605804344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1061f10 msgr2=0x55f8d1065e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:34.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.884+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/1605804344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1061f10 0x55f8d1065e50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f30f800eda0 tx=0x7f30f800e960 comp rx=0 tx=0).stop 2024-06-06T04:30:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.884+0000 7f311aa7bb80 1 -- 172.21.15.45:0/1605804344 shutdown_connections 2024-06-06T04:30:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.884+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/1605804344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d1062830 0x55f8d10633e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.884+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/1605804344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d105c750 0x55f8d105c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.884+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/1605804344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1061f10 0x55f8d1065e50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.884+0000 7f311aa7bb80 1 -- 172.21.15.45:0/1605804344 >> 172.21.15.45:0/1605804344 conn(0x55f8d0f89030 msgr2=0x55f8d1062cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.884+0000 7f311aa7bb80 1 -- 172.21.15.45:0/1605804344 shutdown_connections 2024-06-06T04:30:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.884+0000 7f311aa7bb80 1 -- 172.21.15.45:0/1605804344 wait complete. 2024-06-06T04:30:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.884+0000 7f311aa7bb80 1 -- start start 2024-06-06T04:30:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.885+0000 7f311aa7bb80 1 Processor -- start 2024-06-06T04:30:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.885+0000 7f311aa7bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d105c750 0x55f8d1080f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.885+0000 7f311aa7bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1061f10 0x55f8d1056400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.885+0000 7f311aa7bb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d1062830 0x55f8d1056940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.885+0000 7f311aa7bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f8d1077410 con 0x55f8d1061f10 2024-06-06T04:30:34.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.885+0000 7f311aa7bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8d1076d40 con 0x55f8d105c750 2024-06-06T04:30:34.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.885+0000 7f311aa7bb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f8d1077ae0 con 0x55f8d1062830 2024-06-06T04:30:34.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.885+0000 7f311aa7bb80 10 client.0 _ll_register_callbacks cb 0x55f8d0f43560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:30:34.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.885+0000 7f310d793700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1061f10 0x55f8d1056400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.885+0000 7f310df94700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d105c750 0x55f8d1080f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.885+0000 7f310d793700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1061f10 0x55f8d1056400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37834/0 (socket says 172.21.15.45:37834) 2024-06-06T04:30:34.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.885+0000 7f310d793700 1 -- 172.21.15.45:0/2828023458 learned_addr learned my addr 172.21.15.45:0/2828023458 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:34.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.885+0000 7f310e795700 1 --2- 172.21.15.45:0/2828023458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d1062830 0x55f8d1056940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.885+0000 7f310d793700 1 -- 172.21.15.45:0/2828023458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d1062830 msgr2=0x55f8d1056940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:34.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.886+0000 7f310d793700 1 --2- 172.21.15.45:0/2828023458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d1062830 0x55f8d1056940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.886+0000 7f310d793700 1 -- 172.21.15.45:0/2828023458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d105c750 msgr2=0x55f8d1080f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:30:34.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.886+0000 7f310d793700 1 --2- 172.21.15.45:0/2828023458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d105c750 0x55f8d1080f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.886+0000 7f310d793700 1 -- 172.21.15.45:0/2828023458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8d1056f80 con 0x55f8d1061f10 2024-06-06T04:30:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.886+0000 7f310e795700 1 --2- 172.21.15.45:0/2828023458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d1062830 0x55f8d1056940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:30:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.886+0000 7f310df94700 1 --2- 172.21.15.45:0/2828023458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d105c750 0x55f8d1080f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:30:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.886+0000 7f310d793700 1 --2- 172.21.15.45:0/2828023458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1061f10 0x55f8d1056400 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009410 tx=0x7f30f8011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e9af94f8665312 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.886+0000 7f30f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:30:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.886+0000 7f30f17fa700 1 -- 172.21.15.45:0/2828023458 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8037030 con 0x55f8d1061f10 2024-06-06T04:30:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.886+0000 7f30f17fa700 1 -- 172.21.15.45:0/2828023458 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f30f8023040 con 0x55f8d1061f10 2024-06-06T04:30:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.886+0000 7f311aa7bb80 1 -- 172.21.15.45:0/2828023458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f8017030 con 0x55f8d1061f10 2024-06-06T04:30:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.886+0000 7f30f17fa700 1 -- 172.21.15.45:0/2828023458 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f30f8005350 con 0x55f8d1061f10 2024-06-06T04:30:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.886+0000 7f311aa7bb80 10 client.15090 fetch_fsmap learned FSMap version 26 2024-06-06T04:30:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.886+0000 7f311aa7bb80 1 -- 172.21.15.45:0/2828023458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f8d107f890 con 0x55f8d1061f10 2024-06-06T04:30:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.886+0000 7f30f17fa700 1 -- 172.21.15.45:0/2828023458 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30f8017030 con 0x55f8d1061f10 2024-06-06T04:30:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.887+0000 7f30f17fa700 1 -- 172.21.15.45:0/2828023458 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f30f800f040 con 0x55f8d1061f10 2024-06-06T04:30:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.887+0000 7f311aa7bb80 10 client.15090 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T04:30:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.887+0000 7f311aa7bb80 10 client.15090 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T04:30:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.887+0000 7f311aa7bb80 20 client.15090 populate_metadata read hostname 'smithi045' 2024-06-06T04:30:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.887+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/2828023458 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55f8d10e5c00 0x55f8d10e4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.887+0000 7f311aa7bb80 4 client.15090 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T04:30:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.887+0000 7f311aa7bb80 1 -- 172.21.15.45:0/2828023458 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "status"}) -- 0x55f8d0fb5280 con 0x55f8d10e5c00 2024-06-06T04:30:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.887+0000 7f310df94700 1 --2- 172.21.15.45:0/2828023458 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55f8d10e5c00 0x55f8d10e4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.887+0000 7f310df94700 1 --2- 172.21.15.45:0/2828023458 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55f8d10e5c00 0x55f8d10e4910 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84b297fb3f67b336 server_cookie=e09c87c086ba77ad in_seq=0 out_seq=0 2024-06-06T04:30:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.887+0000 7f30f17fa700 10 client.15090 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:30:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.887+0000 7f30f17fa700 1 -- 172.21.15.45:0/2828023458 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55f8d0fb5280 con 0x55f8d10e5c00 2024-06-06T04:30:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.887+0000 7f30f17fa700 10 client.15090 handle_command_reply: tid=0 2024-06-06T04:30:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f311aa7bb80 1 client.15090 shutdown 2024-06-06T04:30:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f311aa7bb80 1 -- 172.21.15.45:0/2828023458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1061f10 msgr2=0x55f8d1056400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/2828023458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1061f10 0x55f8d1056400 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009410 tx=0x7f30f8011e50 comp rx=0 tx=0).stop 2024-06-06T04:30:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f311aa7bb80 1 -- 172.21.15.45:0/2828023458 shutdown_connections 2024-06-06T04:30:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/2828023458 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55f8d10e5c00 0x55f8d10e4910 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/2828023458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f8d1062830 0x55f8d1056940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/2828023458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8d105c750 0x55f8d1080f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f30f17fa700 0 client.15090 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:30:34.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f311aa7bb80 1 --2- 172.21.15.45:0/2828023458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8d1061f10 0x55f8d1056400 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f311aa7bb80 1 -- 172.21.15.45:0/2828023458 >> 172.21.15.45:0/2828023458 conn(0x55f8d0f89030 msgr2=0x55f8d10609d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:34.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f311aa7bb80 1 -- 172.21.15.45:0/2828023458 shutdown_connections 2024-06-06T04:30:34.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f311aa7bb80 1 -- 172.21.15.45:0/2828023458 wait complete. 2024-06-06T04:30:34.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f311aa7bb80 20 client.15090 trim_cache size 0 max 16384 2024-06-06T04:30:34.894 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:30:34.894 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "c4f6b938-23bb-11ef-bc9b-c7b262605968", 2024-06-06T04:30:34.894 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-06-06T04:30:34.894 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24473, 2024-06-06T04:30:34.894 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-06-06T04:30:34.895 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-06-06T04:30:34.895 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-06-06T04:30:34.895 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 190.32447764899999, 2024-06-06T04:30:34.895 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2024-06-06T04:30:34.895 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 100, 2024-06-06T04:30:34.895 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 98, 2024-06-06T04:30:34.895 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 205.56195097899999 2024-06-06T04:30:34.895 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:30:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f31101f8700 1 -- 172.21.15.45:0/4185434938 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f30f40a2ca0 msgr2=0x7f30f40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f31101f8700 1 --2- 172.21.15.45:0/4185434938 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f30f40a2ca0 0x7f30f40a5160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f30f8005fd0 tx=0x7f30f8005ee0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f31101f8700 1 -- 172.21.15.45:0/4185434938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f310806ac40 msgr2=0x7f310819ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.888+0000 7f31101f8700 1 --2- 172.21.15.45:0/4185434938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f310806ac40 0x7f310819ae10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3104000c00 tx=0x7f3104003b30 comp rx=0 tx=0).stop 2024-06-06T04:30:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.889+0000 7f31101f8700 1 -- 172.21.15.45:0/4185434938 shutdown_connections 2024-06-06T04:30:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.889+0000 7f31101f8700 1 --2- 172.21.15.45:0/4185434938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f310810f240 0x7f310819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.889+0000 7f31101f8700 1 --2- 172.21.15.45:0/4185434938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f310806ac40 0x7f310819ae10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.889+0000 7f31101f8700 1 --2- 172.21.15.45:0/4185434938 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f30f40a2ca0 0x7f30f40a5160 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.889+0000 7f31101f8700 1 --2- 172.21.15.45:0/4185434938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f310806b560 0x7f310819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.889+0000 7f31101f8700 1 -- 172.21.15.45:0/4185434938 >> 172.21.15.45:0/4185434938 conn(0x7f3108078190 msgr2=0x7f31081109f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.889+0000 7f31101f8700 1 -- 172.21.15.45:0/4185434938 shutdown_connections 2024-06-06T04:30:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:34.889+0000 7f31101f8700 1 -- 172.21.15.45:0/4185434938 wait complete. 2024-06-06T04:30:35.608 INFO:tasks.cephadm:Setup complete, yielding 2024-06-06T04:30:35.609 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-06T04:30:35.619 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-06-06T04:30:35.619 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-06-06T04:30:35.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:35 smithi086 ceph-mon[36053]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:35.867 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:35.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:35 smithi154 ceph-mon[36428]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:35.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:35 smithi045 ceph-mon[32299]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:37.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.600+0000 7f80b6e83700 1 -- 172.21.15.45:0/3110405579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b0102fa0 msgr2=0x7f80b0105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:37.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.600+0000 7f80b6e83700 1 --2- 172.21.15.45:0/3110405579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b0102fa0 0x7f80b0105390 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009990 tx=0x7f80a4009ca0 comp rx=0 tx=0).stop 2024-06-06T04:30:37.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.600+0000 7f80b6e83700 1 -- 172.21.15.45:0/3110405579 shutdown_connections 2024-06-06T04:30:37.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.600+0000 7f80b6e83700 1 --2- 172.21.15.45:0/3110405579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80b0108200 0x7f80b010a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:37.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.600+0000 7f80b6e83700 1 --2- 172.21.15.45:0/3110405579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b0102fa0 0x7f80b0105390 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:37.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.600+0000 7f80b6e83700 1 --2- 172.21.15.45:0/3110405579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b01058d0 0x7f80b0107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:37.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.600+0000 7f80b6e83700 1 -- 172.21.15.45:0/3110405579 >> 172.21.15.45:0/3110405579 conn(0x7f80b00fc9b0 msgr2=0x7f80b00fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:37.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.600+0000 7f80b6e83700 1 -- 172.21.15.45:0/3110405579 shutdown_connections 2024-06-06T04:30:37.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.600+0000 7f80b6e83700 1 -- 172.21.15.45:0/3110405579 wait complete. 2024-06-06T04:30:37.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.600+0000 7f80b6e83700 1 Processor -- start 2024-06-06T04:30:37.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.600+0000 7f80b6e83700 1 -- start start 2024-06-06T04:30:37.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b6e83700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b0102fa0 0x7f80b01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:37.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b6e83700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80b01058d0 0x7f80b0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:37.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b6e83700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0108200 0x7f80b019fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:37.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b6e83700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f80b010df90 con 0x7f80b0102fa0 2024-06-06T04:30:37.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b6e83700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80b010de10 con 0x7f80b0108200 2024-06-06T04:30:37.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b6e83700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f80b010e110 con 0x7f80b01058d0 2024-06-06T04:30:37.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b4c1f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b0102fa0 0x7f80b01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:37.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b4c1f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b0102fa0 0x7f80b01984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37860/0 (socket says 172.21.15.45:37860) 2024-06-06T04:30:37.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b4c1f700 1 -- 172.21.15.45:0/2456489256 learned_addr learned my addr 172.21.15.45:0/2456489256 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:37.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80affff700 1 --2- 172.21.15.45:0/2456489256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80b01058d0 0x7f80b0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:37.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b5420700 1 --2- 172.21.15.45:0/2456489256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0108200 0x7f80b019fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:37.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b4c1f700 1 -- 172.21.15.45:0/2456489256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80b01058d0 msgr2=0x7f80b0198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:37.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b4c1f700 1 --2- 172.21.15.45:0/2456489256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80b01058d0 0x7f80b0198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:37.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b4c1f700 1 -- 172.21.15.45:0/2456489256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0108200 msgr2=0x7f80b019fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:37.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b4c1f700 1 --2- 172.21.15.45:0/2456489256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0108200 0x7f80b019fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:37.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b4c1f700 1 -- 172.21.15.45:0/2456489256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b01a0150 con 0x7f80b0102fa0 2024-06-06T04:30:37.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b5420700 1 --2- 172.21.15.45:0/2456489256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0108200 0x7f80b019fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:30:37.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80b4c1f700 1 --2- 172.21.15.45:0/2456489256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b0102fa0 0x7f80b01984f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f80a4000c00 tx=0x7f80a40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:37.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.601+0000 7f80adffb700 1 -- 172.21.15.45:0/2456489256 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4027070 con 0x7f80b0102fa0 2024-06-06T04:30:37.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.602+0000 7f80b6e83700 1 -- 172.21.15.45:0/2456489256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b01a03e0 con 0x7f80b0102fa0 2024-06-06T04:30:37.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.602+0000 7f80adffb700 1 -- 172.21.15.45:0/2456489256 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80a4022470 con 0x7f80b0102fa0 2024-06-06T04:30:37.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.602+0000 7f80adffb700 1 -- 172.21.15.45:0/2456489256 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a400f6c0 con 0x7f80b0102fa0 2024-06-06T04:30:37.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.602+0000 7f80b6e83700 1 -- 172.21.15.45:0/2456489256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b01a0890 con 0x7f80b0102fa0 2024-06-06T04:30:37.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.603+0000 7f80b6e83700 1 -- 172.21.15.45:0/2456489256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b0069180 con 0x7f80b0102fa0 2024-06-06T04:30:37.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.604+0000 7f80adffb700 1 -- 172.21.15.45:0/2456489256 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f80a402a070 con 0x7f80b0102fa0 2024-06-06T04:30:37.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.605+0000 7f80adffb700 1 --2- 172.21.15.45:0/2456489256 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f80a00a2c20 0x7f80a00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:37.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.605+0000 7f80affff700 1 --2- 172.21.15.45:0/2456489256 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f80a00a2c20 0x7f80a00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:37.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.605+0000 7f80adffb700 1 -- 172.21.15.45:0/2456489256 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f80a40d0710 con 0x7f80b0102fa0 2024-06-06T04:30:37.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.605+0000 7f80affff700 1 --2- 172.21.15.45:0/2456489256 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f80a00a2c20 0x7f80a00a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f809c00ac50 tx=0x7f809c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:37.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.609+0000 7f80adffb700 1 -- 172.21.15.45:0/2456489256 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f80a4031080 con 0x7f80b0102fa0 2024-06-06T04:30:37.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:37 smithi045 ceph-mon[32299]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:30:37.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.832+0000 7f80b6e83700 1 -- 172.21.15.45:0/2456489256 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f80b00628d0 con 0x7f80a00a2c20 2024-06-06T04:30:37.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.834+0000 7f80adffb700 1 -- 172.21.15.45:0/2456489256 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f80b00628d0 con 0x7f80a00a2c20 2024-06-06T04:30:37.834 INFO:teuthology.orchestra.run.smithi045.stdout:Backend: cephadm 2024-06-06T04:30:37.834 INFO:teuthology.orchestra.run.smithi045.stdout:Available: Yes 2024-06-06T04:30:37.835 INFO:teuthology.orchestra.run.smithi045.stdout:Paused: No 2024-06-06T04:30:37.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.836+0000 7f80b6e83700 1 -- 172.21.15.45:0/2456489256 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f80a00a2c20 msgr2=0x7f80a00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:37.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.836+0000 7f80b6e83700 1 --2- 172.21.15.45:0/2456489256 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f80a00a2c20 0x7f80a00a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f809c00ac50 tx=0x7f809c00a380 comp rx=0 tx=0).stop 2024-06-06T04:30:37.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.836+0000 7f80b6e83700 1 -- 172.21.15.45:0/2456489256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b0102fa0 msgr2=0x7f80b01984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:37.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.836+0000 7f80b6e83700 1 --2- 172.21.15.45:0/2456489256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b0102fa0 0x7f80b01984f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f80a4000c00 tx=0x7f80a40057c0 comp rx=0 tx=0).stop 2024-06-06T04:30:37.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.836+0000 7f80b6e83700 1 -- 172.21.15.45:0/2456489256 shutdown_connections 2024-06-06T04:30:37.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.836+0000 7f80b6e83700 1 --2- 172.21.15.45:0/2456489256 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f80a00a2c20 0x7f80a00a50e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:37.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.836+0000 7f80b6e83700 1 --2- 172.21.15.45:0/2456489256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0108200 0x7f80b019fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:37.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.836+0000 7f80b6e83700 1 --2- 172.21.15.45:0/2456489256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80b0102fa0 0x7f80b01984f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:37.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.836+0000 7f80b6e83700 1 --2- 172.21.15.45:0/2456489256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80b01058d0 0x7f80b0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:37.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.836+0000 7f80b6e83700 1 -- 172.21.15.45:0/2456489256 >> 172.21.15.45:0/2456489256 conn(0x7f80b00fc9b0 msgr2=0x7f80b01038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:37.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.836+0000 7f80b6e83700 1 -- 172.21.15.45:0/2456489256 shutdown_connections 2024-06-06T04:30:37.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:37.836+0000 7f80b6e83700 1 -- 172.21.15.45:0/2456489256 wait complete. 2024-06-06T04:30:37.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:37 smithi086 ceph-mon[36053]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:30:37.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:37 smithi154 ceph-mon[36428]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:30:38.471 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-06T04:30:38.722 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:38.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:38 smithi045 ceph-mon[32299]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:30:38.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:38 smithi086 ceph-mon[36053]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:30:38.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:38 smithi154 ceph-mon[36428]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:30:39.839 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:39 smithi045 ceph-mon[32299]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:30:39.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:39 smithi086 ceph-mon[36053]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:30:39.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:39 smithi154 ceph-mon[36428]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:30:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.415+0000 7f0dd24b4700 1 -- 172.21.15.45:0/2738471429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dcc103700 msgr2=0x7f0dcc103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.415+0000 7f0dd24b4700 1 --2- 172.21.15.45:0/2738471429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dcc103700 0x7f0dcc103b60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00099c0 tx=0x7f0dc0009cd0 comp rx=0 tx=0).stop 2024-06-06T04:30:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.415+0000 7f0dd24b4700 1 -- 172.21.15.45:0/2738471429 shutdown_connections 2024-06-06T04:30:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.415+0000 7f0dd24b4700 1 --2- 172.21.15.45:0/2738471429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0dcc1040a0 0x7f0dcc10a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.415+0000 7f0dd24b4700 1 --2- 172.21.15.45:0/2738471429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dcc102500 0x7f0dcc102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.415+0000 7f0dd24b4700 1 --2- 172.21.15.45:0/2738471429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dcc103700 0x7f0dcc103b60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.415+0000 7f0dd24b4700 1 -- 172.21.15.45:0/2738471429 >> 172.21.15.45:0/2738471429 conn(0x7f0dcc0fdaa0 msgr2=0x7f0dcc0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.415+0000 7f0dd24b4700 1 -- 172.21.15.45:0/2738471429 shutdown_connections 2024-06-06T04:30:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.415+0000 7f0dd24b4700 1 -- 172.21.15.45:0/2738471429 wait complete. 2024-06-06T04:30:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.416+0000 7f0dd24b4700 1 Processor -- start 2024-06-06T04:30:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.416+0000 7f0dd24b4700 1 -- start start 2024-06-06T04:30:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.416+0000 7f0dd24b4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dcc102500 0x7f0dcc198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.416+0000 7f0dd24b4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dcc103700 0x7f0dcc198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.416+0000 7f0dd24b4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0dcc1040a0 0x7f0dcc19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.416+0000 7f0dd24b4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc10dfa0 con 0x7f0dcc102500 2024-06-06T04:30:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.416+0000 7f0dd24b4700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc10de20 con 0x7f0dcc103700 2024-06-06T04:30:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.416+0000 7f0dd24b4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc10e120 con 0x7f0dcc1040a0 2024-06-06T04:30:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.416+0000 7f0dcbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dcc102500 0x7f0dcc198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.416+0000 7f0dcbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dcc102500 0x7f0dcc198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36916/0 (socket says 172.21.15.45:36916) 2024-06-06T04:30:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.416+0000 7f0dcbfff700 1 -- 172.21.15.45:0/830207488 learned_addr learned my addr 172.21.15.45:0/830207488 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:40.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.417+0000 7f0dcb7fe700 1 --2- 172.21.15.45:0/830207488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dcc103700 0x7f0dcc198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.417+0000 7f0dd0a51700 1 --2- 172.21.15.45:0/830207488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0dcc1040a0 0x7f0dcc19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.417+0000 7f0dcbfff700 1 -- 172.21.15.45:0/830207488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0dcc1040a0 msgr2=0x7f0dcc19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.417+0000 7f0dcbfff700 1 --2- 172.21.15.45:0/830207488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0dcc1040a0 0x7f0dcc19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.417+0000 7f0dcbfff700 1 -- 172.21.15.45:0/830207488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dcc103700 msgr2=0x7f0dcc198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.417+0000 7f0dcbfff700 1 --2- 172.21.15.45:0/830207488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dcc103700 0x7f0dcc198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.417+0000 7f0dcbfff700 1 -- 172.21.15.45:0/830207488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc1a0260 con 0x7f0dcc102500 2024-06-06T04:30:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.417+0000 7f0dcbfff700 1 --2- 172.21.15.45:0/830207488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dcc102500 0x7f0dcc198600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00d9b0 tx=0x7f0dbc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.417+0000 7f0dc97fa700 1 -- 172.21.15.45:0/830207488 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc016070 con 0x7f0dcc102500 2024-06-06T04:30:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.417+0000 7f0dd24b4700 1 -- 172.21.15.45:0/830207488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dcc1a0550 con 0x7f0dcc102500 2024-06-06T04:30:40.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.417+0000 7f0dc97fa700 1 -- 172.21.15.45:0/830207488 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dbc00be00 con 0x7f0dcc102500 2024-06-06T04:30:40.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.417+0000 7f0dc97fa700 1 -- 172.21.15.45:0/830207488 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc00f3c0 con 0x7f0dcc102500 2024-06-06T04:30:40.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.417+0000 7f0dd24b4700 1 -- 172.21.15.45:0/830207488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dcc1a0ae0 con 0x7f0dcc102500 2024-06-06T04:30:40.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.418+0000 7f0dd24b4700 1 -- 172.21.15.45:0/830207488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dcc069180 con 0x7f0dcc102500 2024-06-06T04:30:40.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.420+0000 7f0dc97fa700 1 -- 172.21.15.45:0/830207488 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0dbc019070 con 0x7f0dcc102500 2024-06-06T04:30:40.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.420+0000 7f0dc97fa700 1 --2- 172.21.15.45:0/830207488 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0db40a2c20 0x7f0db40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:40.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.420+0000 7f0dcb7fe700 1 --2- 172.21.15.45:0/830207488 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0db40a2c20 0x7f0db40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:40.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.420+0000 7f0dc97fa700 1 -- 172.21.15.45:0/830207488 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f0dbc0cf820 con 0x7f0dcc102500 2024-06-06T04:30:40.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.421+0000 7f0dcb7fe700 1 --2- 172.21.15.45:0/830207488 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0db40a2c20 0x7f0db40a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00037f0 tx=0x7f0dc0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:40.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.424+0000 7f0dc97fa700 1 -- 172.21.15.45:0/830207488 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0dbc020020 con 0x7f0dcc102500 2024-06-06T04:30:40.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.641+0000 7f0dd24b4700 1 -- 172.21.15.45:0/830207488 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0dcc0628d0 con 0x7f0db40a2c20 2024-06-06T04:30:40.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.654+0000 7f0dc97fa700 1 -- 172.21.15.45:0/830207488 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f0dcc0628d0 con 0x7f0db40a2c20 2024-06-06T04:30:40.655 INFO:teuthology.orchestra.run.smithi045.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-06T04:30:40.655 INFO:teuthology.orchestra.run.smithi045.stdout:mds.a smithi045 running (3m) 40s ago 3m 16.6M - 19.0.0-2946-g3d89accf b045ac545f82 4c72eb5e2764 2024-06-06T04:30:40.655 INFO:teuthology.orchestra.run.smithi045.stdout:mds.b smithi086 running (3m) 75s ago 3m 28.5M - 19.0.0-2946-g3d89accf b045ac545f82 fca99af9487b 2024-06-06T04:30:40.655 INFO:teuthology.orchestra.run.smithi045.stdout:mds.c smithi154 running (3m) 75s ago 3m 18.1M - 19.0.0-2946-g3d89accf b045ac545f82 4bf74394285b 2024-06-06T04:30:40.655 INFO:teuthology.orchestra.run.smithi045.stdout:mds.d smithi045 running (3m) 40s ago 3m 14.4M - 19.0.0-2946-g3d89accf b045ac545f82 fc958d977ba8 2024-06-06T04:30:40.655 INFO:teuthology.orchestra.run.smithi045.stdout:mds.e smithi086 running (3m) 75s ago 3m 17.4M - 19.0.0-2946-g3d89accf b045ac545f82 3feb81881c0d 2024-06-06T04:30:40.655 INFO:teuthology.orchestra.run.smithi045.stdout:mds.f smithi154 running (3m) 75s ago 3m 17.0M - 19.0.0-2946-g3d89accf b045ac545f82 b1c620c0d3be 2024-06-06T04:30:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:mds.g smithi045 running (3m) 40s ago 3m 15.8M - 19.0.0-2946-g3d89accf b045ac545f82 011a566e7a77 2024-06-06T04:30:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:mds.h smithi086 running (3m) 75s ago 3m 20.0M - 19.0.0-2946-g3d89accf b045ac545f82 8c8c48e2f255 2024-06-06T04:30:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:mds.i smithi154 running (3m) 75s ago 3m 18.2M - 19.0.0-2946-g3d89accf b045ac545f82 456a88d250d1 2024-06-06T04:30:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:mds.j smithi045 running (4m) 40s ago 3m 14.6M - 19.0.0-2946-g3d89accf b045ac545f82 ef69f4b851cd 2024-06-06T04:30:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:mds.k smithi086 running (4m) 75s ago 4m 16.0M - 19.0.0-2946-g3d89accf b045ac545f82 21cbfb7954d1 2024-06-06T04:30:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:mds.l smithi154 running (4m) 75s ago 4m 14.5M - 19.0.0-2946-g3d89accf b045ac545f82 3ca7f1261d92 2024-06-06T04:30:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.x smithi045 *:9283,8765 running (11m) 40s ago 11m 518M - 19.0.0-2946-g3d89accf b045ac545f82 8bc2882f9457 2024-06-06T04:30:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.y smithi086 *:8443,8765 running (9m) 75s ago 9m 456M - 19.0.0-2946-g3d89accf b045ac545f82 98975c069f1f 2024-06-06T04:30:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.z smithi154 *:8443,8765 running (9m) 75s ago 9m 455M - 19.0.0-2946-g3d89accf b045ac545f82 c34767ed5a64 2024-06-06T04:30:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:mon.a smithi045 running (11m) 40s ago 11m 50.7M 2048M 19.0.0-2946-g3d89accf b045ac545f82 3a75fd6d5b46 2024-06-06T04:30:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:mon.b smithi086 running (9m) 75s ago 9m 40.3M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b0595627ff89 2024-06-06T04:30:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:mon.c smithi154 running (9m) 75s ago 9m 38.9M 2048M 19.0.0-2946-g3d89accf b045ac545f82 6742d6b94ee1 2024-06-06T04:30:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 smithi045 running (9m) 40s ago 9m 59.3M 4273M 19.0.0-2946-g3d89accf b045ac545f82 073ab2c9831f 2024-06-06T04:30:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 smithi086 running (8m) 75s ago 8m 50.1M 4272M 19.0.0-2946-g3d89accf b045ac545f82 d341ee4b1d67 2024-06-06T04:30:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 smithi154 running (8m) 75s ago 8m 46.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 48ae47577009 2024-06-06T04:30:40.656 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 smithi045 running (7m) 40s ago 7m 55.4M 4273M 19.0.0-2946-g3d89accf b045ac545f82 a5661291dc16 2024-06-06T04:30:40.657 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 smithi086 running (7m) 75s ago 7m 43.5M 4272M 19.0.0-2946-g3d89accf b045ac545f82 ef5f3da54c6c 2024-06-06T04:30:40.657 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 smithi154 running (7m) 75s ago 7m 56.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 2b985706de73 2024-06-06T04:30:40.657 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 smithi045 running (6m) 40s ago 6m 52.8M 4273M 19.0.0-2946-g3d89accf b045ac545f82 cfd33efb5f7f 2024-06-06T04:30:40.657 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 smithi086 running (6m) 75s ago 6m 55.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 539d7e11074d 2024-06-06T04:30:40.657 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 smithi154 running (5m) 75s ago 5m 44.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 5c829d3bf712 2024-06-06T04:30:40.657 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 smithi045 running (5m) 40s ago 5m 45.4M 4273M 19.0.0-2946-g3d89accf b045ac545f82 9c168550f573 2024-06-06T04:30:40.657 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 smithi086 running (4m) 75s ago 4m 44.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 171a037fe253 2024-06-06T04:30:40.657 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 smithi154 running (4m) 75s ago 4m 52.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 c4e6efc6431e 2024-06-06T04:30:40.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.657+0000 7f0dd24b4700 1 -- 172.21.15.45:0/830207488 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0db40a2c20 msgr2=0x7f0db40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:40.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.657+0000 7f0dd24b4700 1 --2- 172.21.15.45:0/830207488 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0db40a2c20 0x7f0db40a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00037f0 tx=0x7f0dc0005fb0 comp rx=0 tx=0).stop 2024-06-06T04:30:40.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.657+0000 7f0dd24b4700 1 -- 172.21.15.45:0/830207488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dcc102500 msgr2=0x7f0dcc198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:40.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.657+0000 7f0dd24b4700 1 --2- 172.21.15.45:0/830207488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dcc102500 0x7f0dcc198600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00d9b0 tx=0x7f0dbc00dcc0 comp rx=0 tx=0).stop 2024-06-06T04:30:40.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.657+0000 7f0dd24b4700 1 -- 172.21.15.45:0/830207488 shutdown_connections 2024-06-06T04:30:40.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.657+0000 7f0dd24b4700 1 --2- 172.21.15.45:0/830207488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0dcc1040a0 0x7f0dcc19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:40.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.657+0000 7f0dd24b4700 1 --2- 172.21.15.45:0/830207488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dcc102500 0x7f0dcc198600 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:40.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.657+0000 7f0dd24b4700 1 --2- 172.21.15.45:0/830207488 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f0db40a2c20 0x7f0db40a50e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.657+0000 7f0dd24b4700 1 --2- 172.21.15.45:0/830207488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dcc103700 0x7f0dcc198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.657+0000 7f0dd24b4700 1 -- 172.21.15.45:0/830207488 >> 172.21.15.45:0/830207488 conn(0x7f0dcc0fdaa0 msgr2=0x7f0dcc106aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.657+0000 7f0dd24b4700 1 -- 172.21.15.45:0/830207488 shutdown_connections 2024-06-06T04:30:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:40.657+0000 7f0dd24b4700 1 -- 172.21.15.45:0/830207488 wait complete. 2024-06-06T04:30:41.356 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-06T04:30:41.609 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:41.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:41 smithi045 ceph-mon[32299]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:41.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:41 smithi045 ceph-mon[32299]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:30:41.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:41 smithi086 ceph-mon[36053]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:41.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:41 smithi086 ceph-mon[36053]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:30:41.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:41 smithi154 ceph-mon[36428]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:41.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:41 smithi154 ceph-mon[36428]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:30:43.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.313+0000 7f2069c88700 1 -- 172.21.15.45:0/3649991857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20641036f0 msgr2=0x7f2064103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:43.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.313+0000 7f2069c88700 1 --2- 172.21.15.45:0/3649991857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20641036f0 0x7f2064103b50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f20580099c0 tx=0x7f2058009cd0 comp rx=0 tx=0).stop 2024-06-06T04:30:43.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.313+0000 7f2069c88700 1 -- 172.21.15.45:0/3649991857 shutdown_connections 2024-06-06T04:30:43.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.313+0000 7f2069c88700 1 --2- 172.21.15.45:0/3649991857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2064104090 0x7f206410a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:43.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.313+0000 7f2069c88700 1 --2- 172.21.15.45:0/3649991857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20641024f0 0x7f2064102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:43.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.313+0000 7f2069c88700 1 --2- 172.21.15.45:0/3649991857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20641036f0 0x7f2064103b50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:43.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.313+0000 7f2069c88700 1 -- 172.21.15.45:0/3649991857 >> 172.21.15.45:0/3649991857 conn(0x7f20640fda90 msgr2=0x7f20640ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:43.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.313+0000 7f2069c88700 1 -- 172.21.15.45:0/3649991857 shutdown_connections 2024-06-06T04:30:43.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.313+0000 7f2069c88700 1 -- 172.21.15.45:0/3649991857 wait complete. 2024-06-06T04:30:43.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.313+0000 7f2069c88700 1 Processor -- start 2024-06-06T04:30:43.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f2069c88700 1 -- start start 2024-06-06T04:30:43.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f2069c88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20641024f0 0x7f20640761e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:43.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f2069c88700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20641036f0 0x7f2064074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:43.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f2069c88700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2064104090 0x7f2064074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:43.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f2069c88700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f206410e080 con 0x7f20641024f0 2024-06-06T04:30:43.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f2069c88700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f206410df00 con 0x7f2064104090 2024-06-06T04:30:43.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f2069c88700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f206410e200 con 0x7f20641036f0 2024-06-06T04:30:43.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f20637fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20641024f0 0x7f20640761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:43.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f20637fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20641024f0 0x7f20640761e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36934/0 (socket says 172.21.15.45:36934) 2024-06-06T04:30:43.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f20637fe700 1 -- 172.21.15.45:0/2255864534 learned_addr learned my addr 172.21.15.45:0/2255864534 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f2062ffd700 1 --2- 172.21.15.45:0/2255864534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20641036f0 0x7f2064074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f2063fff700 1 --2- 172.21.15.45:0/2255864534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2064104090 0x7f2064074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f20637fe700 1 -- 172.21.15.45:0/2255864534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20641036f0 msgr2=0x7f2064074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f20637fe700 1 --2- 172.21.15.45:0/2255864534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20641036f0 0x7f2064074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f20637fe700 1 -- 172.21.15.45:0/2255864534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2064104090 msgr2=0x7f2064074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f20637fe700 1 --2- 172.21.15.45:0/2255864534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2064104090 0x7f2064074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.314+0000 7f20637fe700 1 -- 172.21.15.45:0/2255864534 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2064075690 con 0x7f20641024f0 2024-06-06T04:30:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.315+0000 7f20637fe700 1 --2- 172.21.15.45:0/2255864534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20641024f0 0x7f20640761e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f205400b500 tx=0x7f205400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.315+0000 7f2060ff9700 1 -- 172.21.15.45:0/2255864534 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054015020 con 0x7f20641024f0 2024-06-06T04:30:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.315+0000 7f2060ff9700 1 -- 172.21.15.45:0/2255864534 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2054011ce0 con 0x7f20641024f0 2024-06-06T04:30:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.315+0000 7f2060ff9700 1 -- 172.21.15.45:0/2255864534 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054010c80 con 0x7f20641024f0 2024-06-06T04:30:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.315+0000 7f2069c88700 1 -- 172.21.15.45:0/2255864534 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20641a28f0 con 0x7f20641024f0 2024-06-06T04:30:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.315+0000 7f2069c88700 1 -- 172.21.15.45:0/2255864534 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20641a2ed0 con 0x7f20641024f0 2024-06-06T04:30:43.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.316+0000 7f2069c88700 1 -- 172.21.15.45:0/2255864534 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2064069180 con 0x7f20641024f0 2024-06-06T04:30:43.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.318+0000 7f2060ff9700 1 -- 172.21.15.45:0/2255864534 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2054018020 con 0x7f20641024f0 2024-06-06T04:30:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.319+0000 7f2060ff9700 1 --2- 172.21.15.45:0/2255864534 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f204c0a2ca0 0x7f204c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.319+0000 7f2060ff9700 1 -- 172.21.15.45:0/2255864534 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f20540cf0f0 con 0x7f20641024f0 2024-06-06T04:30:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.319+0000 7f2062ffd700 1 --2- 172.21.15.45:0/2255864534 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f204c0a2ca0 0x7f204c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:43.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.319+0000 7f2062ffd700 1 --2- 172.21.15.45:0/2255864534 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f204c0a2ca0 0x7f204c0a5160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2058000c00 tx=0x7f2058005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:43.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.323+0000 7f2060ff9700 1 -- 172.21.15.45:0/2255864534 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f205401f020 con 0x7f20641024f0 2024-06-06T04:30:43.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.541+0000 7f2069c88700 1 -- 172.21.15.45:0/2255864534 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f20640628d0 con 0x7f204c0a2ca0 2024-06-06T04:30:43.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.548+0000 7f2060ff9700 1 -- 172.21.15.45:0/2255864534 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f20640628d0 con 0x7f204c0a2ca0 2024-06-06T04:30:43.548 INFO:teuthology.orchestra.run.smithi045.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-06T04:30:43.548 INFO:teuthology.orchestra.run.smithi045.stdout:mds.all 12/12 78s ago 4m smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l;count:12 2024-06-06T04:30:43.548 INFO:teuthology.orchestra.run.smithi045.stdout:mgr 3/3 78s ago 9m smithi045=x;smithi086=y;smithi154=z;count:3 2024-06-06T04:30:43.548 INFO:teuthology.orchestra.run.smithi045.stdout:mon 3/3 78s ago 9m smithi045:172.21.15.45=a;smithi086:172.21.15.86=b;smithi154:172.21.15.154=c;count:3 2024-06-06T04:30:43.548 INFO:teuthology.orchestra.run.smithi045.stdout:osd 12 78s ago - 2024-06-06T04:30:43.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.550+0000 7f2069c88700 1 -- 172.21.15.45:0/2255864534 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f204c0a2ca0 msgr2=0x7f204c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:43.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.550+0000 7f2069c88700 1 --2- 172.21.15.45:0/2255864534 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f204c0a2ca0 0x7f204c0a5160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2058000c00 tx=0x7f2058005fb0 comp rx=0 tx=0).stop 2024-06-06T04:30:43.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.550+0000 7f2069c88700 1 -- 172.21.15.45:0/2255864534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20641024f0 msgr2=0x7f20640761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:43.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.550+0000 7f2069c88700 1 --2- 172.21.15.45:0/2255864534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20641024f0 0x7f20640761e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f205400b500 tx=0x7f205400b810 comp rx=0 tx=0).stop 2024-06-06T04:30:43.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.550+0000 7f2069c88700 1 -- 172.21.15.45:0/2255864534 shutdown_connections 2024-06-06T04:30:43.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.550+0000 7f2069c88700 1 --2- 172.21.15.45:0/2255864534 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f204c0a2ca0 0x7f204c0a5160 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:43.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.550+0000 7f2069c88700 1 --2- 172.21.15.45:0/2255864534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2064104090 0x7f2064074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:43.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.550+0000 7f2069c88700 1 --2- 172.21.15.45:0/2255864534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20641024f0 0x7f20640761e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:43.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.550+0000 7f2069c88700 1 --2- 172.21.15.45:0/2255864534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20641036f0 0x7f2064074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:43.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.550+0000 7f2069c88700 1 -- 172.21.15.45:0/2255864534 >> 172.21.15.45:0/2255864534 conn(0x7f20640fda90 msgr2=0x7f2064106c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:43.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.550+0000 7f2069c88700 1 -- 172.21.15.45:0/2255864534 shutdown_connections 2024-06-06T04:30:43.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:43.550+0000 7f2069c88700 1 -- 172.21.15.45:0/2255864534 wait complete. 2024-06-06T04:30:43.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:43 smithi086 ceph-mon[36053]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:30:43.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:43 smithi154 ceph-mon[36428]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:30:43.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:43 smithi045 ceph-mon[32299]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:30:44.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-06T04:30:44.596 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:44.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:44 smithi086 ceph-mon[36053]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:30:44.860 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:44 smithi045 ceph-mon[32299]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:30:44.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:44 smithi154 ceph-mon[36428]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:30:45.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:45 smithi045 ceph-mon[32299]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:45.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:45 smithi086 ceph-mon[36053]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:45.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:45 smithi154 ceph-mon[36428]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:46.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.303+0000 7f3b91870700 1 -- 172.21.15.45:0/458906338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c102600 msgr2=0x7f3b8c102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:46.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.303+0000 7f3b91870700 1 --2- 172.21.15.45:0/458906338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c102600 0x7f3b8c102a60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3b800099c0 tx=0x7f3b80009cd0 comp rx=0 tx=0).stop 2024-06-06T04:30:46.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.304+0000 7f3b91870700 1 -- 172.21.15.45:0/458906338 shutdown_connections 2024-06-06T04:30:46.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.304+0000 7f3b91870700 1 --2- 172.21.15.45:0/458906338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c102fa0 0x7f3b8c109850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:46.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.304+0000 7f3b91870700 1 --2- 172.21.15.45:0/458906338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b8c101400 0x7f3b8c101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:46.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.304+0000 7f3b91870700 1 --2- 172.21.15.45:0/458906338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c102600 0x7f3b8c102a60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:46.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.304+0000 7f3b91870700 1 -- 172.21.15.45:0/458906338 >> 172.21.15.45:0/458906338 conn(0x7f3b8c0fc9c0 msgr2=0x7f3b8c0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:46.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.304+0000 7f3b91870700 1 -- 172.21.15.45:0/458906338 shutdown_connections 2024-06-06T04:30:46.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.304+0000 7f3b91870700 1 -- 172.21.15.45:0/458906338 wait complete. 2024-06-06T04:30:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.304+0000 7f3b91870700 1 Processor -- start 2024-06-06T04:30:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.304+0000 7f3b91870700 1 -- start start 2024-06-06T04:30:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b91870700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c101400 0x7f3b8c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b91870700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c102600 0x7f3b8c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b91870700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b8c102fa0 0x7f3b8c19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b91870700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c10ce80 con 0x7f3b8c102600 2024-06-06T04:30:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b91870700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c10cd00 con 0x7f3b8c101400 2024-06-06T04:30:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b91870700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c10d000 con 0x7f3b8c102fa0 2024-06-06T04:30:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b8a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c102600 0x7f3b8c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b8a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c102600 0x7f3b8c198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36954/0 (socket says 172.21.15.45:36954) 2024-06-06T04:30:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b8a7fc700 1 -- 172.21.15.45:0/3920831836 learned_addr learned my addr 172.21.15.45:0/3920831836 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b8affd700 1 --2- 172.21.15.45:0/3920831836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c101400 0x7f3b8c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b8b7fe700 1 --2- 172.21.15.45:0/3920831836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b8c102fa0 0x7f3b8c19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b8a7fc700 1 -- 172.21.15.45:0/3920831836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b8c102fa0 msgr2=0x7f3b8c19fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b8a7fc700 1 --2- 172.21.15.45:0/3920831836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b8c102fa0 0x7f3b8c19fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b8a7fc700 1 -- 172.21.15.45:0/3920831836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c101400 msgr2=0x7f3b8c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b8a7fc700 1 --2- 172.21.15.45:0/3920831836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c101400 0x7f3b8c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b8a7fc700 1 -- 172.21.15.45:0/3920831836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8c1a02c0 con 0x7f3b8c102600 2024-06-06T04:30:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b8a7fc700 1 --2- 172.21.15.45:0/3920831836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c102600 0x7f3b8c198ba0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3b80000c00 tx=0x7f3b800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.305+0000 7f3b9086e700 1 -- 172.21.15.45:0/3920831836 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80027070 con 0x7f3b8c102600 2024-06-06T04:30:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.306+0000 7f3b9086e700 1 -- 172.21.15.45:0/3920831836 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b80004050 con 0x7f3b8c102600 2024-06-06T04:30:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.306+0000 7f3b91870700 1 -- 172.21.15.45:0/3920831836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8c1a0550 con 0x7f3b8c102600 2024-06-06T04:30:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.306+0000 7f3b9086e700 1 -- 172.21.15.45:0/3920831836 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80022cd0 con 0x7f3b8c102600 2024-06-06T04:30:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.306+0000 7f3b91870700 1 -- 172.21.15.45:0/3920831836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b8c1a0a00 con 0x7f3b8c102600 2024-06-06T04:30:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.307+0000 7f3b91870700 1 -- 172.21.15.45:0/3920831836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b8c069180 con 0x7f3b8c102600 2024-06-06T04:30:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.308+0000 7f3b9086e700 1 -- 172.21.15.45:0/3920831836 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3b8002a070 con 0x7f3b8c102600 2024-06-06T04:30:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.308+0000 7f3b9086e700 1 --2- 172.21.15.45:0/3920831836 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b740a2ca0 0x7f3b740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.309+0000 7f3b8affd700 1 --2- 172.21.15.45:0/3920831836 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b740a2ca0 0x7f3b740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.309+0000 7f3b9086e700 1 -- 172.21.15.45:0/3920831836 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f3b800d0960 con 0x7f3b8c102600 2024-06-06T04:30:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.309+0000 7f3b8affd700 1 --2- 172.21.15.45:0/3920831836 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b740a2ca0 0x7f3b740a5160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c102460 tx=0x7f3b7c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:46.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.312+0000 7f3b9086e700 1 -- 172.21.15.45:0/3920831836 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3b80031070 con 0x7f3b8c102600 2024-06-06T04:30:46.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.526+0000 7f3b91870700 1 -- 172.21.15.45:0/3920831836 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f3b8c1a0d20 con 0x7f3b740a2ca0 2024-06-06T04:30:46.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.527+0000 7f3b9086e700 1 -- 172.21.15.45:0/3920831836 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f3b8c1a0d20 con 0x7f3b740a2ca0 2024-06-06T04:30:46.528 INFO:teuthology.orchestra.run.smithi045.stdout:HOST ADDR LABELS STATUS 2024-06-06T04:30:46.528 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 172.21.15.45 2024-06-06T04:30:46.528 INFO:teuthology.orchestra.run.smithi045.stdout:smithi086 172.21.15.86 2024-06-06T04:30:46.528 INFO:teuthology.orchestra.run.smithi045.stdout:smithi154 172.21.15.154 2024-06-06T04:30:46.528 INFO:teuthology.orchestra.run.smithi045.stdout:3 hosts in cluster 2024-06-06T04:30:46.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.530+0000 7f3b91870700 1 -- 172.21.15.45:0/3920831836 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b740a2ca0 msgr2=0x7f3b740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:46.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.530+0000 7f3b91870700 1 --2- 172.21.15.45:0/3920831836 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b740a2ca0 0x7f3b740a5160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c102460 tx=0x7f3b7c00a380 comp rx=0 tx=0).stop 2024-06-06T04:30:46.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.530+0000 7f3b91870700 1 -- 172.21.15.45:0/3920831836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c102600 msgr2=0x7f3b8c198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:46.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.530+0000 7f3b91870700 1 --2- 172.21.15.45:0/3920831836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c102600 0x7f3b8c198ba0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3b80000c00 tx=0x7f3b800047b0 comp rx=0 tx=0).stop 2024-06-06T04:30:46.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.530+0000 7f3b91870700 1 -- 172.21.15.45:0/3920831836 shutdown_connections 2024-06-06T04:30:46.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.530+0000 7f3b91870700 1 --2- 172.21.15.45:0/3920831836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b8c102fa0 0x7f3b8c19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:46.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.530+0000 7f3b91870700 1 --2- 172.21.15.45:0/3920831836 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3b740a2ca0 0x7f3b740a5160 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:46.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.530+0000 7f3b91870700 1 --2- 172.21.15.45:0/3920831836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b8c101400 0x7f3b8c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:46.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.530+0000 7f3b91870700 1 --2- 172.21.15.45:0/3920831836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c102600 0x7f3b8c198ba0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:46.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.530+0000 7f3b91870700 1 -- 172.21.15.45:0/3920831836 >> 172.21.15.45:0/3920831836 conn(0x7f3b8c0fc9c0 msgr2=0x7f3b8c0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:46.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.530+0000 7f3b91870700 1 -- 172.21.15.45:0/3920831836 shutdown_connections 2024-06-06T04:30:46.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:46.530+0000 7f3b91870700 1 -- 172.21.15.45:0/3920831836 wait complete. 2024-06-06T04:30:47.216 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-06T04:30:47.469 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:47.856 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:47 smithi045 ceph-mon[32299]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:30:47.856 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:47 smithi045 ceph-mon[32299]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:30:47.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:47 smithi086 ceph-mon[36053]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:30:47.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:47 smithi086 ceph-mon[36053]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:30:47.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:47 smithi154 ceph-mon[36428]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:30:47.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:47 smithi154 ceph-mon[36428]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:30:49.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.210+0000 7f54e9c31700 1 -- 172.21.15.45:0/3060962089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54e41024f0 msgr2=0x7f54e4102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:49.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.210+0000 7f54e9c31700 1 --2- 172.21.15.45:0/3060962089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54e41024f0 0x7f54e4102910 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f54d40099c0 tx=0x7f54d4009cd0 comp rx=0 tx=0).stop 2024-06-06T04:30:49.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.210+0000 7f54e9c31700 1 -- 172.21.15.45:0/3060962089 shutdown_connections 2024-06-06T04:30:49.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.210+0000 7f54e9c31700 1 --2- 172.21.15.45:0/3060962089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54e4104090 0x7f54e410a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:49.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.210+0000 7f54e9c31700 1 --2- 172.21.15.45:0/3060962089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54e41024f0 0x7f54e4102910 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:49.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.210+0000 7f54e9c31700 1 --2- 172.21.15.45:0/3060962089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54e41036f0 0x7f54e4103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:49.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.210+0000 7f54e9c31700 1 -- 172.21.15.45:0/3060962089 >> 172.21.15.45:0/3060962089 conn(0x7f54e40fda90 msgr2=0x7f54e40ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:49.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.210+0000 7f54e9c31700 1 -- 172.21.15.45:0/3060962089 shutdown_connections 2024-06-06T04:30:49.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.210+0000 7f54e9c31700 1 -- 172.21.15.45:0/3060962089 wait complete. 2024-06-06T04:30:49.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.211+0000 7f54e9c31700 1 Processor -- start 2024-06-06T04:30:49.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.211+0000 7f54e9c31700 1 -- start start 2024-06-06T04:30:49.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.211+0000 7f54e9c31700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54e41024f0 0x7f54e4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:49.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.211+0000 7f54e9c31700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54e41036f0 0x7f54e4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:49.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.211+0000 7f54e9c31700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54e4104090 0x7f54e419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:49.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.211+0000 7f54e9c31700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f54e410dff0 con 0x7f54e41024f0 2024-06-06T04:30:49.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.211+0000 7f54e9c31700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f54e410de70 con 0x7f54e4104090 2024-06-06T04:30:49.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.211+0000 7f54e9c31700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f54e410e170 con 0x7f54e41036f0 2024-06-06T04:30:49.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.211+0000 7f54e37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54e41024f0 0x7f54e4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:49.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.211+0000 7f54e37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54e41024f0 0x7f54e4198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39772/0 (socket says 172.21.15.45:39772) 2024-06-06T04:30:49.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.211+0000 7f54e37fe700 1 -- 172.21.15.45:0/2591166539 learned_addr learned my addr 172.21.15.45:0/2591166539 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:49.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.211+0000 7f54e3fff700 1 --2- 172.21.15.45:0/2591166539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54e4104090 0x7f54e419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:49.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.211+0000 7f54e2ffd700 1 --2- 172.21.15.45:0/2591166539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54e41036f0 0x7f54e4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:49.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.212+0000 7f54e37fe700 1 -- 172.21.15.45:0/2591166539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54e41036f0 msgr2=0x7f54e4198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:49.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.212+0000 7f54e37fe700 1 --2- 172.21.15.45:0/2591166539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54e41036f0 0x7f54e4198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:49.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.212+0000 7f54e37fe700 1 -- 172.21.15.45:0/2591166539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54e4104090 msgr2=0x7f54e419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:49.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.212+0000 7f54e37fe700 1 --2- 172.21.15.45:0/2591166539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54e4104090 0x7f54e419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:49.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.212+0000 7f54e37fe700 1 -- 172.21.15.45:0/2591166539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e41a0370 con 0x7f54e41024f0 2024-06-06T04:30:49.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.212+0000 7f54e2ffd700 1 --2- 172.21.15.45:0/2591166539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54e41036f0 0x7f54e4198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:30:49.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.212+0000 7f54e37fe700 1 --2- 172.21.15.45:0/2591166539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54e41024f0 0x7f54e4198710 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f54d4005b10 tx=0x7f54d4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:49.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.212+0000 7f54e0ff9700 1 -- 172.21.15.45:0/2591166539 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d4027070 con 0x7f54e41024f0 2024-06-06T04:30:49.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.212+0000 7f54e0ff9700 1 -- 172.21.15.45:0/2591166539 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54d4004050 con 0x7f54e41024f0 2024-06-06T04:30:49.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.212+0000 7f54e9c31700 1 -- 172.21.15.45:0/2591166539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e41a0600 con 0x7f54e41024f0 2024-06-06T04:30:49.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.212+0000 7f54e0ff9700 1 -- 172.21.15.45:0/2591166539 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d4022c70 con 0x7f54e41024f0 2024-06-06T04:30:49.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.212+0000 7f54e9c31700 1 -- 172.21.15.45:0/2591166539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e41a0b60 con 0x7f54e41024f0 2024-06-06T04:30:49.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.213+0000 7f54e9c31700 1 -- 172.21.15.45:0/2591166539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e4069180 con 0x7f54e41024f0 2024-06-06T04:30:49.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.214+0000 7f54e0ff9700 1 -- 172.21.15.45:0/2591166539 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f54d402a070 con 0x7f54e41024f0 2024-06-06T04:30:49.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.215+0000 7f54e0ff9700 1 --2- 172.21.15.45:0/2591166539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f54cc0a2c20 0x7f54cc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:49.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.215+0000 7f54e2ffd700 1 --2- 172.21.15.45:0/2591166539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f54cc0a2c20 0x7f54cc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:49.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.215+0000 7f54e0ff9700 1 -- 172.21.15.45:0/2591166539 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f54d40d0540 con 0x7f54e41024f0 2024-06-06T04:30:49.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.216+0000 7f54e2ffd700 1 --2- 172.21.15.45:0/2591166539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f54cc0a2c20 0x7f54cc0a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f54d80099d0 tx=0x7f54d8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:49.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.220+0000 7f54e0ff9700 1 -- 172.21.15.45:0/2591166539 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f54d405e850 con 0x7f54e41024f0 2024-06-06T04:30:49.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.435+0000 7f54e9c31700 1 -- 172.21.15.45:0/2591166539 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f54e40628d0 con 0x7f54cc0a2c20 2024-06-06T04:30:49.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.436+0000 7f54e0ff9700 1 -- 172.21.15.45:0/2591166539 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f54e40628d0 con 0x7f54cc0a2c20 2024-06-06T04:30:49.437 INFO:teuthology.orchestra.run.smithi045.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-06T04:30:49.437 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120098480DGN 447G No 5m ago Has a FileSystem, LVM detected 2024-06-06T04:30:49.437 INFO:teuthology.orchestra.run.smithi045.stdout:smithi086 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UL400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-06-06T04:30:49.437 INFO:teuthology.orchestra.run.smithi045.stdout:smithi154 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233003B400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-06-06T04:30:49.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.439+0000 7f54e9c31700 1 -- 172.21.15.45:0/2591166539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f54cc0a2c20 msgr2=0x7f54cc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:49.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.439+0000 7f54e9c31700 1 --2- 172.21.15.45:0/2591166539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f54cc0a2c20 0x7f54cc0a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f54d80099d0 tx=0x7f54d8008040 comp rx=0 tx=0).stop 2024-06-06T04:30:49.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.439+0000 7f54e9c31700 1 -- 172.21.15.45:0/2591166539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54e41024f0 msgr2=0x7f54e4198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:49.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.439+0000 7f54e9c31700 1 --2- 172.21.15.45:0/2591166539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54e41024f0 0x7f54e4198710 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f54d4005b10 tx=0x7f54d4009580 comp rx=0 tx=0).stop 2024-06-06T04:30:49.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.439+0000 7f54e9c31700 1 -- 172.21.15.45:0/2591166539 shutdown_connections 2024-06-06T04:30:49.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.439+0000 7f54e9c31700 1 --2- 172.21.15.45:0/2591166539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f54cc0a2c20 0x7f54cc0a50e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:49.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.439+0000 7f54e9c31700 1 --2- 172.21.15.45:0/2591166539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54e4104090 0x7f54e419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:49.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.439+0000 7f54e9c31700 1 --2- 172.21.15.45:0/2591166539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54e41024f0 0x7f54e4198710 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:49.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.439+0000 7f54e9c31700 1 --2- 172.21.15.45:0/2591166539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54e41036f0 0x7f54e4198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:49.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.439+0000 7f54e9c31700 1 -- 172.21.15.45:0/2591166539 >> 172.21.15.45:0/2591166539 conn(0x7f54e40fda90 msgr2=0x7f54e4106ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:49.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.439+0000 7f54e9c31700 1 -- 172.21.15.45:0/2591166539 shutdown_connections 2024-06-06T04:30:49.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:49.439+0000 7f54e9c31700 1 -- 172.21.15.45:0/2591166539 wait complete. 2024-06-06T04:30:49.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:49 smithi086 ceph-mon[36053]: pgmap v412: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:30:49.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:49 smithi154 ceph-mon[36428]: pgmap v412: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:30:49.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:49 smithi045 ceph-mon[32299]: pgmap v412: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:30:50.131 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-06T04:30:50.140 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-06-06T04:30:50.140 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-06T04:30:50.396 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:50.782 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:50 smithi045 ceph-mon[32299]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:30:50.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:50 smithi086 ceph-mon[36053]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:30:50.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:50 smithi154 ceph-mon[36428]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:30:51.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:51 smithi086 ceph-mon[36053]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:51.877 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:51 smithi045 ceph-mon[32299]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:51.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:51 smithi154 ceph-mon[36428]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:52.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.134+0000 7f1c78425700 1 -- 172.21.15.45:0/3110985164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c70104090 msgr2=0x7f1c7010a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:52.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.134+0000 7f1c78425700 1 --2- 172.21.15.45:0/3110985164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c70104090 0x7f1c7010a940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800b0d0 tx=0x7f1c6800b3e0 comp rx=0 tx=0).stop 2024-06-06T04:30:52.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.134+0000 7f1c78425700 1 -- 172.21.15.45:0/3110985164 shutdown_connections 2024-06-06T04:30:52.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.134+0000 7f1c78425700 1 --2- 172.21.15.45:0/3110985164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c70104090 0x7f1c7010a940 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:52.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.134+0000 7f1c78425700 1 --2- 172.21.15.45:0/3110985164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c701024f0 0x7f1c70102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:52.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.134+0000 7f1c78425700 1 --2- 172.21.15.45:0/3110985164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c701036f0 0x7f1c70103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:52.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.134+0000 7f1c78425700 1 -- 172.21.15.45:0/3110985164 >> 172.21.15.45:0/3110985164 conn(0x7f1c700fda90 msgr2=0x7f1c700ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:52.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.134+0000 7f1c78425700 1 -- 172.21.15.45:0/3110985164 shutdown_connections 2024-06-06T04:30:52.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.134+0000 7f1c78425700 1 -- 172.21.15.45:0/3110985164 wait complete. 2024-06-06T04:30:52.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c78425700 1 Processor -- start 2024-06-06T04:30:52.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c78425700 1 -- start start 2024-06-06T04:30:52.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c78425700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c701024f0 0x7f1c70198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:52.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c78425700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c701036f0 0x7f1c70198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:52.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c78425700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70104090 0x7f1c7019fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:52.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c78425700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7010df70 con 0x7f1c701036f0 2024-06-06T04:30:52.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c78425700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7010ddf0 con 0x7f1c701024f0 2024-06-06T04:30:52.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c78425700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7010e0f0 con 0x7f1c70104090 2024-06-06T04:30:52.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c759c0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c701036f0 0x7f1c70198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:52.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c759c0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c701036f0 0x7f1c70198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39776/0 (socket says 172.21.15.45:39776) 2024-06-06T04:30:52.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c759c0700 1 -- 172.21.15.45:0/1357499718 learned_addr learned my addr 172.21.15.45:0/1357499718 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:52.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c761c1700 1 --2- 172.21.15.45:0/1357499718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c701024f0 0x7f1c70198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:52.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c769c2700 1 --2- 172.21.15.45:0/1357499718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70104090 0x7f1c7019fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:52.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c759c0700 1 -- 172.21.15.45:0/1357499718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70104090 msgr2=0x7f1c7019fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:52.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c759c0700 1 --2- 172.21.15.45:0/1357499718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70104090 0x7f1c7019fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:52.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c759c0700 1 -- 172.21.15.45:0/1357499718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c701024f0 msgr2=0x7f1c70198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:52.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c759c0700 1 --2- 172.21.15.45:0/1357499718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c701024f0 0x7f1c70198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:52.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.135+0000 7f1c759c0700 1 -- 172.21.15.45:0/1357499718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c701a01e0 con 0x7f1c701036f0 2024-06-06T04:30:52.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.136+0000 7f1c759c0700 1 --2- 172.21.15.45:0/1357499718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c701036f0 0x7f1c70198ac0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000d980 tx=0x7f1c6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:52.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.136+0000 7f1c677fe700 1 -- 172.21.15.45:0/1357499718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60016070 con 0x7f1c701036f0 2024-06-06T04:30:52.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.136+0000 7f1c677fe700 1 -- 172.21.15.45:0/1357499718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c60009d80 con 0x7f1c701036f0 2024-06-06T04:30:52.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.136+0000 7f1c78425700 1 -- 172.21.15.45:0/1357499718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c701a04d0 con 0x7f1c701036f0 2024-06-06T04:30:52.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.136+0000 7f1c677fe700 1 -- 172.21.15.45:0/1357499718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6000f3c0 con 0x7f1c701036f0 2024-06-06T04:30:52.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.136+0000 7f1c78425700 1 -- 172.21.15.45:0/1357499718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c701a0a10 con 0x7f1c701036f0 2024-06-06T04:30:52.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.137+0000 7f1c78425700 1 -- 172.21.15.45:0/1357499718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c70069180 con 0x7f1c701036f0 2024-06-06T04:30:52.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.138+0000 7f1c677fe700 1 -- 172.21.15.45:0/1357499718 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1c60019070 con 0x7f1c701036f0 2024-06-06T04:30:52.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.139+0000 7f1c677fe700 1 --2- 172.21.15.45:0/1357499718 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1c5c0a2c20 0x7f1c5c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:52.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.139+0000 7f1c761c1700 1 --2- 172.21.15.45:0/1357499718 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1c5c0a2c20 0x7f1c5c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:52.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.139+0000 7f1c677fe700 1 -- 172.21.15.45:0/1357499718 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f1c600ceeb0 con 0x7f1c701036f0 2024-06-06T04:30:52.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.139+0000 7f1c761c1700 1 --2- 172.21.15.45:0/1357499718 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1c5c0a2c20 0x7f1c5c0a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1c70103550 tx=0x7f1c6c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:52.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.143+0000 7f1c677fe700 1 -- 172.21.15.45:0/1357499718 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1c6009ba50 con 0x7f1c701036f0 2024-06-06T04:30:52.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.394+0000 7f1c78425700 1 -- 172.21.15.45:0/1357499718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1c700630a0 con 0x7f1c701036f0 2024-06-06T04:30:52.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.395+0000 7f1c677fe700 1 -- 172.21.15.45:0/1357499718 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3042 (secure 0 0 0) 0x7f1c6009b350 con 0x7f1c701036f0 2024-06-06T04:30:52.395 INFO:teuthology.orchestra.run.smithi045.stdout:e26 2024-06-06T04:30:52.396 INFO:teuthology.orchestra.run.smithi045.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-06T04:30:52.396 INFO:teuthology.orchestra.run.smithi045.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T04:30:52.396 INFO:teuthology.orchestra.run.smithi045.stdout:legacy client fscid: 1 2024-06-06T04:30:52.396 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:30:52.396 INFO:teuthology.orchestra.run.smithi045.stdout:Filesystem 'cephfs' (1) 2024-06-06T04:30:52.396 INFO:teuthology.orchestra.run.smithi045.stdout:fs_name cephfs 2024-06-06T04:30:52.396 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 26 2024-06-06T04:30:52.396 INFO:teuthology.orchestra.run.smithi045.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-06-06T04:30:52.396 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-06-06T04:27:24.462897+0000 2024-06-06T04:30:52.396 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2024-06-06T04:28:32.515629+0000 2024-06-06T04:30:52.396 INFO:teuthology.orchestra.run.smithi045.stdout:tableserver 0 2024-06-06T04:30:52.396 INFO:teuthology.orchestra.run.smithi045.stdout:root 0 2024-06-06T04:30:52.396 INFO:teuthology.orchestra.run.smithi045.stdout:session_timeout 300 2024-06-06T04:30:52.396 INFO:teuthology.orchestra.run.smithi045.stdout:session_autoclose 300 2024-06-06T04:30:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:max_file_size 1099511627776 2024-06-06T04:30:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:max_xattr_size 65536 2024-06-06T04:30:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:required_client_features {} 2024-06-06T04:30:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure 0 2024-06-06T04:30:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure_osd_epoch 0 2024-06-06T04:30:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T04:30:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:max_mds 3 2024-06-06T04:30:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:in 0,1,2 2024-06-06T04:30:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:up {0=24473,1=24463,2=24451} 2024-06-06T04:30:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:failed 2024-06-06T04:30:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:damaged 2024-06-06T04:30:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:stopped 2024-06-06T04:30:52.398 INFO:teuthology.orchestra.run.smithi045.stdout:data_pools [3,4] 2024-06-06T04:30:52.398 INFO:teuthology.orchestra.run.smithi045.stdout:metadata_pool 2 2024-06-06T04:30:52.398 INFO:teuthology.orchestra.run.smithi045.stdout:inline_data disabled 2024-06-06T04:30:52.398 INFO:teuthology.orchestra.run.smithi045.stdout:balancer 2024-06-06T04:30:52.398 INFO:teuthology.orchestra.run.smithi045.stdout:bal_rank_mask -1 2024-06-06T04:30:52.398 INFO:teuthology.orchestra.run.smithi045.stdout:standby_count_wanted 0 2024-06-06T04:30:52.398 INFO:teuthology.orchestra.run.smithi045.stdout:qdb_cluster leader: 24473 members: 24463,24473,24451 2024-06-06T04:30:52.398 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b{0:24473} state up:active seq 5 addr [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T04:30:52.398 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.h{0:24449} state up:standby-replay seq 1 addr [v2:172.21.15.86:6836/414874753,v1:172.21.15.86:6837/414874753] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T04:30:52.398 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.i{1:24463} state up:active seq 23 addr [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T04:30:52.398 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.f{1:24445} state up:standby-replay seq 1 addr [v2:172.21.15.154:6834/954391764,v1:172.21.15.154:6835/954391764] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T04:30:52.398 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c{2:24451} state up:active seq 25 addr [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T04:30:52.399 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.e{2:24419} state up:standby-replay seq 1 addr [v2:172.21.15.86:6834/4268654313,v1:172.21.15.86:6835/4268654313] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T04:30:52.399 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:30:52.399 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:30:52.399 INFO:teuthology.orchestra.run.smithi045.stdout:Standby daemons: 2024-06-06T04:30:52.399 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:30:52.399 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.j{-1:14529} state up:standby seq 1 addr [v2:172.21.15.45:6834/1876588301,v1:172.21.15.45:6835/1876588301] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T04:30:52.399 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d{-1:14541} state up:standby seq 1 addr [v2:172.21.15.45:6836/3532043360,v1:172.21.15.45:6837/3532043360] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T04:30:52.399 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.g{-1:14589} state up:standby seq 1 addr [v2:172.21.15.45:6838/2563890632,v1:172.21.15.45:6839/2563890632] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T04:30:52.399 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a{-1:14601} state up:standby seq 1 addr [v2:172.21.15.45:6840/611193504,v1:172.21.15.45:6841/611193504] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T04:30:52.399 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.k{-1:24398} state up:standby seq 1 addr [v2:172.21.15.86:6832/452357683,v1:172.21.15.86:6833/452357683] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T04:30:52.399 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.l{-1:24412} state up:standby seq 1 addr [v2:172.21.15.154:6832/2851773245,v1:172.21.15.154:6833/2851773245] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T04:30:52.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.397+0000 7f1c78425700 1 -- 172.21.15.45:0/1357499718 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1c5c0a2c20 msgr2=0x7f1c5c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:52.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.397+0000 7f1c78425700 1 --2- 172.21.15.45:0/1357499718 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1c5c0a2c20 0x7f1c5c0a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1c70103550 tx=0x7f1c6c00b410 comp rx=0 tx=0).stop 2024-06-06T04:30:52.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.397+0000 7f1c78425700 1 -- 172.21.15.45:0/1357499718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c701036f0 msgr2=0x7f1c70198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:52.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.397+0000 7f1c78425700 1 --2- 172.21.15.45:0/1357499718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c701036f0 0x7f1c70198ac0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000d980 tx=0x7f1c6000dc90 comp rx=0 tx=0).stop 2024-06-06T04:30:52.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.397+0000 7f1c78425700 1 -- 172.21.15.45:0/1357499718 shutdown_connections 2024-06-06T04:30:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.397+0000 7f1c78425700 1 --2- 172.21.15.45:0/1357499718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70104090 0x7f1c7019fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.397+0000 7f1c78425700 1 --2- 172.21.15.45:0/1357499718 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1c5c0a2c20 0x7f1c5c0a50e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.397+0000 7f1c78425700 1 --2- 172.21.15.45:0/1357499718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c701024f0 0x7f1c70198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.397+0000 7f1c78425700 1 --2- 172.21.15.45:0/1357499718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c701036f0 0x7f1c70198ac0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.397+0000 7f1c78425700 1 -- 172.21.15.45:0/1357499718 >> 172.21.15.45:0/1357499718 conn(0x7f1c700fda90 msgr2=0x7f1c701069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.397+0000 7f1c78425700 1 -- 172.21.15.45:0/1357499718 shutdown_connections 2024-06-06T04:30:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:52.397+0000 7f1c78425700 1 -- 172.21.15.45:0/1357499718 wait complete. 2024-06-06T04:30:52.401 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-06-06T04:30:52.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:52 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1357499718' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-06T04:30:52.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:52 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1357499718' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-06T04:30:52.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:52 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1357499718' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-06T04:30:53.164 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-06-06T04:30:53.421 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:53.849 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:53 smithi045 ceph-mon[32299]: pgmap v414: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:30:53.858 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:53 smithi086 ceph-mon[36053]: pgmap v414: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:30:53.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:53 smithi154 ceph-mon[36428]: pgmap v414: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:30:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.024+0000 7f32d36d8700 1 -- 172.21.15.45:0/1968176664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32cc100d70 msgr2=0x7f32cc101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.024+0000 7f32d36d8700 1 --2- 172.21.15.45:0/1968176664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32cc100d70 0x7f32cc101190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009990 tx=0x7f32c8009ca0 comp rx=0 tx=0).stop 2024-06-06T04:30:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.024+0000 7f32d36d8700 1 -- 172.21.15.45:0/1968176664 shutdown_connections 2024-06-06T04:30:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.024+0000 7f32d36d8700 1 --2- 172.21.15.45:0/1968176664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32cc102090 0x7f32cc102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.024+0000 7f32d36d8700 1 --2- 172.21.15.45:0/1968176664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32cc100d70 0x7f32cc101190 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.024+0000 7f32d36d8700 1 --2- 172.21.15.45:0/1968176664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32cc1016d0 0x7f32cc101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.024+0000 7f32d36d8700 1 -- 172.21.15.45:0/1968176664 >> 172.21.15.45:0/1968176664 conn(0x7f32cc0fc990 msgr2=0x7f32cc0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.024+0000 7f32d36d8700 1 -- 172.21.15.45:0/1968176664 shutdown_connections 2024-06-06T04:30:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.024+0000 7f32d36d8700 1 -- 172.21.15.45:0/1968176664 wait complete. 2024-06-06T04:30:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.025+0000 7f32d36d8700 1 Processor -- start 2024-06-06T04:30:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.025+0000 7f32d36d8700 1 -- start start 2024-06-06T04:30:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.025+0000 7f32d36d8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32cc100d70 0x7f32cc19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.025+0000 7f32d36d8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32cc1016d0 0x7f32cc19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.025+0000 7f32d36d8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32cc102090 0x7f32cc1a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.025+0000 7f32d36d8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc110140 con 0x7f32cc100d70 2024-06-06T04:30:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.025+0000 7f32d36d8700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc10ffc0 con 0x7f32cc1016d0 2024-06-06T04:30:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.025+0000 7f32d36d8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1102c0 con 0x7f32cc102090 2024-06-06T04:30:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.025+0000 7f32d1474700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32cc100d70 0x7f32cc19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.025+0000 7f32d1474700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32cc100d70 0x7f32cc19a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39792/0 (socket says 172.21.15.45:39792) 2024-06-06T04:30:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.025+0000 7f32d1474700 1 -- 172.21.15.45:0/3875850910 learned_addr learned my addr 172.21.15.45:0/3875850910 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.025+0000 7f32d1c75700 1 --2- 172.21.15.45:0/3875850910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32cc102090 0x7f32cc1a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.026+0000 7f32d0c73700 1 --2- 172.21.15.45:0/3875850910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32cc1016d0 0x7f32cc19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.026+0000 7f32d1474700 1 -- 172.21.15.45:0/3875850910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32cc102090 msgr2=0x7f32cc1a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.026+0000 7f32d1474700 1 --2- 172.21.15.45:0/3875850910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32cc102090 0x7f32cc1a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.026+0000 7f32d1474700 1 -- 172.21.15.45:0/3875850910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32cc1016d0 msgr2=0x7f32cc19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.026+0000 7f32d1474700 1 --2- 172.21.15.45:0/3875850910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32cc1016d0 0x7f32cc19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.026+0000 7f32d1474700 1 -- 172.21.15.45:0/3875850910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc1a2480 con 0x7f32cc100d70 2024-06-06T04:30:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.026+0000 7f32d1474700 1 --2- 172.21.15.45:0/3875850910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32cc100d70 0x7f32cc19a820 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f32c80098e0 tx=0x7f32c80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.026+0000 7f32c27fc700 1 -- 172.21.15.45:0/3875850910 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8027070 con 0x7f32cc100d70 2024-06-06T04:30:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.026+0000 7f32c27fc700 1 -- 172.21.15.45:0/3875850910 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32c8004010 con 0x7f32cc100d70 2024-06-06T04:30:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.026+0000 7f32d36d8700 1 -- 172.21.15.45:0/3875850910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32cc1a2710 con 0x7f32cc100d70 2024-06-06T04:30:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.026+0000 7f32c27fc700 1 -- 172.21.15.45:0/3875850910 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8022da0 con 0x7f32cc100d70 2024-06-06T04:30:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.026+0000 7f32d36d8700 1 -- 172.21.15.45:0/3875850910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32cc1a2b60 con 0x7f32cc100d70 2024-06-06T04:30:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.027+0000 7f32d36d8700 1 -- 172.21.15.45:0/3875850910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32cc069180 con 0x7f32cc100d70 2024-06-06T04:30:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.028+0000 7f32c27fc700 1 -- 172.21.15.45:0/3875850910 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f32c802a070 con 0x7f32cc100d70 2024-06-06T04:30:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.029+0000 7f32c27fc700 1 --2- 172.21.15.45:0/3875850910 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f32b80a2c20 0x7f32b80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.029+0000 7f32d0c73700 1 --2- 172.21.15.45:0/3875850910 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f32b80a2c20 0x7f32b80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.029+0000 7f32c27fc700 1 -- 172.21.15.45:0/3875850910 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f32c80d0110 con 0x7f32cc100d70 2024-06-06T04:30:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.029+0000 7f32d0c73700 1 --2- 172.21.15.45:0/3875850910 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f32b80a2c20 0x7f32b80a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f32bc005950 tx=0x7f32bc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:55.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.033+0000 7f32c27fc700 1 -- 172.21.15.45:0/3875850910 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f32c8013070 con 0x7f32cc100d70 2024-06-06T04:30:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.241+0000 7f32d36d8700 1 -- 172.21.15.45:0/3875850910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f32cc0630a0 con 0x7f32cc100d70 2024-06-06T04:30:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.242+0000 7f32c27fc700 1 -- 172.21.15.45:0/3875850910 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v100) ==== 56+0+5675 (secure 0 0 0) 0x7f32c809cce0 con 0x7f32cc100d70 2024-06-06T04:30:55.243 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 100 2024-06-06T04:30:55.243 INFO:teuthology.orchestra.run.smithi045.stdout:fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:30:55.243 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-06-06T04:19:02.626916+0000 2024-06-06T04:30:55.243 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2024-06-06T04:29:05.901370+0000 2024-06-06T04:30:55.243 INFO:teuthology.orchestra.run.smithi045.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-06-06T04:30:55.243 INFO:teuthology.orchestra.run.smithi045.stdout:crush_version 27 2024-06-06T04:30:55.243 INFO:teuthology.orchestra.run.smithi045.stdout:full_ratio 0.95 2024-06-06T04:30:55.243 INFO:teuthology.orchestra.run.smithi045.stdout:backfillfull_ratio 0.9 2024-06-06T04:30:55.243 INFO:teuthology.orchestra.run.smithi045.stdout:nearfull_ratio 0.85 2024-06-06T04:30:55.243 INFO:teuthology.orchestra.run.smithi045.stdout:require_min_compat_client luminous 2024-06-06T04:30:55.243 INFO:teuthology.orchestra.run.smithi045.stdout:min_compat_client luminous 2024-06-06T04:30:55.244 INFO:teuthology.orchestra.run.smithi045.stdout:require_osd_release squid 2024-06-06T04:30:55.244 INFO:teuthology.orchestra.run.smithi045.stdout:stretch_mode_enabled false 2024-06-06T04:30:55.244 INFO:teuthology.orchestra.run.smithi045.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-06-06T04:30:55.244 INFO:teuthology.orchestra.run.smithi045.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-06-06T04:30:55.244 INFO:teuthology.orchestra.run.smithi045.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-06-06T04:30:55.244 INFO:teuthology.orchestra.run.smithi045.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 96 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-06-06T04:30:55.244 INFO:teuthology.orchestra.run.smithi045.stdout:max_osd 12 2024-06-06T04:30:55.244 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 up in weight 1 up_from 8 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6802/2111759250,v1:172.21.15.45:6803/2111759250] [v2:172.21.15.45:6804/2111759250,v1:172.21.15.45:6805/2111759250] exists,up 7d3cee4a-5025-45b3-b217-082af37e79db 2024-06-06T04:30:55.244 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 up in weight 1 up_from 13 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6800/3732225287,v1:172.21.15.86:6801/3732225287] [v2:172.21.15.86:6802/3732225287,v1:172.21.15.86:6803/3732225287] exists,up 75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad 2024-06-06T04:30:55.244 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 up in weight 1 up_from 18 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6800/2898783039,v1:172.21.15.154:6801/2898783039] [v2:172.21.15.154:6802/2898783039,v1:172.21.15.154:6803/2898783039] exists,up bb20565f-f9ae-47e3-9604-c69f737cff4c 2024-06-06T04:30:55.244 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 up in weight 1 up_from 26 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6810/3436753135,v1:172.21.15.45:6811/3436753135] [v2:172.21.15.45:6812/3436753135,v1:172.21.15.45:6813/3436753135] exists,up e1160799-7811-4326-9792-7e6c7d189c7b 2024-06-06T04:30:55.244 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 up in weight 1 up_from 34 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6808/3030235796,v1:172.21.15.86:6809/3030235796] [v2:172.21.15.86:6810/3030235796,v1:172.21.15.86:6811/3030235796] exists,up 6b389bc8-e19a-47f3-83ff-7b782d146d0c 2024-06-06T04:30:55.244 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 up in weight 1 up_from 39 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6808/2222268646,v1:172.21.15.154:6809/2222268646] [v2:172.21.15.154:6810/2222268646,v1:172.21.15.154:6811/2222268646] exists,up 069de7b2-89bf-4150-bed7-7b8832573dd1 2024-06-06T04:30:55.245 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 up in weight 1 up_from 47 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6818/384734939,v1:172.21.15.45:6819/384734939] [v2:172.21.15.45:6820/384734939,v1:172.21.15.45:6821/384734939] exists,up 4c70b627-eafa-4e27-ba2a-4d8e1e335962 2024-06-06T04:30:55.245 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 up in weight 1 up_from 52 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6816/1119791005,v1:172.21.15.86:6817/1119791005] [v2:172.21.15.86:6818/1119791005,v1:172.21.15.86:6819/1119791005] exists,up 3b09b875-be52-4be0-b116-7fd9b0c26705 2024-06-06T04:30:55.245 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 up in weight 1 up_from 60 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6816/3054812496,v1:172.21.15.154:6817/3054812496] [v2:172.21.15.154:6818/3054812496,v1:172.21.15.154:6819/3054812496] exists,up cdf1dd1b-5392-4836-8e65-4c7c5b688d4a 2024-06-06T04:30:55.245 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 up in weight 1 up_from 68 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6826/1360655510,v1:172.21.15.45:6827/1360655510] [v2:172.21.15.45:6828/1360655510,v1:172.21.15.45:6829/1360655510] exists,up acede92d-2cc1-4967-a357-7e5e201e99d7 2024-06-06T04:30:55.245 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 up in weight 1 up_from 73 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6824/1701691877,v1:172.21.15.86:6825/1701691877] [v2:172.21.15.86:6826/1701691877,v1:172.21.15.86:6827/1701691877] exists,up a2d0c488-1892-4da4-adbf-952c22df0ce6 2024-06-06T04:30:55.245 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 up in weight 1 up_from 81 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6824/2065648007,v1:172.21.15.154:6825/2065648007] [v2:172.21.15.154:6826/2065648007,v1:172.21.15.154:6827/2065648007] exists,up 6b349c4c-b672-480b-ac6a-8594dab0eaca 2024-06-06T04:30:55.245 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.e [2,4] 2024-06-06T04:30:55.245 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.1c [8,11] 2024-06-06T04:30:55.245 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.1d [8,11,2,1] 2024-06-06T04:30:55.245 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.0 [5,9] 2024-06-06T04:30:55.245 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.6 [4,7] 2024-06-06T04:30:55.245 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.e [4,8] 2024-06-06T04:30:55.245 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.10 [5,9] 2024-06-06T04:30:55.245 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.12 [3,9] 2024-06-06T04:30:55.246 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.14 [4,6] 2024-06-06T04:30:55.246 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 4.e [5,8] 2024-06-06T04:30:55.246 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 4.10 [9,10] 2024-06-06T04:30:55.246 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 4.16 [9,8] 2024-06-06T04:30:55.246 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 4.17 [9,8] 2024-06-06T04:30:55.246 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/36732849 expires 2024-06-07T04:20:04.267269+0000 2024-06-06T04:30:55.246 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/1500318775 expires 2024-06-07T04:20:04.267269+0000 2024-06-06T04:30:55.246 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/3465796147 expires 2024-06-07T04:20:04.267269+0000 2024-06-06T04:30:55.246 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/3465796147 expires 2024-06-07T04:20:04.267269+0000 2024-06-06T04:30:55.246 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/2041380391 expires 2024-06-07T04:19:37.026199+0000 2024-06-06T04:30:55.246 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/2504003341 expires 2024-06-07T04:19:37.026199+0000 2024-06-06T04:30:55.246 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/3798552599 expires 2024-06-07T04:20:04.267269+0000 2024-06-06T04:30:55.246 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/498952822 expires 2024-06-07T04:19:37.026199+0000 2024-06-06T04:30:55.246 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/1562511056 expires 2024-06-07T04:19:37.026199+0000 2024-06-06T04:30:55.247 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/1562511056 expires 2024-06-07T04:19:37.026199+0000 2024-06-06T04:30:55.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.244+0000 7f32d36d8700 1 -- 172.21.15.45:0/3875850910 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f32b80a2c20 msgr2=0x7f32b80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:55.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.244+0000 7f32d36d8700 1 --2- 172.21.15.45:0/3875850910 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f32b80a2c20 0x7f32b80a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f32bc005950 tx=0x7f32bc0058e0 comp rx=0 tx=0).stop 2024-06-06T04:30:55.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.244+0000 7f32d36d8700 1 -- 172.21.15.45:0/3875850910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32cc100d70 msgr2=0x7f32cc19a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:55.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.244+0000 7f32d36d8700 1 --2- 172.21.15.45:0/3875850910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32cc100d70 0x7f32cc19a820 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f32c80098e0 tx=0x7f32c80047b0 comp rx=0 tx=0).stop 2024-06-06T04:30:55.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.244+0000 7f32d36d8700 1 -- 172.21.15.45:0/3875850910 shutdown_connections 2024-06-06T04:30:55.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.244+0000 7f32d36d8700 1 --2- 172.21.15.45:0/3875850910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32cc102090 0x7f32cc1a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:55.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.244+0000 7f32d36d8700 1 --2- 172.21.15.45:0/3875850910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32cc100d70 0x7f32cc19a820 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:55.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.244+0000 7f32d36d8700 1 --2- 172.21.15.45:0/3875850910 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f32b80a2c20 0x7f32b80a50e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:55.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.244+0000 7f32d36d8700 1 --2- 172.21.15.45:0/3875850910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32cc1016d0 0x7f32cc19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:55.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.244+0000 7f32d36d8700 1 -- 172.21.15.45:0/3875850910 >> 172.21.15.45:0/3875850910 conn(0x7f32cc0fc990 msgr2=0x7f32cc0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:55.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.245+0000 7f32d36d8700 1 -- 172.21.15.45:0/3875850910 shutdown_connections 2024-06-06T04:30:55.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:55.245+0000 7f32d36d8700 1 -- 172.21.15.45:0/3875850910 wait complete. 2024-06-06T04:30:55.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:55 smithi086 ceph-mon[36053]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:55.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:55 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3875850910' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-06T04:30:55.920 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:55 smithi045 ceph-mon[32299]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:55.920 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:55 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3875850910' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-06T04:30:55.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:55 smithi154 ceph-mon[36428]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:30:55.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:55 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3875850910' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-06T04:30:55.949 INFO:teuthology.run_tasks:Running task fs.ready... 2024-06-06T04:30:55.959 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:30:56.216 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:57.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:57 smithi086 ceph-mon[36053]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:30:57.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:57 smithi045 ceph-mon[32299]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:30:57.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.900+0000 7fe37374e700 1 -- 172.21.15.45:0/2646745896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe36c107400 msgr2=0x7fe36c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:57.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.900+0000 7fe37374e700 1 --2- 172.21.15.45:0/2646745896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe36c107400 0x7fe36c1077e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe368009990 tx=0x7fe368009ca0 comp rx=0 tx=0).stop 2024-06-06T04:30:57.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.900+0000 7fe37374e700 1 -- 172.21.15.45:0/2646745896 shutdown_connections 2024-06-06T04:30:57.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.900+0000 7fe37374e700 1 --2- 172.21.15.45:0/2646745896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe36c101d50 0x7fe36c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:57.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.900+0000 7fe37374e700 1 --2- 172.21.15.45:0/2646745896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe36c107400 0x7fe36c1077e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:57.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.900+0000 7fe37374e700 1 --2- 172.21.15.45:0/2646745896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe36c101390 0x7fe36c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:57.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.900+0000 7fe37374e700 1 -- 172.21.15.45:0/2646745896 >> 172.21.15.45:0/2646745896 conn(0x7fe36c078340 msgr2=0x7fe36c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:57.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.900+0000 7fe37374e700 1 -- 172.21.15.45:0/2646745896 shutdown_connections 2024-06-06T04:30:57.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.900+0000 7fe37374e700 1 -- 172.21.15.45:0/2646745896 wait complete. 2024-06-06T04:30:57.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.901+0000 7fe37374e700 1 Processor -- start 2024-06-06T04:30:57.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.901+0000 7fe37374e700 1 -- start start 2024-06-06T04:30:57.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.901+0000 7fe37374e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe36c101390 0x7fe36c19c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:57.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.901+0000 7fe37374e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe36c101d50 0x7fe36c19cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:57.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.901+0000 7fe37374e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe36c107400 0x7fe36c196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:57.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.901+0000 7fe37374e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c110480 con 0x7fe36c101390 2024-06-06T04:30:57.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.901+0000 7fe37374e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c110300 con 0x7fe36c107400 2024-06-06T04:30:57.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.901+0000 7fe37374e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c110600 con 0x7fe36c101d50 2024-06-06T04:30:57.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.901+0000 7fe3714ea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe36c101390 0x7fe36c19c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:57.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.901+0000 7fe3714ea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe36c101390 0x7fe36c19c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53198/0 (socket says 172.21.15.45:53198) 2024-06-06T04:30:57.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.901+0000 7fe3714ea700 1 -- 172.21.15.45:0/2756245736 learned_addr learned my addr 172.21.15.45:0/2756245736 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:30:57.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.901+0000 7fe370ce9700 1 --2- 172.21.15.45:0/2756245736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe36c101d50 0x7fe36c19cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:57.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.901+0000 7fe371ceb700 1 --2- 172.21.15.45:0/2756245736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe36c107400 0x7fe36c196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:57.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.902+0000 7fe3714ea700 1 -- 172.21.15.45:0/2756245736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe36c101d50 msgr2=0x7fe36c19cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:57.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.902+0000 7fe3714ea700 1 --2- 172.21.15.45:0/2756245736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe36c101d50 0x7fe36c19cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:57.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.902+0000 7fe3714ea700 1 -- 172.21.15.45:0/2756245736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe36c107400 msgr2=0x7fe36c196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:57.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.902+0000 7fe3714ea700 1 --2- 172.21.15.45:0/2756245736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe36c107400 0x7fe36c196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:57.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.902+0000 7fe3714ea700 1 -- 172.21.15.45:0/2756245736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36c196f60 con 0x7fe36c101390 2024-06-06T04:30:57.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.902+0000 7fe371ceb700 1 --2- 172.21.15.45:0/2756245736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe36c107400 0x7fe36c196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:30:57.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.902+0000 7fe3714ea700 1 --2- 172.21.15.45:0/2756245736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe36c101390 0x7fe36c19c600 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe368000c00 tx=0x7fe3680057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:57.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.902+0000 7fe3627fc700 1 -- 172.21.15.45:0/2756245736 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368027070 con 0x7fe36c101390 2024-06-06T04:30:57.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.902+0000 7fe3627fc700 1 -- 172.21.15.45:0/2756245736 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe368022470 con 0x7fe36c101390 2024-06-06T04:30:57.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.902+0000 7fe37374e700 1 -- 172.21.15.45:0/2756245736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36c1971f0 con 0x7fe36c101390 2024-06-06T04:30:57.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.902+0000 7fe3627fc700 1 -- 172.21.15.45:0/2756245736 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368017650 con 0x7fe36c101390 2024-06-06T04:30:57.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.902+0000 7fe37374e700 1 -- 172.21.15.45:0/2756245736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe36c1a3460 con 0x7fe36c101390 2024-06-06T04:30:57.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.903+0000 7fe37374e700 1 -- 172.21.15.45:0/2756245736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe36c102ab0 con 0x7fe36c101390 2024-06-06T04:30:57.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.904+0000 7fe3627fc700 1 -- 172.21.15.45:0/2756245736 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe36802a070 con 0x7fe36c101390 2024-06-06T04:30:57.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.905+0000 7fe3627fc700 1 --2- 172.21.15.45:0/2756245736 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe3580a2c20 0x7fe3580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:30:57.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.905+0000 7fe370ce9700 1 --2- 172.21.15.45:0/2756245736 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe3580a2c20 0x7fe3580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:30:57.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.905+0000 7fe3627fc700 1 -- 172.21.15.45:0/2756245736 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fe3680d0330 con 0x7fe36c101390 2024-06-06T04:30:57.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.905+0000 7fe370ce9700 1 --2- 172.21.15.45:0/2756245736 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe3580a2c20 0x7fe3580a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe36c078a50 tx=0x7fe35c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:30:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:57.909+0000 7fe3627fc700 1 -- 172.21.15.45:0/2756245736 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe36805e850 con 0x7fe36c101390 2024-06-06T04:30:57.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:57 smithi154 ceph-mon[36428]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:30:58.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:58.168+0000 7fe37374e700 1 -- 172.21.15.45:0/2756245736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe36c109850 con 0x7fe36c101390 2024-06-06T04:30:58.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:58.169+0000 7fe3627fc700 1 -- 172.21.15.45:0/2756245736 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11858 (secure 0 0 0) 0x7fe368013070 con 0x7fe36c101390 2024-06-06T04:30:58.169 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:30:58.170 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T04:27:24.462897+0000","modified":"2024-06-06T04:28:32.515629+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24463,24473,24451]},"id":1}]} 2024-06-06T04:30:58.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:58.171+0000 7fe37374e700 1 -- 172.21.15.45:0/2756245736 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe3580a2c20 msgr2=0x7fe3580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:58.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:58.171+0000 7fe37374e700 1 --2- 172.21.15.45:0/2756245736 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe3580a2c20 0x7fe3580a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe36c078a50 tx=0x7fe35c009260 comp rx=0 tx=0).stop 2024-06-06T04:30:58.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:58.171+0000 7fe37374e700 1 -- 172.21.15.45:0/2756245736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe36c101390 msgr2=0x7fe36c19c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:30:58.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:58.171+0000 7fe37374e700 1 --2- 172.21.15.45:0/2756245736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe36c101390 0x7fe36c19c600 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe368000c00 tx=0x7fe3680057a0 comp rx=0 tx=0).stop 2024-06-06T04:30:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:58.171+0000 7fe37374e700 1 -- 172.21.15.45:0/2756245736 shutdown_connections 2024-06-06T04:30:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:58.171+0000 7fe37374e700 1 --2- 172.21.15.45:0/2756245736 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fe3580a2c20 0x7fe3580a50e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:58.171+0000 7fe37374e700 1 --2- 172.21.15.45:0/2756245736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe36c107400 0x7fe36c196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:58.171+0000 7fe37374e700 1 --2- 172.21.15.45:0/2756245736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe36c101390 0x7fe36c19c600 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:58.171+0000 7fe37374e700 1 --2- 172.21.15.45:0/2756245736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe36c101d50 0x7fe36c19cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:30:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:58.171+0000 7fe37374e700 1 -- 172.21.15.45:0/2756245736 >> 172.21.15.45:0/2756245736 conn(0x7fe36c078340 msgr2=0x7fe36c100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:30:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:58.171+0000 7fe37374e700 1 -- 172.21.15.45:0/2756245736 shutdown_connections 2024-06-06T04:30:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:30:58.171+0000 7fe37374e700 1 -- 172.21.15.45:0/2756245736 wait complete. 2024-06-06T04:30:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-06-06T04:30:58.837 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:58 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2756245736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:30:58.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:58 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2756245736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:30:58.865 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:30:58.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:58 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2756245736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:30:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:30:59.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:30:59 smithi154 ceph-mon[36428]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:30:59.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:30:59 smithi045 ceph-mon[32299]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:31:00.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:30:59 smithi086 ceph-mon[36053]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:31:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.841+0000 7f8cf84a8700 1 -- 172.21.15.45:0/3371916657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf006b580 msgr2=0x7f8cf010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:00.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.841+0000 7f8cf84a8700 1 --2- 172.21.15.45:0/3371916657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf006b580 0x7f8cf010f320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800b0a0 tx=0x7f8ce800b3b0 comp rx=0 tx=0).stop 2024-06-06T04:31:00.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.841+0000 7f8cf84a8700 1 -- 172.21.15.45:0/3371916657 shutdown_connections 2024-06-06T04:31:00.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.841+0000 7f8cf84a8700 1 --2- 172.21.15.45:0/3371916657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf006b580 0x7f8cf010f320 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:00.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.841+0000 7f8cf84a8700 1 --2- 172.21.15.45:0/3371916657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8cf0108500 0x7f8cf01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:00.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.841+0000 7f8cf84a8700 1 --2- 172.21.15.45:0/3371916657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cf006abc0 0x7f8cf006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:00.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.841+0000 7f8cf84a8700 1 -- 172.21.15.45:0/3371916657 >> 172.21.15.45:0/3371916657 conn(0x7f8cf0078340 msgr2=0x7f8cf0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:00.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.841+0000 7f8cf84a8700 1 -- 172.21.15.45:0/3371916657 shutdown_connections 2024-06-06T04:31:00.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.841+0000 7f8cf84a8700 1 -- 172.21.15.45:0/3371916657 wait complete. 2024-06-06T04:31:00.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.842+0000 7f8cf84a8700 1 Processor -- start 2024-06-06T04:31:00.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.842+0000 7f8cf84a8700 1 -- start start 2024-06-06T04:31:00.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.842+0000 7f8cf84a8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cf006abc0 0x7f8cf0101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:00.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.842+0000 7f8cf84a8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf006b580 0x7f8cf01019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:00.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.842+0000 7f8cf84a8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8cf0108500 0x7f8cf0101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:00.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.842+0000 7f8cf84a8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0112630 con 0x7f8cf006b580 2024-06-06T04:31:00.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.842+0000 7f8cf84a8700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01124b0 con 0x7f8cf006abc0 2024-06-06T04:31:00.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.842+0000 7f8cf84a8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01127b0 con 0x7f8cf0108500 2024-06-06T04:31:00.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.843+0000 7f8cf5a43700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf006b580 0x7f8cf01019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:00.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.843+0000 7f8cf5a43700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf006b580 0x7f8cf01019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53214/0 (socket says 172.21.15.45:53214) 2024-06-06T04:31:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.843+0000 7f8cf5a43700 1 -- 172.21.15.45:0/4209111926 learned_addr learned my addr 172.21.15.45:0/4209111926 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.843+0000 7f8cf6244700 1 --2- 172.21.15.45:0/4209111926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cf006abc0 0x7f8cf0101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.843+0000 7f8cf6a45700 1 --2- 172.21.15.45:0/4209111926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8cf0108500 0x7f8cf0101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.843+0000 7f8cf5a43700 1 -- 172.21.15.45:0/4209111926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8cf0108500 msgr2=0x7f8cf0101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.843+0000 7f8cf5a43700 1 --2- 172.21.15.45:0/4209111926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8cf0108500 0x7f8cf0101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.843+0000 7f8cf5a43700 1 -- 172.21.15.45:0/4209111926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cf006abc0 msgr2=0x7f8cf0101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.843+0000 7f8cf5a43700 1 --2- 172.21.15.45:0/4209111926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cf006abc0 0x7f8cf0101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.843+0000 7f8cf5a43700 1 -- 172.21.15.45:0/4209111926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf01a6fb0 con 0x7f8cf006b580 2024-06-06T04:31:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.843+0000 7f8cf5a43700 1 --2- 172.21.15.45:0/4209111926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf006b580 0x7f8cf01019d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000c9b0 tx=0x7f8ce000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.843+0000 7f8ce77fe700 1 -- 172.21.15.45:0/4209111926 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0007b00 con 0x7f8cf006b580 2024-06-06T04:31:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.843+0000 7f8cf84a8700 1 -- 172.21.15.45:0/4209111926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf01a72a0 con 0x7f8cf006b580 2024-06-06T04:31:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.843+0000 7f8ce77fe700 1 -- 172.21.15.45:0/4209111926 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ce000ce50 con 0x7f8cf006b580 2024-06-06T04:31:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.843+0000 7f8ce77fe700 1 -- 172.21.15.45:0/4209111926 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0018660 con 0x7f8cf006b580 2024-06-06T04:31:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.843+0000 7f8cf84a8700 1 -- 172.21.15.45:0/4209111926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf01a7830 con 0x7f8cf006b580 2024-06-06T04:31:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.845+0000 7f8cf84a8700 1 -- 172.21.15.45:0/4209111926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf010a9e0 con 0x7f8cf006b580 2024-06-06T04:31:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.846+0000 7f8ce77fe700 1 -- 172.21.15.45:0/4209111926 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8ce000f450 con 0x7f8cf006b580 2024-06-06T04:31:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.846+0000 7f8ce77fe700 1 --2- 172.21.15.45:0/4209111926 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8cdc0a2c20 0x7f8cdc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.846+0000 7f8cf6244700 1 --2- 172.21.15.45:0/4209111926 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8cdc0a2c20 0x7f8cdc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.846+0000 7f8ce77fe700 1 -- 172.21.15.45:0/4209111926 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f8ce00cf330 con 0x7f8cf006b580 2024-06-06T04:31:00.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.847+0000 7f8cf6244700 1 --2- 172.21.15.45:0/4209111926 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8cdc0a2c20 0x7f8cdc0a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0045b0 tx=0x7f8cec00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:00.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:00.851+0000 7f8ce77fe700 1 -- 172.21.15.45:0/4209111926 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8ce0014030 con 0x7f8cf006b580 2024-06-06T04:31:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:01.100+0000 7f8cf84a8700 1 -- 172.21.15.45:0/4209111926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cf01029f0 con 0x7f8cf006b580 2024-06-06T04:31:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:01.101+0000 7f8ce77fe700 1 -- 172.21.15.45:0/4209111926 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11858 (secure 0 0 0) 0x7f8cf01029f0 con 0x7f8cf006b580 2024-06-06T04:31:01.102 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:31:01.102 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T04:27:24.462897+0000","modified":"2024-06-06T04:28:32.515629+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24463,24473,24451]},"id":1}]} 2024-06-06T04:31:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:01.103+0000 7f8cf84a8700 1 -- 172.21.15.45:0/4209111926 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8cdc0a2c20 msgr2=0x7f8cdc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:01.103+0000 7f8cf84a8700 1 --2- 172.21.15.45:0/4209111926 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8cdc0a2c20 0x7f8cdc0a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0045b0 tx=0x7f8cec00b3c0 comp rx=0 tx=0).stop 2024-06-06T04:31:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:01.103+0000 7f8cf84a8700 1 -- 172.21.15.45:0/4209111926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf006b580 msgr2=0x7f8cf01019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:01.103+0000 7f8cf84a8700 1 --2- 172.21.15.45:0/4209111926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf006b580 0x7f8cf01019d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000c9b0 tx=0x7f8ce000ccc0 comp rx=0 tx=0).stop 2024-06-06T04:31:01.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:01.103+0000 7f8cf84a8700 1 -- 172.21.15.45:0/4209111926 shutdown_connections 2024-06-06T04:31:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:01.103+0000 7f8cf84a8700 1 --2- 172.21.15.45:0/4209111926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8cf0108500 0x7f8cf0101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:01.103+0000 7f8cf84a8700 1 --2- 172.21.15.45:0/4209111926 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8cdc0a2c20 0x7f8cdc0a50e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:01.103+0000 7f8cf84a8700 1 --2- 172.21.15.45:0/4209111926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cf006abc0 0x7f8cf0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:01.103+0000 7f8cf84a8700 1 --2- 172.21.15.45:0/4209111926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf006b580 0x7f8cf01019d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:01.103+0000 7f8cf84a8700 1 -- 172.21.15.45:0/4209111926 >> 172.21.15.45:0/4209111926 conn(0x7f8cf0078340 msgr2=0x7f8cf0100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:01.104+0000 7f8cf84a8700 1 -- 172.21.15.45:0/4209111926 shutdown_connections 2024-06-06T04:31:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:01.104+0000 7f8cf84a8700 1 -- 172.21.15.45:0/4209111926 wait complete. 2024-06-06T04:31:01.105 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-06-06T04:31:01.794 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T04:31:01.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:01 smithi154 ceph-mon[36428]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:31:01.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:01 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/4209111926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:31:01.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:01 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:31:01.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:01 smithi045 ceph-mon[32299]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:31:01.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:01 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/4209111926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:31:01.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:01 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:31:02.052 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:31:02.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:01 smithi086 ceph-mon[36053]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:31:02.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:01 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/4209111926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:31:02.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:01 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:31:03.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:02 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:02 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:02 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:03.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.755+0000 7f5e597d4700 1 -- 172.21.15.45:0/1163547753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e54101d30 msgr2=0x7f5e5410e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:03.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.755+0000 7f5e597d4700 1 --2- 172.21.15.45:0/1163547753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e54101d30 0x7f5e5410e1d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00b0a0 tx=0x7f5e4c00b3b0 comp rx=0 tx=0).stop 2024-06-06T04:31:03.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.755+0000 7f5e597d4700 1 -- 172.21.15.45:0/1163547753 shutdown_connections 2024-06-06T04:31:03.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.755+0000 7f5e597d4700 1 --2- 172.21.15.45:0/1163547753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e54101d30 0x7f5e5410e1d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:03.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.755+0000 7f5e597d4700 1 --2- 172.21.15.45:0/1163547753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e54107420 0x7f5e54107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:03.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.755+0000 7f5e597d4700 1 --2- 172.21.15.45:0/1163547753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e54101370 0x7f5e541017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:03.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.755+0000 7f5e597d4700 1 -- 172.21.15.45:0/1163547753 >> 172.21.15.45:0/1163547753 conn(0x7f5e54078340 msgr2=0x7f5e54078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:03.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.755+0000 7f5e597d4700 1 -- 172.21.15.45:0/1163547753 shutdown_connections 2024-06-06T04:31:03.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.755+0000 7f5e597d4700 1 -- 172.21.15.45:0/1163547753 wait complete. 2024-06-06T04:31:03.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.756+0000 7f5e597d4700 1 Processor -- start 2024-06-06T04:31:03.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.756+0000 7f5e597d4700 1 -- start start 2024-06-06T04:31:03.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e597d4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e54101370 0x7f5e54199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:03.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e597d4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e54101d30 0x7f5e54199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:03.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e597d4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e54107420 0x7f5e541a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:03.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e597d4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5e54111530 con 0x7f5e54107420 2024-06-06T04:31:03.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e597d4700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541113b0 con 0x7f5e54101d30 2024-06-06T04:31:03.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e597d4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541116b0 con 0x7f5e54101370 2024-06-06T04:31:03.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e54107420 0x7f5e541a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:03.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e54107420 0x7f5e541a48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53240/0 (socket says 172.21.15.45:53240) 2024-06-06T04:31:03.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e537fe700 1 -- 172.21.15.45:0/3137027162 learned_addr learned my addr 172.21.15.45:0/3137027162 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:03.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e4a7fc700 1 --2- 172.21.15.45:0/3137027162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e54101d30 0x7f5e54199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:03.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e52ffd700 1 --2- 172.21.15.45:0/3137027162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e54101370 0x7f5e54199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:03.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e537fe700 1 -- 172.21.15.45:0/3137027162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e54101370 msgr2=0x7f5e54199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:03.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e537fe700 1 --2- 172.21.15.45:0/3137027162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e54101370 0x7f5e54199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:03.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e537fe700 1 -- 172.21.15.45:0/3137027162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e54101d30 msgr2=0x7f5e54199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:03.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e537fe700 1 --2- 172.21.15.45:0/3137027162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e54101d30 0x7f5e54199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:03.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e537fe700 1 -- 172.21.15.45:0/3137027162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e541a4df0 con 0x7f5e54107420 2024-06-06T04:31:03.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e537fe700 1 --2- 172.21.15.45:0/3137027162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e54107420 0x7f5e541a48b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00b070 tx=0x7f5e4c00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:03.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.757+0000 7f5e50ff9700 1 -- 172.21.15.45:0/3137027162 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c021070 con 0x7f5e54107420 2024-06-06T04:31:03.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.758+0000 7f5e50ff9700 1 -- 172.21.15.45:0/3137027162 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4c007c80 con 0x7f5e54107420 2024-06-06T04:31:03.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.758+0000 7f5e50ff9700 1 -- 172.21.15.45:0/3137027162 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c005440 con 0x7f5e54107420 2024-06-06T04:31:03.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.758+0000 7f5e597d4700 1 -- 172.21.15.45:0/3137027162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e541a4fc0 con 0x7f5e54107420 2024-06-06T04:31:03.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.758+0000 7f5e597d4700 1 -- 172.21.15.45:0/3137027162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e541a5520 con 0x7f5e54107420 2024-06-06T04:31:03.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.759+0000 7f5e597d4700 1 -- 172.21.15.45:0/3137027162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e54109770 con 0x7f5e54107420 2024-06-06T04:31:03.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.760+0000 7f5e50ff9700 1 -- 172.21.15.45:0/3137027162 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5e4c024070 con 0x7f5e54107420 2024-06-06T04:31:03.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.760+0000 7f5e50ff9700 1 --2- 172.21.15.45:0/3137027162 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5e400a2ca0 0x7f5e400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:03.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.761+0000 7f5e52ffd700 1 --2- 172.21.15.45:0/3137027162 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5e400a2ca0 0x7f5e400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:03.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.761+0000 7f5e50ff9700 1 -- 172.21.15.45:0/3137027162 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f5e4c00f020 con 0x7f5e54107420 2024-06-06T04:31:03.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.761+0000 7f5e52ffd700 1 --2- 172.21.15.45:0/3137027162 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5e400a2ca0 0x7f5e400a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5e44005fd0 tx=0x7f5e44005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:03.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.765+0000 7f5e50ff9700 1 -- 172.21.15.45:0/3137027162 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5e4c02b020 con 0x7f5e54107420 2024-06-06T04:31:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.971+0000 7f5e597d4700 1 -- 172.21.15.45:0/3137027162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e54109150 con 0x7f5e54107420 2024-06-06T04:31:03.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.972+0000 7f5e50ff9700 1 -- 172.21.15.45:0/3137027162 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24789 (secure 0 0 0) 0x7f5e4c09d660 con 0x7f5e54107420 2024-06-06T04:31:03.973 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:31:03.973 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":100,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","created":"2024-06-06T04:19:02.626916+0000","modified":"2024-06-06T04:29:05.901370+0000","last_up_change":"2024-06-06T04:26:17.390429+0000","last_in_change":"2024-06-06T04:25:56.938388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T04:22:28.797137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T04:27:17.750625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T04:27:21.486917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T04:27:31.029818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d3cee4a-5025-45b3-b217-082af37e79db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6803","nonce":2111759250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6805","nonce":2111759250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6809","nonce":2111759250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6807","nonce":2111759250}]},"public_addr":"172.21.15.45:6803/2111759250","cluster_addr":"172.21.15.45:6805/2111759250","heartbeat_back_addr":"172.21.15.45:6809/2111759250","heartbeat_front_addr":"172.21.15.45:6807/2111759250","state":["exists","up"]},{"osd":1,"uuid":"75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6801","nonce":3732225287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6803","nonce":3732225287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6807","nonce":3732225287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6805","nonce":3732225287}]},"public_addr":"172.21.15.86:6801/3732225287","cluster_addr":"172.21.15.86:6803/3732225287","heartbeat_back_addr":"172.21.15.86:6807/3732225287","heartbeat_front_addr":"172.21.15.86:6805/3732225287","state":["exists","up"]},{"osd":2,"uuid":"bb20565f-f9ae-47e3-9604-c69f737cff4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6801","nonce":2898783039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6803","nonce":2898783039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6807","nonce":2898783039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6805","nonce":2898783039}]},"public_addr":"172.21.15.154:6801/2898783039","cluster_addr":"172.21.15.154:6803/2898783039","heartbeat_back_addr":"172.21.15.154:6807/2898783039","heartbeat_front_addr":"172.21.15.154:6805/2898783039","state":["exists","up"]},{"osd":3,"uuid":"e1160799-7811-4326-9792-7e6c7d189c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6811","nonce":3436753135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6813","nonce":3436753135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6817","nonce":3436753135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6815","nonce":3436753135}]},"public_addr":"172.21.15.45:6811/3436753135","cluster_addr":"172.21.15.45:6813/3436753135","heartbeat_back_addr":"172.21.15.45:6817/3436753135","heartbeat_front_addr":"172.21.15.45:6815/3436753135","state":["exists","up"]},{"osd":4,"uuid":"6b389bc8-e19a-47f3-83ff-7b782d146d0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6809","nonce":3030235796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6811","nonce":3030235796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6815","nonce":3030235796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6813","nonce":3030235796}]},"public_addr":"172.21.15.86:6809/3030235796","cluster_addr":"172.21.15.86:6811/3030235796","heartbeat_back_addr":"172.21.15.86:6815/3030235796","heartbeat_front_addr":"172.21.15.86:6813/3030235796","state":["exists","up"]},{"osd":5,"uuid":"069de7b2-89bf-4150-bed7-7b8832573dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6809","nonce":2222268646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6811","nonce":2222268646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6815","nonce":2222268646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6813","nonce":2222268646}]},"public_addr":"172.21.15.154:6809/2222268646","cluster_addr":"172.21.15.154:6811/2222268646","heartbeat_back_addr":"172.21.15.154:6815/2222268646","heartbeat_front_addr":"172.21.15.154:6813/2222268646","state":["exists","up"]},{"osd":6,"uuid":"4c70b627-eafa-4e27-ba2a-4d8e1e335962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6819","nonce":384734939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6821","nonce":384734939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6825","nonce":384734939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6823","nonce":384734939}]},"public_addr":"172.21.15.45:6819/384734939","cluster_addr":"172.21.15.45:6821/384734939","heartbeat_back_addr":"172.21.15.45:6825/384734939","heartbeat_front_addr":"172.21.15.45:6823/384734939","state":["exists","up"]},{"osd":7,"uuid":"3b09b875-be52-4be0-b116-7fd9b0c26705","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6817","nonce":1119791005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6819","nonce":1119791005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6823","nonce":1119791005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6821","nonce":1119791005}]},"public_addr":"172.21.15.86:6817/1119791005","cluster_addr":"172.21.15.86:6819/1119791005","heartbeat_back_addr":"172.21.15.86:6823/1119791005","heartbeat_front_addr":"172.21.15.86:6821/1119791005","state":["exists","up"]},{"osd":8,"uuid":"cdf1dd1b-5392-4836-8e65-4c7c5b688d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6817","nonce":3054812496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6819","nonce":3054812496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6823","nonce":3054812496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6821","nonce":3054812496}]},"public_addr":"172.21.15.154:6817/3054812496","cluster_addr":"172.21.15.154:6819/3054812496","heartbeat_back_addr":"172.21.15.154:6823/3054812496","heartbeat_front_addr":"172.21.15.154:6821/3054812496","state":["exists","up"]},{"osd":9,"uuid":"acede92d-2cc1-4967-a357-7e5e201e99d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6827","nonce":1360655510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6829","nonce":1360655510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6833","nonce":1360655510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6831","nonce":1360655510}]},"public_addr":"172.21.15.45:6827/1360655510","cluster_addr":"172.21.15.45:6829/1360655510","heartbeat_back_addr":"172.21.15.45:6833/1360655510","heartbeat_front_addr":"172.21.15.45:6831/1360655510","state":["exists","up"]},{"osd":10,"uuid":"a2d0c488-1892-4da4-adbf-952c22df0ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6825","nonce":1701691877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6827","nonce":1701691877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6831","nonce":1701691877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6829","nonce":1701691877}]},"public_addr":"172.21.15.86:6825/1701691877","cluster_addr":"172.21.15.86:6827/1701691877","heartbeat_back_addr":"172.21.15.86:6831/1701691877","heartbeat_front_addr":"172.21.15.86:6829/1701691877","state":["exists","up"]},{"osd":11,"uuid":"6b349c4c-b672-480b-ac6a-8594dab0eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6825","nonce":2065648007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6827","nonce":2065648007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6831","nonce":2065648007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6829","nonce":2065648007}]},"public_addr":"172.21.15.154:6825/2065648007","cluster_addr":"172.21.15.154:6827/2065648007","heartbeat_back_addr":"172.21.15.154:6831/2065648007","heartbeat_front_addr":"172.21.15.154:6829/2065648007","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:21:37.147394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:00.776617+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:24.786849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:50.487455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:14.869272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:39.815158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:05.972619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:31.410063+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:56.795119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:23.114884+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:49.256504+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:26:15.251967+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/36732849":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/1500318775":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6801/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6800/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/2041380391":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/2504003341":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/3798552599":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/498952822":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6800/1562511056":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6801/1562511056":"2024-06-07T04:19:37.026199+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T04:31:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.974+0000 7f5e597d4700 1 -- 172.21.15.45:0/3137027162 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5e400a2ca0 msgr2=0x7f5e400a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.974+0000 7f5e597d4700 1 --2- 172.21.15.45:0/3137027162 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5e400a2ca0 0x7f5e400a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5e44005fd0 tx=0x7f5e44005db0 comp rx=0 tx=0).stop 2024-06-06T04:31:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.974+0000 7f5e597d4700 1 -- 172.21.15.45:0/3137027162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e54107420 msgr2=0x7f5e541a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.974+0000 7f5e597d4700 1 --2- 172.21.15.45:0/3137027162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e54107420 0x7f5e541a48b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00b070 tx=0x7f5e4c00ce50 comp rx=0 tx=0).stop 2024-06-06T04:31:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.974+0000 7f5e597d4700 1 -- 172.21.15.45:0/3137027162 shutdown_connections 2024-06-06T04:31:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.974+0000 7f5e597d4700 1 --2- 172.21.15.45:0/3137027162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e54107420 0x7f5e541a48b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.974+0000 7f5e597d4700 1 --2- 172.21.15.45:0/3137027162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e54101370 0x7f5e54199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.974+0000 7f5e597d4700 1 --2- 172.21.15.45:0/3137027162 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5e400a2ca0 0x7f5e400a5160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.974+0000 7f5e597d4700 1 --2- 172.21.15.45:0/3137027162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e54101d30 0x7f5e54199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.974+0000 7f5e597d4700 1 -- 172.21.15.45:0/3137027162 >> 172.21.15.45:0/3137027162 conn(0x7f5e54078340 msgr2=0x7f5e541000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.975+0000 7f5e597d4700 1 -- 172.21.15.45:0/3137027162 shutdown_connections 2024-06-06T04:31:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:03.975+0000 7f5e597d4700 1 -- 172.21.15.45:0/3137027162 wait complete. 2024-06-06T04:31:04.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:03 smithi154 ceph-mon[36428]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:31:04.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:31:04.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:31:04.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:03 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:04.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:03 smithi045 ceph-mon[32299]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:31:04.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:31:04.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:31:04.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:03 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:04.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:03 smithi086 ceph-mon[36053]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:31:04.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:31:04.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:31:04.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:03 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:31:04.646 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-06T04:27:24.462897+0000', 'modified': '2024-06-06T04:28:32.515629+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24473, 'mds_1': 24463, 'mds_2': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6835/4268654313', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 4268654313}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 4268654313}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.154:6835/954391764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 954391764}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 954391764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6837/414874753', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 414874753}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 414874753}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.154:6837/1140071744', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 1140071744}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 1140071744}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.154:6839/982197202', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6838', 'nonce': 982197202}, {'type': 'v1', 'addr': '172.21.15.154:6839', 'nonce': 982197202}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.86:6839/816791193', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 816791193}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 816791193}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24473, 'qdb_cluster': [24463, 24473, 24451]} 2024-06-06T04:31:04.646 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-06T04:31:04.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-06T04:31:04.900 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:31:05.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:04 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3137027162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:31:05.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:04 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3137027162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:31:05.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:04 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3137027162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:31:06.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:05 smithi045 ceph-mon[32299]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:31:06.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:05 smithi154 ceph-mon[36428]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:31:06.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:05 smithi086 ceph-mon[36053]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:31:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.617+0000 7fb9675c9700 1 -- 172.21.15.45:0/112324542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb960109480 msgr2=0x7fb960115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.617+0000 7fb9675c9700 1 --2- 172.21.15.45:0/112324542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb960109480 0x7fb960115990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb954009990 tx=0x7fb954009ca0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.617+0000 7fb9675c9700 1 -- 172.21.15.45:0/112324542 shutdown_connections 2024-06-06T04:31:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.617+0000 7fb9675c9700 1 --2- 172.21.15.45:0/112324542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb960109480 0x7fb960115990 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.617+0000 7fb9675c9700 1 --2- 172.21.15.45:0/112324542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb96010eac0 0x7fb96010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.617+0000 7fb9675c9700 1 --2- 172.21.15.45:0/112324542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb960108ac0 0x7fb960108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.617+0000 7fb9675c9700 1 -- 172.21.15.45:0/112324542 >> 172.21.15.45:0/112324542 conn(0x7fb960075c60 msgr2=0x7fb960078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.617+0000 7fb9675c9700 1 -- 172.21.15.45:0/112324542 shutdown_connections 2024-06-06T04:31:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.617+0000 7fb9675c9700 1 -- 172.21.15.45:0/112324542 wait complete. 2024-06-06T04:31:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.617+0000 7fb9675c9700 1 Processor -- start 2024-06-06T04:31:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb9675c9700 1 -- start start 2024-06-06T04:31:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb9675c9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb960108ac0 0x7fb96019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb9675c9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb960109480 0x7fb96019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb9675c9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb96010eac0 0x7fb9601a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb9675c9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb960118d60 con 0x7fb960108ac0 2024-06-06T04:31:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb9675c9700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb960118be0 con 0x7fb96010eac0 2024-06-06T04:31:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb9675c9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb960118ee0 con 0x7fb960109480 2024-06-06T04:31:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb965365700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb960108ac0 0x7fb96019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb965365700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb960108ac0 0x7fb96019f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53266/0 (socket says 172.21.15.45:53266) 2024-06-06T04:31:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb965365700 1 -- 172.21.15.45:0/461038143 learned_addr learned my addr 172.21.15.45:0/461038143 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb965b66700 1 --2- 172.21.15.45:0/461038143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb96010eac0 0x7fb9601a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb964b64700 1 --2- 172.21.15.45:0/461038143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb960109480 0x7fb96019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb965365700 1 -- 172.21.15.45:0/461038143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb960109480 msgr2=0x7fb96019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb965365700 1 --2- 172.21.15.45:0/461038143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb960109480 0x7fb96019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb965365700 1 -- 172.21.15.45:0/461038143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb96010eac0 msgr2=0x7fb9601a3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb965365700 1 --2- 172.21.15.45:0/461038143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb96010eac0 0x7fb9601a3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb965365700 1 -- 172.21.15.45:0/461038143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9601a41f0 con 0x7fb960108ac0 2024-06-06T04:31:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb965b66700 1 --2- 172.21.15.45:0/461038143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb96010eac0 0x7fb9601a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:31:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb964b64700 1 --2- 172.21.15.45:0/461038143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb960109480 0x7fb96019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:31:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.618+0000 7fb965365700 1 --2- 172.21.15.45:0/461038143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb960108ac0 0x7fb96019f230 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb95800b530 tx=0x7fb95800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.619+0000 7fb95e7fc700 1 -- 172.21.15.45:0/461038143 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958015020 con 0x7fb960108ac0 2024-06-06T04:31:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.619+0000 7fb9675c9700 1 -- 172.21.15.45:0/461038143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9601a44e0 con 0x7fb960108ac0 2024-06-06T04:31:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.619+0000 7fb95e7fc700 1 -- 172.21.15.45:0/461038143 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb958011ce0 con 0x7fb960108ac0 2024-06-06T04:31:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.619+0000 7fb95e7fc700 1 -- 172.21.15.45:0/461038143 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958010d60 con 0x7fb960108ac0 2024-06-06T04:31:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.619+0000 7fb9675c9700 1 -- 172.21.15.45:0/461038143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9601abd60 con 0x7fb960108ac0 2024-06-06T04:31:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.621+0000 7fb95e7fc700 1 -- 172.21.15.45:0/461038143 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb958018020 con 0x7fb960108ac0 2024-06-06T04:31:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.622+0000 7fb95e7fc700 1 --2- 172.21.15.45:0/461038143 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb9480a2c20 0x7fb9480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.622+0000 7fb964b64700 1 --2- 172.21.15.45:0/461038143 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb9480a2c20 0x7fb9480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.622+0000 7fb95e7fc700 1 -- 172.21.15.45:0/461038143 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fb9580ced00 con 0x7fb960108ac0 2024-06-06T04:31:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.622+0000 7fb964b64700 1 --2- 172.21.15.45:0/461038143 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb9480a2c20 0x7fb9480a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb96010dff0 tx=0x7fb950009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb971e4cb80 1 Processor -- start 2024-06-06T04:31:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb971e4cb80 1 -- start start 2024-06-06T04:31:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb971e4cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518ae5f0 0x55cd518ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb971e4cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518aef10 0x55cd518a8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb971e4cb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518a8ed0 0x55cd518a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb971e4cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cd517fa7c0 con 0x55cd518ae5f0 2024-06-06T04:31:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb971e4cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cd517a26d0 con 0x55cd518aef10 2024-06-06T04:31:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb971e4cb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cd517f6780 con 0x55cd518a8ed0 2024-06-06T04:31:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb965365700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518ae5f0 0x55cd518ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb965365700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518ae5f0 0x55cd518ae9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53282/0 (socket says 172.21.15.45:53282) 2024-06-06T04:31:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb965365700 1 -- 172.21.15.45:0/4056857022 learned_addr learned my addr 172.21.15.45:0/4056857022 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb965b66700 1 --2- 172.21.15.45:0/4056857022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518a8ed0 0x55cd518a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb965365700 1 -- 172.21.15.45:0/4056857022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518a8ed0 msgr2=0x55cd518a92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb965365700 1 --2- 172.21.15.45:0/4056857022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518a8ed0 0x55cd518a92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb964b64700 1 --2- 172.21.15.45:0/4056857022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518aef10 0x55cd518a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb965365700 1 -- 172.21.15.45:0/4056857022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518aef10 msgr2=0x55cd518a8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb965365700 1 --2- 172.21.15.45:0/4056857022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518aef10 0x55cd518a8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb965365700 1 -- 172.21.15.45:0/4056857022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd51798f00 con 0x55cd518ae5f0 2024-06-06T04:31:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.623+0000 7fb965365700 1 --2- 172.21.15.45:0/4056857022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518ae5f0 0x55cd518ae9d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb9580975a0 tx=0x7fb95809ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364e4c8943cd3740 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.624+0000 7fb9477fe700 1 -- 172.21.15.45:0/4056857022 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95809b840 con 0x55cd518ae5f0 2024-06-06T04:31:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.624+0000 7fb9477fe700 1 -- 172.21.15.45:0/4056857022 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb95809b140 con 0x55cd518ae5f0 2024-06-06T04:31:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.624+0000 7fb9477fe700 1 -- 172.21.15.45:0/4056857022 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9580bb240 con 0x55cd518ae5f0 2024-06-06T04:31:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.624+0000 7fb971e4cb80 1 -- 172.21.15.45:0/4056857022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518ae5f0 msgr2=0x55cd518ae9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.624+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/4056857022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518ae5f0 0x55cd518ae9d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb9580975a0 tx=0x7fb95809ac80 comp rx=0 tx=0).stop 2024-06-06T04:31:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.624+0000 7fb971e4cb80 1 -- 172.21.15.45:0/4056857022 shutdown_connections 2024-06-06T04:31:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.624+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/4056857022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518a8ed0 0x55cd518a92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.624+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/4056857022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518ae5f0 0x55cd518ae9d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.624+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/4056857022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518aef10 0x55cd518a8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.624+0000 7fb971e4cb80 1 -- 172.21.15.45:0/4056857022 >> 172.21.15.45:0/4056857022 conn(0x55cd517d3c70 msgr2=0x55cd518a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.625+0000 7fb971e4cb80 1 -- 172.21.15.45:0/4056857022 shutdown_connections 2024-06-06T04:31:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.625+0000 7fb971e4cb80 1 -- 172.21.15.45:0/4056857022 wait complete. 2024-06-06T04:31:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.625+0000 7fb971e4cb80 1 -- start start 2024-06-06T04:31:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.625+0000 7fb971e4cb80 1 Processor -- start 2024-06-06T04:31:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.625+0000 7fb971e4cb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518a8ed0 0x55cd518b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.625+0000 7fb971e4cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518ae5f0 0x55cd518b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb971e4cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518aef10 0x55cd518b2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb971e4cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cd518b5980 con 0x55cd518aef10 2024-06-06T04:31:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb971e4cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cd5179aba0 con 0x55cd518ae5f0 2024-06-06T04:31:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb971e4cb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cd518b5b00 con 0x55cd518a8ed0 2024-06-06T04:31:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb971e4cb80 10 client.0 _ll_register_callbacks cb 0x55cd517948d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:31:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb965b66700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518aef10 0x55cd518b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb965b66700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518aef10 0x55cd518b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53294/0 (socket says 172.21.15.45:53294) 2024-06-06T04:31:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb965b66700 1 -- 172.21.15.45:0/1704872167 learned_addr learned my addr 172.21.15.45:0/1704872167 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb965365700 1 --2- 172.21.15.45:0/1704872167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518a8ed0 0x55cd518b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb964b64700 1 --2- 172.21.15.45:0/1704872167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518ae5f0 0x55cd518b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb965b66700 1 -- 172.21.15.45:0/1704872167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518a8ed0 msgr2=0x55cd518b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb965b66700 1 --2- 172.21.15.45:0/1704872167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518a8ed0 0x55cd518b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb965b66700 1 -- 172.21.15.45:0/1704872167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518ae5f0 msgr2=0x55cd518b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb965b66700 1 --2- 172.21.15.45:0/1704872167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518ae5f0 0x55cd518b1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb965b66700 1 -- 172.21.15.45:0/1704872167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd51749880 con 0x55cd518aef10 2024-06-06T04:31:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb964b64700 1 --2- 172.21.15.45:0/1704872167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518ae5f0 0x55cd518b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:31:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb965b66700 1 --2- 172.21.15.45:0/1704872167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518aef10 0x55cd518b2450 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb95400b5c0 tx=0x7fb954003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecfdd43c6512982 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb944ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:31:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb944ff9700 1 -- 172.21.15.45:0/1704872167 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954027060 con 0x55cd518aef10 2024-06-06T04:31:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb971e4cb80 1 -- 172.21.15.45:0/1704872167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb95809b140 con 0x55cd518aef10 2024-06-06T04:31:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.626+0000 7fb944ff9700 1 -- 172.21.15.45:0/1704872167 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fb9540054d0 con 0x55cd518aef10 2024-06-06T04:31:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.627+0000 7fb944ff9700 1 -- 172.21.15.45:0/1704872167 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb95402e050 con 0x55cd518aef10 2024-06-06T04:31:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.627+0000 7fb971e4cb80 10 client.15162 fetch_fsmap learned FSMap version 26 2024-06-06T04:31:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.627+0000 7fb971e4cb80 1 -- 172.21.15.45:0/1704872167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cd518c02a0 con 0x55cd518aef10 2024-06-06T04:31:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.627+0000 7fb944ff9700 1 -- 172.21.15.45:0/1704872167 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb95809b140 con 0x55cd518aef10 2024-06-06T04:31:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.627+0000 7fb944ff9700 1 -- 172.21.15.45:0/1704872167 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb954021070 con 0x55cd518aef10 2024-06-06T04:31:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.627+0000 7fb971e4cb80 10 client.15162 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T04:31:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.627+0000 7fb971e4cb80 10 client.15162 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-06T04:31:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.627+0000 7fb971e4cb80 20 client.15162 populate_metadata read hostname 'smithi045' 2024-06-06T04:31:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.627+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/1704872167 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55cd518c2920 0x55cd518c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.627+0000 7fb971e4cb80 4 client.15162 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:31:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.627+0000 7fb971e4cb80 1 -- 172.21.15.45:0/1704872167 --> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd51807cc0 con 0x55cd518c2920 2024-06-06T04:31:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.627+0000 7fb965365700 1 --2- 172.21.15.45:0/1704872167 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55cd518c2920 0x55cd518c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.628+0000 7fb944ff9700 10 client.15162 ms_handle_connect on v2:172.21.15.154:6836/1140071744 2024-06-06T04:31:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.628+0000 7fb965365700 1 --2- 172.21.15.45:0/1704872167 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55cd518c2920 0x55cd518c3d90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=977cc66b239cc482 server_cookie=d2f416bd6234f427 in_seq=0 out_seq=0 2024-06-06T04:31:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.629+0000 7fb944ff9700 1 -- 172.21.15.45:0/1704872167 <== mds.2 v2:172.21.15.154:6836/1140071744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cd51807cc0 con 0x55cd518c2920 2024-06-06T04:31:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.629+0000 7fb944ff9700 10 client.15162 handle_command_reply: tid=0 2024-06-06T04:31:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.629+0000 7fb971e4cb80 1 client.15162 shutdown 2024-06-06T04:31:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.629+0000 7fb971e4cb80 1 -- 172.21.15.45:0/1704872167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518aef10 msgr2=0x55cd518b2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.629+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/1704872167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518aef10 0x55cd518b2450 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb95400b5c0 tx=0x7fb954003f80 comp rx=0 tx=0).stop 2024-06-06T04:31:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.629+0000 7fb971e4cb80 1 -- 172.21.15.45:0/1704872167 shutdown_connections 2024-06-06T04:31:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.629+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/1704872167 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55cd518c2920 0x55cd518c3d90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.629+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/1704872167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518aef10 0x55cd518b2450 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.629+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/1704872167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518a8ed0 0x55cd518b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.629+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/1704872167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518ae5f0 0x55cd518b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.629+0000 7fb971e4cb80 1 -- 172.21.15.45:0/1704872167 >> 172.21.15.45:0/1704872167 conn(0x55cd517d3c70 msgr2=0x55cd518a7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.629+0000 7fb944ff9700 0 client.15162 ms_handle_reset on v2:172.21.15.154:6836/1140071744 2024-06-06T04:31:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.629+0000 7fb971e4cb80 1 -- 172.21.15.45:0/1704872167 shutdown_connections 2024-06-06T04:31:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.629+0000 7fb971e4cb80 1 -- 172.21.15.45:0/1704872167 wait complete. 2024-06-06T04:31:06.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.629+0000 7fb971e4cb80 20 client.15162 trim_cache size 0 max 16384 2024-06-06T04:31:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.650+0000 7fb971e4cb80 1 Processor -- start 2024-06-06T04:31:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.650+0000 7fb971e4cb80 1 -- start start 2024-06-06T04:31:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb971e4cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518a8ed0 0x55cd518a8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb971e4cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518ae5f0 0x55cd518b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb971e4cb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518aef10 0x55cd518b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb971e4cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cd518c20d0 con 0x55cd518a8ed0 2024-06-06T04:31:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb971e4cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cd518c27a0 con 0x55cd518ae5f0 2024-06-06T04:31:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb971e4cb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cd518c2e70 con 0x55cd518aef10 2024-06-06T04:31:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb965365700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518a8ed0 0x55cd518a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb965365700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518a8ed0 0x55cd518a8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53298/0 (socket says 172.21.15.45:53298) 2024-06-06T04:31:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb965365700 1 -- 172.21.15.45:0/3154449435 learned_addr learned my addr 172.21.15.45:0/3154449435 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb964b64700 1 --2- 172.21.15.45:0/3154449435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518ae5f0 0x55cd518b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb965b66700 1 --2- 172.21.15.45:0/3154449435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518aef10 0x55cd518b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb965365700 1 -- 172.21.15.45:0/3154449435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518aef10 msgr2=0x55cd518b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb965365700 1 --2- 172.21.15.45:0/3154449435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518aef10 0x55cd518b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb965365700 1 -- 172.21.15.45:0/3154449435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518ae5f0 msgr2=0x55cd518b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb965365700 1 --2- 172.21.15.45:0/3154449435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518ae5f0 0x55cd518b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb965365700 1 -- 172.21.15.45:0/3154449435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd518c4cf0 con 0x55cd518a8ed0 2024-06-06T04:31:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb965b66700 1 --2- 172.21.15.45:0/3154449435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518aef10 0x55cd518b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:31:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb965365700 1 --2- 172.21.15.45:0/3154449435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518a8ed0 0x55cd518a8990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb958097b10 tx=0x7fb9580bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63af4e0785d0638 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb946ffd700 1 -- 172.21.15.45:0/3154449435 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958065070 con 0x55cd518a8ed0 2024-06-06T04:31:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb946ffd700 1 -- 172.21.15.45:0/3154449435 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb95809f4f0 con 0x55cd518a8ed0 2024-06-06T04:31:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.651+0000 7fb946ffd700 1 -- 172.21.15.45:0/3154449435 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9580bb240 con 0x55cd518a8ed0 2024-06-06T04:31:06.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.652+0000 7fb971e4cb80 1 -- 172.21.15.45:0/3154449435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518a8ed0 msgr2=0x55cd518a8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:06.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.652+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/3154449435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518a8ed0 0x55cd518a8990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb958097b10 tx=0x7fb9580bb040 comp rx=0 tx=0).stop 2024-06-06T04:31:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.652+0000 7fb971e4cb80 1 -- 172.21.15.45:0/3154449435 shutdown_connections 2024-06-06T04:31:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.652+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/3154449435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518aef10 0x55cd518b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.652+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/3154449435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518a8ed0 0x55cd518a8990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.652+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/3154449435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518ae5f0 0x55cd518b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.652+0000 7fb971e4cb80 1 -- 172.21.15.45:0/3154449435 >> 172.21.15.45:0/3154449435 conn(0x55cd517d3c70 msgr2=0x55cd518a3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.652+0000 7fb971e4cb80 1 -- 172.21.15.45:0/3154449435 shutdown_connections 2024-06-06T04:31:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.652+0000 7fb971e4cb80 1 -- 172.21.15.45:0/3154449435 wait complete. 2024-06-06T04:31:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.652+0000 7fb971e4cb80 1 -- start start 2024-06-06T04:31:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb971e4cb80 1 Processor -- start 2024-06-06T04:31:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb971e4cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518a8ed0 0x55cd518b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb971e4cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518ae5f0 0x55cd518b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb971e4cb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518aef10 0x55cd518cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb971e4cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cd518c3c10 con 0x55cd518ae5f0 2024-06-06T04:31:06.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb971e4cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cd518c3540 con 0x55cd518a8ed0 2024-06-06T04:31:06.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb971e4cb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cd518c42e0 con 0x55cd518aef10 2024-06-06T04:31:06.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb971e4cb80 10 client.0 _ll_register_callbacks cb 0x55cd517948d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:31:06.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb964b64700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518ae5f0 0x55cd518b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb964b64700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518ae5f0 0x55cd518b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53300/0 (socket says 172.21.15.45:53300) 2024-06-06T04:31:06.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb964b64700 1 -- 172.21.15.45:0/3037185180 learned_addr learned my addr 172.21.15.45:0/3037185180 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:06.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb965b66700 1 --2- 172.21.15.45:0/3037185180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518aef10 0x55cd518cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb965365700 1 --2- 172.21.15.45:0/3037185180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518a8ed0 0x55cd518b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb964b64700 1 -- 172.21.15.45:0/3037185180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518aef10 msgr2=0x55cd518cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb964b64700 1 --2- 172.21.15.45:0/3037185180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518aef10 0x55cd518cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb964b64700 1 -- 172.21.15.45:0/3037185180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518a8ed0 msgr2=0x55cd518b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb964b64700 1 --2- 172.21.15.45:0/3037185180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518a8ed0 0x55cd518b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.653+0000 7fb964b64700 1 -- 172.21.15.45:0/3037185180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd518cc740 con 0x55cd518ae5f0 2024-06-06T04:31:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.654+0000 7fb964b64700 1 --2- 172.21.15.45:0/3037185180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518ae5f0 0x55cd518b1b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb9500098b0 tx=0x7fb950004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf961dc1d2a8f06 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.654+0000 7fb944ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:31:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.654+0000 7fb944ff9700 1 -- 172.21.15.45:0/3037185180 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950028020 con 0x55cd518ae5f0 2024-06-06T04:31:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.654+0000 7fb944ff9700 1 -- 172.21.15.45:0/3037185180 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fb950012030 con 0x55cd518ae5f0 2024-06-06T04:31:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.654+0000 7fb971e4cb80 1 -- 172.21.15.45:0/3037185180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb95809f4f0 con 0x55cd518ae5f0 2024-06-06T04:31:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.654+0000 7fb944ff9700 1 -- 172.21.15.45:0/3037185180 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb950033050 con 0x55cd518ae5f0 2024-06-06T04:31:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.654+0000 7fb971e4cb80 10 client.15168 fetch_fsmap learned FSMap version 26 2024-06-06T04:31:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.654+0000 7fb971e4cb80 1 -- 172.21.15.45:0/3037185180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cd518cd9f0 con 0x55cd518ae5f0 2024-06-06T04:31:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.654+0000 7fb944ff9700 1 -- 172.21.15.45:0/3037185180 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb95809f4f0 con 0x55cd518ae5f0 2024-06-06T04:31:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.654+0000 7fb944ff9700 1 -- 172.21.15.45:0/3037185180 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb950024830 con 0x55cd518ae5f0 2024-06-06T04:31:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.654+0000 7fb971e4cb80 10 client.15168 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T04:31:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.655+0000 7fb971e4cb80 10 client.15168 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-06T04:31:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.655+0000 7fb971e4cb80 20 client.15168 populate_metadata read hostname 'smithi045' 2024-06-06T04:31:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.655+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/3037185180 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55cd51932820 0x55cd51931530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.655+0000 7fb971e4cb80 4 client.15168 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T04:31:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.655+0000 7fb971e4cb80 1 -- 172.21.15.45:0/3037185180 --> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] -- command(tid 0: {"prefix": "status"}) -- 0x55cd51802240 con 0x55cd51932820 2024-06-06T04:31:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.655+0000 7fb965365700 1 --2- 172.21.15.45:0/3037185180 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55cd51932820 0x55cd51931530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.655+0000 7fb965365700 1 --2- 172.21.15.45:0/3037185180 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55cd51932820 0x55cd51931530 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a5af42b293f40f3 server_cookie=e4a13fc09a3fabcc in_seq=0 out_seq=0 2024-06-06T04:31:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.655+0000 7fb944ff9700 10 client.15168 ms_handle_connect on v2:172.21.15.154:6836/1140071744 2024-06-06T04:31:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.656+0000 7fb944ff9700 1 -- 172.21.15.45:0/3037185180 <== mds.2 v2:172.21.15.154:6836/1140071744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55cd51802240 con 0x55cd51932820 2024-06-06T04:31:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.656+0000 7fb944ff9700 10 client.15168 handle_command_reply: tid=0 2024-06-06T04:31:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.656+0000 7fb971e4cb80 1 client.15168 shutdown 2024-06-06T04:31:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.656+0000 7fb971e4cb80 1 -- 172.21.15.45:0/3037185180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518ae5f0 msgr2=0x55cd518b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.656+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/3037185180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518ae5f0 0x55cd518b1b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb9500098b0 tx=0x7fb950004ad0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.656+0000 7fb971e4cb80 1 -- 172.21.15.45:0/3037185180 shutdown_connections 2024-06-06T04:31:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.656+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/3037185180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd518aef10 0x55cd518cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.656+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/3037185180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd518a8ed0 0x55cd518b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.656+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/3037185180 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55cd51932820 0x55cd51931530 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.656+0000 7fb971e4cb80 1 --2- 172.21.15.45:0/3037185180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd518ae5f0 0x55cd518b1b90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.656+0000 7fb971e4cb80 1 -- 172.21.15.45:0/3037185180 >> 172.21.15.45:0/3037185180 conn(0x55cd517d3c70 msgr2=0x55cd518af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.656+0000 7fb944ff9700 0 client.15168 ms_handle_reset on v2:172.21.15.154:6836/1140071744 2024-06-06T04:31:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.656+0000 7fb971e4cb80 1 -- 172.21.15.45:0/3037185180 shutdown_connections 2024-06-06T04:31:06.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.656+0000 7fb971e4cb80 1 -- 172.21.15.45:0/3037185180 wait complete. 2024-06-06T04:31:06.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.656+0000 7fb971e4cb80 20 client.15168 trim_cache size 0 max 16384 2024-06-06T04:31:06.663 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:31:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "c4f6b938-23bb-11ef-bc9b-c7b262605968", 2024-06-06T04:31:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 2, 2024-06-06T04:31:06.663 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24451, 2024-06-06T04:31:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-06-06T04:31:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-06-06T04:31:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-06-06T04:31:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 158.94604996300001, 2024-06-06T04:31:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2024-06-06T04:31:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 100, 2024-06-06T04:31:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 98, 2024-06-06T04:31:06.664 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 253.43432111199999 2024-06-06T04:31:06.664 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:31:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.657+0000 7fb9675c9700 1 -- 172.21.15.45:0/461038143 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb9480a2c20 msgr2=0x7fb9480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.657+0000 7fb9675c9700 1 --2- 172.21.15.45:0/461038143 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb9480a2c20 0x7fb9480a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb96010dff0 tx=0x7fb950009570 comp rx=0 tx=0).stop 2024-06-06T04:31:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.657+0000 7fb9675c9700 1 -- 172.21.15.45:0/461038143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb960108ac0 msgr2=0x7fb96019f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.657+0000 7fb9675c9700 1 --2- 172.21.15.45:0/461038143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb960108ac0 0x7fb96019f230 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb95800b530 tx=0x7fb95800b8f0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.657+0000 7fb9675c9700 1 -- 172.21.15.45:0/461038143 shutdown_connections 2024-06-06T04:31:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.657+0000 7fb9675c9700 1 --2- 172.21.15.45:0/461038143 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb9480a2c20 0x7fb9480a50e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.657+0000 7fb9675c9700 1 --2- 172.21.15.45:0/461038143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb96010eac0 0x7fb9601a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.657+0000 7fb9675c9700 1 --2- 172.21.15.45:0/461038143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb960108ac0 0x7fb96019f230 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.657+0000 7fb9675c9700 1 --2- 172.21.15.45:0/461038143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb960109480 0x7fb96019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:06.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.657+0000 7fb9675c9700 1 -- 172.21.15.45:0/461038143 >> 172.21.15.45:0/461038143 conn(0x7fb960075c60 msgr2=0x7fb960076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:06.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.657+0000 7fb9675c9700 1 -- 172.21.15.45:0/461038143 shutdown_connections 2024-06-06T04:31:06.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:06.657+0000 7fb9675c9700 1 -- 172.21.15.45:0/461038143 wait complete. 2024-06-06T04:31:07.365 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-06T04:31:07.614 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:31:08.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:07 smithi045 ceph-mon[32299]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:31:08.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:07 smithi154 ceph-mon[36428]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:31:08.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:07 smithi086 ceph-mon[36053]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:31:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.346+0000 7f181e3fa700 1 -- 172.21.15.45:0/623922976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1818102520 msgr2=0x7f18181029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.346+0000 7f181e3fa700 1 --2- 172.21.15.45:0/623922976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1818102520 0x7f18181029a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f18000099c0 tx=0x7f1800009cd0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.346+0000 7f181e3fa700 1 -- 172.21.15.45:0/623922976 shutdown_connections 2024-06-06T04:31:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.346+0000 7f181e3fa700 1 --2- 172.21.15.45:0/623922976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1818102ee0 0x7f181810f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.346+0000 7f181e3fa700 1 --2- 172.21.15.45:0/623922976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18181085c0 0x7f18181089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.346+0000 7f181e3fa700 1 --2- 172.21.15.45:0/623922976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1818102520 0x7f18181029a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.346+0000 7f181e3fa700 1 -- 172.21.15.45:0/623922976 >> 172.21.15.45:0/623922976 conn(0x7f1818079680 msgr2=0x7f1818079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.347+0000 7f181e3fa700 1 -- 172.21.15.45:0/623922976 shutdown_connections 2024-06-06T04:31:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.347+0000 7f181e3fa700 1 -- 172.21.15.45:0/623922976 wait complete. 2024-06-06T04:31:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.347+0000 7f181e3fa700 1 Processor -- start 2024-06-06T04:31:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.347+0000 7f181e3fa700 1 -- start start 2024-06-06T04:31:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.347+0000 7f181e3fa700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1818102520 0x7f181819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.347+0000 7f181e3fa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1818102ee0 0x7f181819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.347+0000 7f181e3fa700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18181085c0 0x7f1818198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.347+0000 7f181e3fa700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18181126d0 con 0x7f1818102ee0 2024-06-06T04:31:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.347+0000 7f181e3fa700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1818112550 con 0x7f1818102520 2024-06-06T04:31:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.347+0000 7f181e3fa700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1818112850 con 0x7f18181085c0 2024-06-06T04:31:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f180ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1818102ee0 0x7f181819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f180ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1818102ee0 0x7f181819ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55248/0 (socket says 172.21.15.45:55248) 2024-06-06T04:31:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f180ffff700 1 -- 172.21.15.45:0/3852914968 learned_addr learned my addr 172.21.15.45:0/3852914968 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f1817fff700 1 --2- 172.21.15.45:0/3852914968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1818102520 0x7f181819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f181c997700 1 --2- 172.21.15.45:0/3852914968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18181085c0 0x7f1818198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f180ffff700 1 -- 172.21.15.45:0/3852914968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18181085c0 msgr2=0x7f1818198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f180ffff700 1 --2- 172.21.15.45:0/3852914968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18181085c0 0x7f1818198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f180ffff700 1 -- 172.21.15.45:0/3852914968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1818102520 msgr2=0x7f181819e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f180ffff700 1 --2- 172.21.15.45:0/3852914968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1818102520 0x7f181819e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f180ffff700 1 -- 172.21.15.45:0/3852914968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1818199090 con 0x7f1818102ee0 2024-06-06T04:31:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f181c997700 1 --2- 172.21.15.45:0/3852914968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18181085c0 0x7f1818198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:31:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f180ffff700 1 --2- 172.21.15.45:0/3852914968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1818102ee0 0x7f181819ecb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1800006010 tx=0x7f1800009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f1815ffb700 1 -- 172.21.15.45:0/3852914968 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800027070 con 0x7f1818102ee0 2024-06-06T04:31:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f1815ffb700 1 -- 172.21.15.45:0/3852914968 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1800003f50 con 0x7f1818102ee0 2024-06-06T04:31:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f181e3fa700 1 -- 172.21.15.45:0/3852914968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1818199320 con 0x7f1818102ee0 2024-06-06T04:31:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f1815ffb700 1 -- 172.21.15.45:0/3852914968 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800022cd0 con 0x7f1818102ee0 2024-06-06T04:31:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.348+0000 7f181e3fa700 1 -- 172.21.15.45:0/3852914968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18181a5640 con 0x7f1818102ee0 2024-06-06T04:31:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.351+0000 7f1815ffb700 1 -- 172.21.15.45:0/3852914968 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f180002a070 con 0x7f1818102ee0 2024-06-06T04:31:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.351+0000 7f1815ffb700 1 --2- 172.21.15.45:0/3852914968 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f18040a2c20 0x7f18040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.351+0000 7f1815ffb700 1 -- 172.21.15.45:0/3852914968 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f18000d0210 con 0x7f1818102ee0 2024-06-06T04:31:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.351+0000 7f1817fff700 1 --2- 172.21.15.45:0/3852914968 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f18040a2c20 0x7f18040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.352+0000 7f1817fff700 1 --2- 172.21.15.45:0/3852914968 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f18040a2c20 0x7f18040a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1818199a70 tx=0x7f180800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.352+0000 7f1828c7db80 1 Processor -- start 2024-06-06T04:31:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.352+0000 7f1828c7db80 1 -- start start 2024-06-06T04:31:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.352+0000 7f1828c7db80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e115f0 0x55ab95e119d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.352+0000 7f1828c7db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e11f10 0x55ab95e0b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.352+0000 7f1828c7db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e0bed0 0x55ab95e0c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.352+0000 7f1828c7db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ab95d5d7c0 con 0x55ab95e11f10 2024-06-06T04:31:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.352+0000 7f1828c7db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ab95d056d0 con 0x55ab95e0bed0 2024-06-06T04:31:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.352+0000 7f1828c7db80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ab95d59780 con 0x55ab95e115f0 2024-06-06T04:31:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.353+0000 7f180ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e11f10 0x55ab95e0b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.353+0000 7f180ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e11f10 0x55ab95e0b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55254/0 (socket says 172.21.15.45:55254) 2024-06-06T04:31:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.353+0000 7f180ffff700 1 -- 172.21.15.45:0/1018546732 learned_addr learned my addr 172.21.15.45:0/1018546732 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.353+0000 7f181c997700 1 --2- 172.21.15.45:0/1018546732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e0bed0 0x55ab95e0c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.353+0000 7f1817fff700 1 --2- 172.21.15.45:0/1018546732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e115f0 0x55ab95e119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.353+0000 7f180ffff700 1 -- 172.21.15.45:0/1018546732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e115f0 msgr2=0x55ab95e119d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.353+0000 7f180ffff700 1 --2- 172.21.15.45:0/1018546732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e115f0 0x55ab95e119d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.353+0000 7f180ffff700 1 -- 172.21.15.45:0/1018546732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e0bed0 msgr2=0x55ab95e0c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.353+0000 7f180ffff700 1 --2- 172.21.15.45:0/1018546732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e0bed0 0x55ab95e0c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.353+0000 7f180ffff700 1 -- 172.21.15.45:0/1018546732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab95cfbf00 con 0x55ab95e11f10 2024-06-06T04:31:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.353+0000 7f181c997700 1 --2- 172.21.15.45:0/1018546732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e0bed0 0x55ab95e0c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:31:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.353+0000 7f180ffff700 1 --2- 172.21.15.45:0/1018546732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e11f10 0x55ab95e0b990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1800098ba0 tx=0x7f180009c190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7f86c45ed5ddf6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.353+0000 7f180e7fc700 1 -- 172.21.15.45:0/1018546732 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180009ce40 con 0x55ab95e11f10 2024-06-06T04:31:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.353+0000 7f180e7fc700 1 -- 172.21.15.45:0/1018546732 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f180009c740 con 0x55ab95e11f10 2024-06-06T04:31:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.353+0000 7f180e7fc700 1 -- 172.21.15.45:0/1018546732 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18000a4e90 con 0x55ab95e11f10 2024-06-06T04:31:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.354+0000 7f1828c7db80 1 -- 172.21.15.45:0/1018546732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e11f10 msgr2=0x55ab95e0b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.354+0000 7f1828c7db80 1 --2- 172.21.15.45:0/1018546732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e11f10 0x55ab95e0b990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1800098ba0 tx=0x7f180009c190 comp rx=0 tx=0).stop 2024-06-06T04:31:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.354+0000 7f1828c7db80 1 -- 172.21.15.45:0/1018546732 shutdown_connections 2024-06-06T04:31:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.354+0000 7f1828c7db80 1 --2- 172.21.15.45:0/1018546732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e0bed0 0x55ab95e0c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.354+0000 7f1828c7db80 1 --2- 172.21.15.45:0/1018546732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e115f0 0x55ab95e119d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.354+0000 7f1828c7db80 1 --2- 172.21.15.45:0/1018546732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e11f10 0x55ab95e0b990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.354+0000 7f1828c7db80 1 -- 172.21.15.45:0/1018546732 >> 172.21.15.45:0/1018546732 conn(0x55ab95d36c70 msgr2=0x55ab95e06800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.354+0000 7f1828c7db80 1 -- 172.21.15.45:0/1018546732 shutdown_connections 2024-06-06T04:31:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.354+0000 7f1828c7db80 1 -- 172.21.15.45:0/1018546732 wait complete. 2024-06-06T04:31:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.354+0000 7f1828c7db80 1 -- start start 2024-06-06T04:31:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f1828c7db80 1 Processor -- start 2024-06-06T04:31:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f1828c7db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e0bed0 0x55ab95e17ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f1828c7db80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e115f0 0x55ab95e14fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f1828c7db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e11f10 0x55ab95e15520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f1828c7db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ab95e186a0 con 0x55ab95e0bed0 2024-06-06T04:31:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f1828c7db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ab95cfdba0 con 0x55ab95e11f10 2024-06-06T04:31:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f1828c7db80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ab95e18820 con 0x55ab95e115f0 2024-06-06T04:31:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f1828c7db80 10 client.0 _ll_register_callbacks cb 0x55ab95cf78d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:31:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f1817fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e0bed0 0x55ab95e17ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f1817fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e0bed0 0x55ab95e17ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55270/0 (socket says 172.21.15.45:55270) 2024-06-06T04:31:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f1817fff700 1 -- 172.21.15.45:0/2297649079 learned_addr learned my addr 172.21.15.45:0/2297649079 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f180ffff700 1 --2- 172.21.15.45:0/2297649079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e115f0 0x55ab95e14fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f181c997700 1 --2- 172.21.15.45:0/2297649079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e11f10 0x55ab95e15520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f1817fff700 1 -- 172.21.15.45:0/2297649079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e115f0 msgr2=0x55ab95e14fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f1817fff700 1 --2- 172.21.15.45:0/2297649079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e115f0 0x55ab95e14fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f1817fff700 1 -- 172.21.15.45:0/2297649079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e11f10 msgr2=0x55ab95e15520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f1817fff700 1 --2- 172.21.15.45:0/2297649079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e11f10 0x55ab95e15520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f1817fff700 1 -- 172.21.15.45:0/2297649079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab95cac880 con 0x55ab95e0bed0 2024-06-06T04:31:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.355+0000 7f180ffff700 1 --2- 172.21.15.45:0/2297649079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e115f0 0x55ab95e14fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:31:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.356+0000 7f1817fff700 1 --2- 172.21.15.45:0/2297649079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e0bed0 0x55ab95e17ce0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1808003d10 tx=0x7f18080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e213cb8d7c9d83 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.356+0000 7f17f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:31:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.356+0000 7f17f77fe700 1 -- 172.21.15.45:0/2297649079 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808011e60 con 0x55ab95e0bed0 2024-06-06T04:31:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.356+0000 7f17f77fe700 1 -- 172.21.15.45:0/2297649079 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f1808028470 con 0x55ab95e0bed0 2024-06-06T04:31:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.356+0000 7f1828c7db80 1 -- 172.21.15.45:0/2297649079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f180009c740 con 0x55ab95e0bed0 2024-06-06T04:31:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.356+0000 7f17f77fe700 1 -- 172.21.15.45:0/2297649079 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1808031050 con 0x55ab95e0bed0 2024-06-06T04:31:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.356+0000 7f1828c7db80 10 client.15180 fetch_fsmap learned FSMap version 26 2024-06-06T04:31:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.356+0000 7f1828c7db80 1 -- 172.21.15.45:0/2297649079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ab95e23080 con 0x55ab95e0bed0 2024-06-06T04:31:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.356+0000 7f17f77fe700 1 -- 172.21.15.45:0/2297649079 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f180009c740 con 0x55ab95e0bed0 2024-06-06T04:31:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.356+0000 7f17f77fe700 1 -- 172.21.15.45:0/2297649079 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f180801e5f0 con 0x55ab95e0bed0 2024-06-06T04:31:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.356+0000 7f1828c7db80 10 client.15180 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T04:31:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.356+0000 7f1828c7db80 10 client.15180 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T04:31:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.356+0000 7f1828c7db80 20 client.15180 populate_metadata read hostname 'smithi045' 2024-06-06T04:31:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.356+0000 7f1828c7db80 1 --2- 172.21.15.45:0/2297649079 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55ab95e25690 0x55ab95e26b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.356+0000 7f1828c7db80 4 client.15180 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:31:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.356+0000 7f1828c7db80 1 -- 172.21.15.45:0/2297649079 --> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab95d6acc0 con 0x55ab95e25690 2024-06-06T04:31:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.357+0000 7f180ffff700 1 --2- 172.21.15.45:0/2297649079 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55ab95e25690 0x55ab95e26b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.357+0000 7f17f77fe700 10 client.15180 ms_handle_connect on v2:172.21.15.154:6838/982197202 2024-06-06T04:31:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.357+0000 7f180ffff700 1 --2- 172.21.15.45:0/2297649079 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55ab95e25690 0x55ab95e26b00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89c8e0a69f087c81 server_cookie=83994ee3fdb1cc70 in_seq=0 out_seq=0 2024-06-06T04:31:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.358+0000 7f17f77fe700 1 -- 172.21.15.45:0/2297649079 <== mds.1 v2:172.21.15.154:6838/982197202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ab95d6acc0 con 0x55ab95e25690 2024-06-06T04:31:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.358+0000 7f17f77fe700 10 client.15180 handle_command_reply: tid=0 2024-06-06T04:31:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.358+0000 7f1828c7db80 1 client.15180 shutdown 2024-06-06T04:31:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.358+0000 7f1828c7db80 1 -- 172.21.15.45:0/2297649079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e0bed0 msgr2=0x55ab95e17ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.358+0000 7f1828c7db80 1 --2- 172.21.15.45:0/2297649079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e0bed0 0x55ab95e17ce0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1808003d10 tx=0x7f18080047c0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.358+0000 7f1828c7db80 1 -- 172.21.15.45:0/2297649079 shutdown_connections 2024-06-06T04:31:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.358+0000 7f1828c7db80 1 --2- 172.21.15.45:0/2297649079 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55ab95e25690 0x55ab95e26b00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.358+0000 7f1828c7db80 1 --2- 172.21.15.45:0/2297649079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e11f10 0x55ab95e15520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.358+0000 7f1828c7db80 1 --2- 172.21.15.45:0/2297649079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e0bed0 0x55ab95e17ce0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.358+0000 7f1828c7db80 1 --2- 172.21.15.45:0/2297649079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e115f0 0x55ab95e14fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.358+0000 7f1828c7db80 1 -- 172.21.15.45:0/2297649079 >> 172.21.15.45:0/2297649079 conn(0x55ab95d36c70 msgr2=0x55ab95e0a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.358+0000 7f17f77fe700 0 client.15180 ms_handle_reset on v2:172.21.15.154:6838/982197202 2024-06-06T04:31:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.358+0000 7f1828c7db80 1 -- 172.21.15.45:0/2297649079 shutdown_connections 2024-06-06T04:31:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.358+0000 7f1828c7db80 1 -- 172.21.15.45:0/2297649079 wait complete. 2024-06-06T04:31:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.359+0000 7f1828c7db80 20 client.15180 trim_cache size 0 max 16384 2024-06-06T04:31:09.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.380+0000 7f1828c7db80 1 Processor -- start 2024-06-06T04:31:09.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.380+0000 7f1828c7db80 1 -- start start 2024-06-06T04:31:09.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.380+0000 7f1828c7db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e0bed0 0x55ab95e0b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.380+0000 7f1828c7db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e115f0 0x55ab95e069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.380+0000 7f1828c7db80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e11f10 0x55ab95e0f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.380+0000 7f1828c7db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ab95e24e40 con 0x55ab95e0bed0 2024-06-06T04:31:09.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.380+0000 7f1828c7db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ab95e25510 con 0x55ab95e115f0 2024-06-06T04:31:09.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.380+0000 7f1828c7db80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ab95e25be0 con 0x55ab95e11f10 2024-06-06T04:31:09.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.380+0000 7f1817fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e0bed0 0x55ab95e0b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.380+0000 7f1817fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e0bed0 0x55ab95e0b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55278/0 (socket says 172.21.15.45:55278) 2024-06-06T04:31:09.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.380+0000 7f1817fff700 1 -- 172.21.15.45:0/777604668 learned_addr learned my addr 172.21.15.45:0/777604668 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:09.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.380+0000 7f180ffff700 1 --2- 172.21.15.45:0/777604668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e115f0 0x55ab95e069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.381+0000 7f181c997700 1 --2- 172.21.15.45:0/777604668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e11f10 0x55ab95e0f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.381+0000 7f1817fff700 1 -- 172.21.15.45:0/777604668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e11f10 msgr2=0x55ab95e0f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:09.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.381+0000 7f1817fff700 1 --2- 172.21.15.45:0/777604668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e11f10 0x55ab95e0f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.381+0000 7f1817fff700 1 -- 172.21.15.45:0/777604668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e115f0 msgr2=0x55ab95e069a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:09.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.381+0000 7f1817fff700 1 --2- 172.21.15.45:0/777604668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e115f0 0x55ab95e069a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.381+0000 7f1817fff700 1 -- 172.21.15.45:0/777604668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab95e13570 con 0x55ab95e0bed0 2024-06-06T04:31:09.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.381+0000 7f1817fff700 1 --2- 172.21.15.45:0/777604668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e0bed0 0x55ab95e0b990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f180801e9c0 tx=0x7f180801ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fc836b544178e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:09.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.381+0000 7f180dffb700 1 -- 172.21.15.45:0/777604668 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180801e630 con 0x55ab95e0bed0 2024-06-06T04:31:09.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.381+0000 7f180dffb700 1 -- 172.21.15.45:0/777604668 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1808023070 con 0x55ab95e0bed0 2024-06-06T04:31:09.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.381+0000 7f180dffb700 1 -- 172.21.15.45:0/777604668 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808028a70 con 0x55ab95e0bed0 2024-06-06T04:31:09.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.382+0000 7f1828c7db80 1 -- 172.21.15.45:0/777604668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e0bed0 msgr2=0x55ab95e0b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:09.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.382+0000 7f1828c7db80 1 --2- 172.21.15.45:0/777604668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e0bed0 0x55ab95e0b990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f180801e9c0 tx=0x7f180801ee40 comp rx=0 tx=0).stop 2024-06-06T04:31:09.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.382+0000 7f1828c7db80 1 -- 172.21.15.45:0/777604668 shutdown_connections 2024-06-06T04:31:09.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.382+0000 7f1828c7db80 1 --2- 172.21.15.45:0/777604668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e11f10 0x55ab95e0f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.382+0000 7f1828c7db80 1 --2- 172.21.15.45:0/777604668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e0bed0 0x55ab95e0b990 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.382+0000 7f1828c7db80 1 --2- 172.21.15.45:0/777604668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e115f0 0x55ab95e069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.382+0000 7f1828c7db80 1 -- 172.21.15.45:0/777604668 >> 172.21.15.45:0/777604668 conn(0x55ab95d36c70 msgr2=0x55ab95e0f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:09.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.382+0000 7f1828c7db80 1 -- 172.21.15.45:0/777604668 shutdown_connections 2024-06-06T04:31:09.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.382+0000 7f1828c7db80 1 -- 172.21.15.45:0/777604668 wait complete. 2024-06-06T04:31:09.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.382+0000 7f1828c7db80 1 -- start start 2024-06-06T04:31:09.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.382+0000 7f1828c7db80 1 Processor -- start 2024-06-06T04:31:09.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.382+0000 7f1828c7db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e0bed0 0x55ab95e0ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f1828c7db80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e115f0 0x55ab95e0b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f1828c7db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e11f10 0x55ab95e0b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f1828c7db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ab95e26980 con 0x55ab95e11f10 2024-06-06T04:31:09.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f1828c7db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ab95e262b0 con 0x55ab95e0bed0 2024-06-06T04:31:09.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f1828c7db80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ab95e27050 con 0x55ab95e115f0 2024-06-06T04:31:09.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f1828c7db80 10 client.0 _ll_register_callbacks cb 0x55ab95cf78d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:31:09.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f181c997700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e11f10 0x55ab95e0b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f181c997700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e11f10 0x55ab95e0b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55290/0 (socket says 172.21.15.45:55290) 2024-06-06T04:31:09.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f181c997700 1 -- 172.21.15.45:0/318790993 learned_addr learned my addr 172.21.15.45:0/318790993 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:09.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f1817fff700 1 --2- 172.21.15.45:0/318790993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e0bed0 0x55ab95e0ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f180ffff700 1 --2- 172.21.15.45:0/318790993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e115f0 0x55ab95e0b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f181c997700 1 -- 172.21.15.45:0/318790993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e115f0 msgr2=0x55ab95e0b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:09.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f181c997700 1 --2- 172.21.15.45:0/318790993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e115f0 0x55ab95e0b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f181c997700 1 -- 172.21.15.45:0/318790993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e0bed0 msgr2=0x55ab95e0ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:09.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f181c997700 1 --2- 172.21.15.45:0/318790993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e0bed0 0x55ab95e0ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f181c997700 1 -- 172.21.15.45:0/318790993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab95e94200 con 0x55ab95e11f10 2024-06-06T04:31:09.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f181c997700 1 --2- 172.21.15.45:0/318790993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e11f10 0x55ab95e0b5e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f18100128a0 tx=0x7f1810012bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2b3d943105341c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:09.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f17f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:31:09.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f17f77fe700 1 -- 172.21.15.45:0/318790993 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810016870 con 0x55ab95e11f10 2024-06-06T04:31:09.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f17f77fe700 1 -- 172.21.15.45:0/318790993 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f1810016a10 con 0x55ab95e11f10 2024-06-06T04:31:09.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.383+0000 7f1828c7db80 1 -- 172.21.15.45:0/318790993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1808023070 con 0x55ab95e11f10 2024-06-06T04:31:09.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.384+0000 7f17f77fe700 1 -- 172.21.15.45:0/318790993 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1810027050 con 0x55ab95e11f10 2024-06-06T04:31:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.384+0000 7f1828c7db80 10 client.15186 fetch_fsmap learned FSMap version 26 2024-06-06T04:31:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.384+0000 7f1828c7db80 1 -- 172.21.15.45:0/318790993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ab95e95560 con 0x55ab95e11f10 2024-06-06T04:31:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.384+0000 7f17f77fe700 1 -- 172.21.15.45:0/318790993 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1808023070 con 0x55ab95e11f10 2024-06-06T04:31:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.384+0000 7f17f77fe700 1 -- 172.21.15.45:0/318790993 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f181001e400 con 0x55ab95e11f10 2024-06-06T04:31:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.384+0000 7f1828c7db80 10 client.15186 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T04:31:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.384+0000 7f1828c7db80 10 client.15186 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T04:31:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.384+0000 7f1828c7db80 20 client.15186 populate_metadata read hostname 'smithi045' 2024-06-06T04:31:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.384+0000 7f1828c7db80 1 --2- 172.21.15.45:0/318790993 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55ab95e9a180 0x55ab95e98e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.384+0000 7f1828c7db80 4 client.15186 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T04:31:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.384+0000 7f1828c7db80 1 -- 172.21.15.45:0/318790993 --> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] -- command(tid 0: {"prefix": "status"}) -- 0x55ab95d65240 con 0x55ab95e9a180 2024-06-06T04:31:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.384+0000 7f1817fff700 1 --2- 172.21.15.45:0/318790993 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55ab95e9a180 0x55ab95e98e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f17f77fe700 10 client.15186 ms_handle_connect on v2:172.21.15.154:6838/982197202 2024-06-06T04:31:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f1817fff700 1 --2- 172.21.15.45:0/318790993 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55ab95e9a180 0x55ab95e98e90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=318ae172638378b7 server_cookie=a8e3be48dc36d4e2 in_seq=0 out_seq=0 2024-06-06T04:31:09.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f17f77fe700 1 -- 172.21.15.45:0/318790993 <== mds.1 v2:172.21.15.154:6838/982197202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55ab95d65240 con 0x55ab95e9a180 2024-06-06T04:31:09.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f17f77fe700 10 client.15186 handle_command_reply: tid=0 2024-06-06T04:31:09.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f1828c7db80 1 client.15186 shutdown 2024-06-06T04:31:09.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f1828c7db80 1 -- 172.21.15.45:0/318790993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e11f10 msgr2=0x55ab95e0b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:09.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f1828c7db80 1 --2- 172.21.15.45:0/318790993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e11f10 0x55ab95e0b5e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f18100128a0 tx=0x7f1810012bb0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f1828c7db80 1 -- 172.21.15.45:0/318790993 shutdown_connections 2024-06-06T04:31:09.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f1828c7db80 1 --2- 172.21.15.45:0/318790993 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55ab95e9a180 0x55ab95e98e90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f1828c7db80 1 --2- 172.21.15.45:0/318790993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab95e11f10 0x55ab95e0b5e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f1828c7db80 1 --2- 172.21.15.45:0/318790993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab95e0bed0 0x55ab95e0ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f1828c7db80 1 --2- 172.21.15.45:0/318790993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab95e115f0 0x55ab95e0b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f1828c7db80 1 -- 172.21.15.45:0/318790993 >> 172.21.15.45:0/318790993 conn(0x55ab95d36c70 msgr2=0x55ab95dffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:09.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f17f77fe700 0 client.15186 ms_handle_reset on v2:172.21.15.154:6838/982197202 2024-06-06T04:31:09.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f1828c7db80 1 -- 172.21.15.45:0/318790993 shutdown_connections 2024-06-06T04:31:09.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f1828c7db80 1 -- 172.21.15.45:0/318790993 wait complete. 2024-06-06T04:31:09.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.385+0000 7f1828c7db80 20 client.15186 trim_cache size 0 max 16384 2024-06-06T04:31:09.394 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:31:09.395 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "c4f6b938-23bb-11ef-bc9b-c7b262605968", 2024-06-06T04:31:09.395 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 1, 2024-06-06T04:31:09.395 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24463, 2024-06-06T04:31:09.395 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-06-06T04:31:09.395 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-06-06T04:31:09.395 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-06-06T04:31:09.395 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 162.68217758899999, 2024-06-06T04:31:09.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2024-06-06T04:31:09.395 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 100, 2024-06-06T04:31:09.395 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 98, 2024-06-06T04:31:09.395 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 250.48964222800001 2024-06-06T04:31:09.395 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:31:09.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.386+0000 7f181e3fa700 1 -- 172.21.15.45:0/3852914968 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f18040a2c20 msgr2=0x7f18040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:09.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.386+0000 7f181e3fa700 1 --2- 172.21.15.45:0/3852914968 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f18040a2c20 0x7f18040a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1818199a70 tx=0x7f180800b3c0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.386+0000 7f181e3fa700 1 -- 172.21.15.45:0/3852914968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1818102ee0 msgr2=0x7f181819ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:09.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.386+0000 7f181e3fa700 1 --2- 172.21.15.45:0/3852914968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1818102ee0 0x7f181819ecb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1800006010 tx=0x7f1800009f80 comp rx=0 tx=0).stop 2024-06-06T04:31:09.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.386+0000 7f181e3fa700 1 -- 172.21.15.45:0/3852914968 shutdown_connections 2024-06-06T04:31:09.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.386+0000 7f181e3fa700 1 --2- 172.21.15.45:0/3852914968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18181085c0 0x7f1818198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.386+0000 7f181e3fa700 1 --2- 172.21.15.45:0/3852914968 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f18040a2c20 0x7f18040a50e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.386+0000 7f181e3fa700 1 --2- 172.21.15.45:0/3852914968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1818102520 0x7f181819e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.386+0000 7f181e3fa700 1 --2- 172.21.15.45:0/3852914968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1818102ee0 0x7f181819ecb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:09.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.386+0000 7f181e3fa700 1 -- 172.21.15.45:0/3852914968 >> 172.21.15.45:0/3852914968 conn(0x7f1818079680 msgr2=0x7f18181011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:09.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.386+0000 7f181e3fa700 1 -- 172.21.15.45:0/3852914968 shutdown_connections 2024-06-06T04:31:09.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:09.386+0000 7f181e3fa700 1 -- 172.21.15.45:0/3852914968 wait complete. 2024-06-06T04:31:09.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:09 smithi045 ceph-mon[32299]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:31:10.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-06T04:31:10.314 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:31:10.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:09 smithi086 ceph-mon[36053]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:31:10.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:09 smithi154 ceph-mon[36428]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:31:12.058 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:11 smithi045 ceph-mon[32299]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:31:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.067+0000 7fdf7964c700 1 -- 172.21.15.45:0/147488691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf74101480 msgr2=0x7fdf7410d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.067+0000 7fdf7964c700 1 --2- 172.21.15.45:0/147488691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf74101480 0x7fdf7410d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00c210 tx=0x7fdf6c00c520 comp rx=0 tx=0).stop 2024-06-06T04:31:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.067+0000 7fdf7964c700 1 -- 172.21.15.45:0/147488691 shutdown_connections 2024-06-06T04:31:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.067+0000 7fdf7964c700 1 --2- 172.21.15.45:0/147488691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf74101480 0x7fdf7410d990 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.067+0000 7fdf7964c700 1 --2- 172.21.15.45:0/147488691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf74106ac0 0x7fdf74106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.067+0000 7fdf7964c700 1 --2- 172.21.15.45:0/147488691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf74100ac0 0x7fdf74100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.067+0000 7fdf7964c700 1 -- 172.21.15.45:0/147488691 >> 172.21.15.45:0/147488691 conn(0x7fdf74077a00 msgr2=0x7fdf740fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.067+0000 7fdf7964c700 1 -- 172.21.15.45:0/147488691 shutdown_connections 2024-06-06T04:31:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.067+0000 7fdf7964c700 1 -- 172.21.15.45:0/147488691 wait complete. 2024-06-06T04:31:12.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.068+0000 7fdf7964c700 1 Processor -- start 2024-06-06T04:31:12.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.068+0000 7fdf7964c700 1 -- start start 2024-06-06T04:31:12.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.068+0000 7fdf7964c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf74100ac0 0x7fdf7419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.068+0000 7fdf7964c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf74101480 0x7fdf7419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.068+0000 7fdf7964c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf74106ac0 0x7fdf741988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.068+0000 7fdf7964c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf74110410 con 0x7fdf74100ac0 2024-06-06T04:31:12.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.068+0000 7fdf7964c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf74110290 con 0x7fdf74106ac0 2024-06-06T04:31:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.068+0000 7fdf7964c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdf74110590 con 0x7fdf74101480 2024-06-06T04:31:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.068+0000 7fdf72ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf74100ac0 0x7fdf7419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.068+0000 7fdf72ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf74100ac0 0x7fdf7419e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55302/0 (socket says 172.21.15.45:55302) 2024-06-06T04:31:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.068+0000 7fdf72ffd700 1 -- 172.21.15.45:0/2421445399 learned_addr learned my addr 172.21.15.45:0/2421445399 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.069+0000 7fdf727fc700 1 --2- 172.21.15.45:0/2421445399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf74101480 0x7fdf7419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.069+0000 7fdf737fe700 1 --2- 172.21.15.45:0/2421445399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf74106ac0 0x7fdf741988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.069+0000 7fdf72ffd700 1 -- 172.21.15.45:0/2421445399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf74101480 msgr2=0x7fdf7419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.069+0000 7fdf72ffd700 1 --2- 172.21.15.45:0/2421445399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf74101480 0x7fdf7419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.069+0000 7fdf72ffd700 1 -- 172.21.15.45:0/2421445399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf74106ac0 msgr2=0x7fdf741988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.069+0000 7fdf72ffd700 1 --2- 172.21.15.45:0/2421445399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf74106ac0 0x7fdf741988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.069+0000 7fdf72ffd700 1 -- 172.21.15.45:0/2421445399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf74199050 con 0x7fdf74100ac0 2024-06-06T04:31:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.069+0000 7fdf72ffd700 1 --2- 172.21.15.45:0/2421445399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf74100ac0 0x7fdf7419e6e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400d950 tx=0x7fdf6400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.069+0000 7fdf5bfff700 1 -- 172.21.15.45:0/2421445399 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64016070 con 0x7fdf74100ac0 2024-06-06T04:31:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.069+0000 7fdf5bfff700 1 -- 172.21.15.45:0/2421445399 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf64010460 con 0x7fdf74100ac0 2024-06-06T04:31:12.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.069+0000 7fdf7964c700 1 -- 172.21.15.45:0/2421445399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf74199340 con 0x7fdf74100ac0 2024-06-06T04:31:12.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.069+0000 7fdf5bfff700 1 -- 172.21.15.45:0/2421445399 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6400f620 con 0x7fdf74100ac0 2024-06-06T04:31:12.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.069+0000 7fdf7964c700 1 -- 172.21.15.45:0/2421445399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf741a55b0 con 0x7fdf74100ac0 2024-06-06T04:31:12.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.071+0000 7fdf5bfff700 1 -- 172.21.15.45:0/2421445399 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdf64019070 con 0x7fdf74100ac0 2024-06-06T04:31:12.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.072+0000 7fdf5bfff700 1 --2- 172.21.15.45:0/2421445399 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdf5c0a2c20 0x7fdf5c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.072+0000 7fdf727fc700 1 --2- 172.21.15.45:0/2421445399 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdf5c0a2c20 0x7fdf5c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.072+0000 7fdf5bfff700 1 -- 172.21.15.45:0/2421445399 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fdf640cf010 con 0x7fdf74100ac0 2024-06-06T04:31:12.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.073+0000 7fdf727fc700 1 --2- 172.21.15.45:0/2421445399 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdf5c0a2c20 0x7fdf5c0a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdf68005950 tx=0x7fdf680058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:12.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.073+0000 7fdf83ecfb80 1 Processor -- start 2024-06-06T04:31:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.073+0000 7fdf83ecfb80 1 -- start start 2024-06-06T04:31:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.073+0000 7fdf83ecfb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b42395f0 0x5579b42399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.073+0000 7fdf83ecfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4239f10 0x5579b4233990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.073+0000 7fdf83ecfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b4233ed0 0x5579b42342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.073+0000 7fdf83ecfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5579b41857c0 con 0x5579b4239f10 2024-06-06T04:31:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.073+0000 7fdf83ecfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5579b412d6d0 con 0x5579b4233ed0 2024-06-06T04:31:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.073+0000 7fdf83ecfb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5579b4181780 con 0x5579b42395f0 2024-06-06T04:31:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.073+0000 7fdf727fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4239f10 0x5579b4233990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.073+0000 7fdf727fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4239f10 0x5579b4233990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55304/0 (socket says 172.21.15.45:55304) 2024-06-06T04:31:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.073+0000 7fdf727fc700 1 -- 172.21.15.45:0/3547024867 learned_addr learned my addr 172.21.15.45:0/3547024867 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.073+0000 7fdf72ffd700 1 --2- 172.21.15.45:0/3547024867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b42395f0 0x5579b42399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.074+0000 7fdf737fe700 1 --2- 172.21.15.45:0/3547024867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b4233ed0 0x5579b42342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.074+0000 7fdf727fc700 1 -- 172.21.15.45:0/3547024867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b42395f0 msgr2=0x5579b42399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.074+0000 7fdf727fc700 1 --2- 172.21.15.45:0/3547024867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b42395f0 0x5579b42399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.074+0000 7fdf727fc700 1 -- 172.21.15.45:0/3547024867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b4233ed0 msgr2=0x5579b42342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.074+0000 7fdf727fc700 1 --2- 172.21.15.45:0/3547024867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b4233ed0 0x5579b42342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.074+0000 7fdf727fc700 1 -- 172.21.15.45:0/3547024867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579b4123f00 con 0x5579b4239f10 2024-06-06T04:31:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.074+0000 7fdf727fc700 1 --2- 172.21.15.45:0/3547024867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4239f10 0x5579b4233990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800fd40 tx=0x7fdf68012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77cee2fcfff6d745 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.074+0000 7fdf597fa700 1 -- 172.21.15.45:0/3547024867 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68022070 con 0x5579b4239f10 2024-06-06T04:31:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.074+0000 7fdf597fa700 1 -- 172.21.15.45:0/3547024867 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf68011bb0 con 0x5579b4239f10 2024-06-06T04:31:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.074+0000 7fdf597fa700 1 -- 172.21.15.45:0/3547024867 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6801d610 con 0x5579b4239f10 2024-06-06T04:31:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.075+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/3547024867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4239f10 msgr2=0x5579b4233990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.075+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/3547024867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4239f10 0x5579b4233990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800fd40 tx=0x7fdf68012000 comp rx=0 tx=0).stop 2024-06-06T04:31:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.075+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/3547024867 shutdown_connections 2024-06-06T04:31:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.075+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/3547024867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b4233ed0 0x5579b42342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.075+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/3547024867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b42395f0 0x5579b42399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.075+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/3547024867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4239f10 0x5579b4233990 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.075+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/3547024867 >> 172.21.15.45:0/3547024867 conn(0x5579b415ec70 msgr2=0x5579b422e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.075+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/3547024867 shutdown_connections 2024-06-06T04:31:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.075+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/3547024867 wait complete. 2024-06-06T04:31:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.075+0000 7fdf83ecfb80 1 -- start start 2024-06-06T04:31:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.075+0000 7fdf83ecfb80 1 Processor -- start 2024-06-06T04:31:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf83ecfb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b4233ed0 0x5579b423fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf83ecfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b42395f0 0x5579b423cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf83ecfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b4239f10 0x5579b423d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf83ecfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5579b4240980 con 0x5579b42395f0 2024-06-06T04:31:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf83ecfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5579b4125ba0 con 0x5579b4239f10 2024-06-06T04:31:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf83ecfb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5579b4240b00 con 0x5579b4233ed0 2024-06-06T04:31:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf83ecfb80 10 client.0 _ll_register_callbacks cb 0x5579b411f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:31:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf727fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b42395f0 0x5579b423cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf727fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b42395f0 0x5579b423cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55312/0 (socket says 172.21.15.45:55312) 2024-06-06T04:31:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf727fc700 1 -- 172.21.15.45:0/373861068 learned_addr learned my addr 172.21.15.45:0/373861068 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf72ffd700 1 --2- 172.21.15.45:0/373861068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b4233ed0 0x5579b423fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf737fe700 1 --2- 172.21.15.45:0/373861068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b4239f10 0x5579b423d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf727fc700 1 -- 172.21.15.45:0/373861068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b4233ed0 msgr2=0x5579b423fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf727fc700 1 --2- 172.21.15.45:0/373861068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b4233ed0 0x5579b423fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf727fc700 1 -- 172.21.15.45:0/373861068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b4239f10 msgr2=0x5579b423d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf727fc700 1 --2- 172.21.15.45:0/373861068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b4239f10 0x5579b423d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf727fc700 1 -- 172.21.15.45:0/373861068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579b40d4880 con 0x5579b42395f0 2024-06-06T04:31:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf727fc700 1 --2- 172.21.15.45:0/373861068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b42395f0 0x5579b423cf80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdf68011fc0 tx=0x7fdf6800fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252796bcff94f30b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.076+0000 7fdf56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:31:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.077+0000 7fdf56ff5700 1 -- 172.21.15.45:0/373861068 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68035060 con 0x5579b42395f0 2024-06-06T04:31:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.077+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/373861068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf68011bb0 con 0x5579b42395f0 2024-06-06T04:31:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.077+0000 7fdf56ff5700 1 -- 172.21.15.45:0/373861068 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fdf68005240 con 0x5579b42395f0 2024-06-06T04:31:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.077+0000 7fdf56ff5700 1 -- 172.21.15.45:0/373861068 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdf6803c050 con 0x5579b42395f0 2024-06-06T04:31:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.077+0000 7fdf83ecfb80 10 client.15198 fetch_fsmap learned FSMap version 26 2024-06-06T04:31:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.077+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/373861068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5579b424b280 con 0x5579b42395f0 2024-06-06T04:31:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.077+0000 7fdf56ff5700 1 -- 172.21.15.45:0/373861068 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf68011bb0 con 0x5579b42395f0 2024-06-06T04:31:12.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.077+0000 7fdf56ff5700 1 -- 172.21.15.45:0/373861068 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fdf680049b0 con 0x5579b42395f0 2024-06-06T04:31:12.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.077+0000 7fdf83ecfb80 10 client.15198 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T04:31:12.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.077+0000 7fdf83ecfb80 10 client.15198 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T04:31:12.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.077+0000 7fdf83ecfb80 20 client.15198 populate_metadata read hostname 'smithi045' 2024-06-06T04:31:12.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.077+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/373861068 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5579b424d4c0 0x5579b424e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.077+0000 7fdf83ecfb80 4 client.15198 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:31:12.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.077+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/373861068 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579b4192cc0 con 0x5579b424d4c0 2024-06-06T04:31:12.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.078+0000 7fdf72ffd700 1 --2- 172.21.15.45:0/373861068 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5579b424d4c0 0x5579b424e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.078+0000 7fdf72ffd700 1 --2- 172.21.15.45:0/373861068 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5579b424d4c0 0x5579b424e930 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6af3dad7b3a653b server_cookie=c829d3bec3750e5f in_seq=0 out_seq=0 2024-06-06T04:31:12.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.078+0000 7fdf56ff5700 10 client.15198 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:12.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.079+0000 7fdf56ff5700 1 -- 172.21.15.45:0/373861068 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5579b4192cc0 con 0x5579b424d4c0 2024-06-06T04:31:12.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.079+0000 7fdf56ff5700 10 client.15198 handle_command_reply: tid=0 2024-06-06T04:31:12.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.079+0000 7fdf83ecfb80 1 client.15198 shutdown 2024-06-06T04:31:12.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.079+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/373861068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b42395f0 msgr2=0x5579b423cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:12.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.079+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/373861068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b42395f0 0x5579b423cf80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdf68011fc0 tx=0x7fdf6800fd70 comp rx=0 tx=0).stop 2024-06-06T04:31:12.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.079+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/373861068 shutdown_connections 2024-06-06T04:31:12.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.079+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/373861068 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5579b424d4c0 0x5579b424e930 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.079+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/373861068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b4239f10 0x5579b423d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.079+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/373861068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b4233ed0 0x5579b423fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.079+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/373861068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b42395f0 0x5579b423cf80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.079+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/373861068 >> 172.21.15.45:0/373861068 conn(0x5579b415ec70 msgr2=0x5579b422e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:12.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.079+0000 7fdf56ff5700 0 client.15198 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:12.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.079+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/373861068 shutdown_connections 2024-06-06T04:31:12.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.079+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/373861068 wait complete. 2024-06-06T04:31:12.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.079+0000 7fdf83ecfb80 20 client.15198 trim_cache size 0 max 16384 2024-06-06T04:31:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.101+0000 7fdf83ecfb80 1 Processor -- start 2024-06-06T04:31:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.101+0000 7fdf83ecfb80 1 -- start start 2024-06-06T04:31:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.101+0000 7fdf83ecfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4233ed0 0x5579b4233990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.101+0000 7fdf83ecfb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b4239f10 0x5579b4228ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.101+0000 7fdf83ecfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b423aac0 0x5579b4257d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.101+0000 7fdf83ecfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5579b424cc70 con 0x5579b4233ed0 2024-06-06T04:31:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.101+0000 7fdf83ecfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5579b424d340 con 0x5579b423aac0 2024-06-06T04:31:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.101+0000 7fdf83ecfb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5579b424da10 con 0x5579b4239f10 2024-06-06T04:31:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.101+0000 7fdf72ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4233ed0 0x5579b4233990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.101+0000 7fdf72ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4233ed0 0x5579b4233990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55326/0 (socket says 172.21.15.45:55326) 2024-06-06T04:31:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.101+0000 7fdf72ffd700 1 -- 172.21.15.45:0/496301914 learned_addr learned my addr 172.21.15.45:0/496301914 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.101+0000 7fdf737fe700 1 --2- 172.21.15.45:0/496301914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b423aac0 0x5579b4257d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.101+0000 7fdf727fc700 1 --2- 172.21.15.45:0/496301914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b4239f10 0x5579b4228ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.102+0000 7fdf72ffd700 1 -- 172.21.15.45:0/496301914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b4239f10 msgr2=0x5579b4228ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.102+0000 7fdf72ffd700 1 --2- 172.21.15.45:0/496301914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b4239f10 0x5579b4228ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.102+0000 7fdf72ffd700 1 -- 172.21.15.45:0/496301914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b423aac0 msgr2=0x5579b4257d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.102+0000 7fdf72ffd700 1 --2- 172.21.15.45:0/496301914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b423aac0 0x5579b4257d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.102+0000 7fdf72ffd700 1 -- 172.21.15.45:0/496301914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579b424f890 con 0x5579b4233ed0 2024-06-06T04:31:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.102+0000 7fdf737fe700 1 --2- 172.21.15.45:0/496301914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b423aac0 0x5579b4257d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:31:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.102+0000 7fdf72ffd700 1 --2- 172.21.15.45:0/496301914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4233ed0 0x5579b4233990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400b2f0 tx=0x7fdf640cc620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79a779c9dde7242 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.102+0000 7fdf58ff9700 1 -- 172.21.15.45:0/496301914 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64020040 con 0x5579b4233ed0 2024-06-06T04:31:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.102+0000 7fdf58ff9700 1 -- 172.21.15.45:0/496301914 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf6409f860 con 0x5579b4233ed0 2024-06-06T04:31:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.102+0000 7fdf58ff9700 1 -- 172.21.15.45:0/496301914 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf640997b0 con 0x5579b4233ed0 2024-06-06T04:31:12.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.102+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/496301914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4233ed0 msgr2=0x5579b4233990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:12.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.102+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/496301914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4233ed0 0x5579b4233990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400b2f0 tx=0x7fdf640cc620 comp rx=0 tx=0).stop 2024-06-06T04:31:12.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/496301914 shutdown_connections 2024-06-06T04:31:12.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/496301914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b423aac0 0x5579b4257d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/496301914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4233ed0 0x5579b4233990 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/496301914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b4239f10 0x5579b4228ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/496301914 >> 172.21.15.45:0/496301914 conn(0x5579b415ec70 msgr2=0x5579b423a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/496301914 shutdown_connections 2024-06-06T04:31:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/496301914 wait complete. 2024-06-06T04:31:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf83ecfb80 1 -- start start 2024-06-06T04:31:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf83ecfb80 1 Processor -- start 2024-06-06T04:31:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf83ecfb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b4233ed0 0x5579b4258380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf83ecfb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4239f10 0x5579b4252d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf83ecfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b423aac0 0x5579b42532a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf83ecfb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5579b424e7b0 con 0x5579b4239f10 2024-06-06T04:31:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf83ecfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5579b424e0e0 con 0x5579b423aac0 2024-06-06T04:31:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf83ecfb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5579b424ee80 con 0x5579b4233ed0 2024-06-06T04:31:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf83ecfb80 10 client.0 _ll_register_callbacks cb 0x5579b411f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:31:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf727fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4239f10 0x5579b4252d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf727fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4239f10 0x5579b4252d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55342/0 (socket says 172.21.15.45:55342) 2024-06-06T04:31:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.103+0000 7fdf727fc700 1 -- 172.21.15.45:0/3889334457 learned_addr learned my addr 172.21.15.45:0/3889334457 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf72ffd700 1 --2- 172.21.15.45:0/3889334457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b4233ed0 0x5579b4258380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf737fe700 1 --2- 172.21.15.45:0/3889334457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b423aac0 0x5579b42532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf727fc700 1 -- 172.21.15.45:0/3889334457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b4233ed0 msgr2=0x5579b4258380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf727fc700 1 --2- 172.21.15.45:0/3889334457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b4233ed0 0x5579b4258380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf727fc700 1 -- 172.21.15.45:0/3889334457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b423aac0 msgr2=0x5579b42532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf727fc700 1 --2- 172.21.15.45:0/3889334457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b423aac0 0x5579b42532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf727fc700 1 -- 172.21.15.45:0/3889334457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579b42390f0 con 0x5579b4239f10 2024-06-06T04:31:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf737fe700 1 --2- 172.21.15.45:0/3889334457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b423aac0 0x5579b42532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:31:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf727fc700 1 --2- 172.21.15.45:0/3889334457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4239f10 0x5579b4252d60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdf68004dd0 tx=0x7fdf68011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e236c3956b3db4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:31:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf56ff5700 1 -- 172.21.15.45:0/3889334457 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68015070 con 0x5579b4239f10 2024-06-06T04:31:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf56ff5700 1 -- 172.21.15.45:0/3889334457 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fdf68005450 con 0x5579b4239f10 2024-06-06T04:31:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/3889334457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf6409f860 con 0x5579b4239f10 2024-06-06T04:31:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf56ff5700 1 -- 172.21.15.45:0/3889334457 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdf68035040 con 0x5579b4239f10 2024-06-06T04:31:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf83ecfb80 10 client.15204 fetch_fsmap learned FSMap version 26 2024-06-06T04:31:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/3889334457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5579b42b9a10 con 0x5579b4239f10 2024-06-06T04:31:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.104+0000 7fdf56ff5700 1 -- 172.21.15.45:0/3889334457 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf6409f860 con 0x5579b4239f10 2024-06-06T04:31:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.105+0000 7fdf56ff5700 1 -- 172.21.15.45:0/3889334457 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fdf6801e7c0 con 0x5579b4239f10 2024-06-06T04:31:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.105+0000 7fdf83ecfb80 10 client.15204 fetch_fsmap finished waiting for FSMap version 26 2024-06-06T04:31:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.105+0000 7fdf83ecfb80 10 client.15204 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T04:31:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.105+0000 7fdf83ecfb80 20 client.15204 populate_metadata read hostname 'smithi045' 2024-06-06T04:31:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.105+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/3889334457 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5579b42be590 0x5579b42bd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.105+0000 7fdf83ecfb80 4 client.15204 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T04:31:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.105+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/3889334457 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "status"}) -- 0x5579b418d240 con 0x5579b42be590 2024-06-06T04:31:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.105+0000 7fdf72ffd700 1 --2- 172.21.15.45:0/3889334457 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5579b42be590 0x5579b42bd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.105+0000 7fdf72ffd700 1 --2- 172.21.15.45:0/3889334457 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5579b42be590 0x5579b42bd2a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d435c8ecdd84ac8 server_cookie=dbaf3a322b725da in_seq=0 out_seq=0 2024-06-06T04:31:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.105+0000 7fdf56ff5700 10 client.15204 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf56ff5700 1 -- 172.21.15.45:0/3889334457 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5579b418d240 con 0x5579b42be590 2024-06-06T04:31:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf56ff5700 10 client.15204 handle_command_reply: tid=0 2024-06-06T04:31:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf83ecfb80 1 client.15204 shutdown 2024-06-06T04:31:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/3889334457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4239f10 msgr2=0x5579b4252d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/3889334457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4239f10 0x5579b4252d60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdf68004dd0 tx=0x7fdf68011c30 comp rx=0 tx=0).stop 2024-06-06T04:31:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/3889334457 shutdown_connections 2024-06-06T04:31:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/3889334457 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5579b42be590 0x5579b42bd2a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/3889334457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579b423aac0 0x5579b42532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/3889334457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5579b4233ed0 0x5579b4258380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf56ff5700 0 client.15204 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf83ecfb80 1 --2- 172.21.15.45:0/3889334457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b4239f10 0x5579b4252d60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/3889334457 >> 172.21.15.45:0/3889334457 conn(0x5579b415ec70 msgr2=0x5579b4227db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/3889334457 shutdown_connections 2024-06-06T04:31:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf83ecfb80 1 -- 172.21.15.45:0/3889334457 wait complete. 2024-06-06T04:31:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf83ecfb80 20 client.15204 trim_cache size 0 max 16384 2024-06-06T04:31:12.112 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:31:12.112 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "c4f6b938-23bb-11ef-bc9b-c7b262605968", 2024-06-06T04:31:12.112 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-06-06T04:31:12.112 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24473, 2024-06-06T04:31:12.112 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-06-06T04:31:12.112 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-06-06T04:31:12.113 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-06-06T04:31:12.113 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 227.54256049899999, 2024-06-06T04:31:12.113 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2024-06-06T04:31:12.113 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 100, 2024-06-06T04:31:12.113 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 98, 2024-06-06T04:31:12.113 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 242.780033742 2024-06-06T04:31:12.113 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:31:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf7964c700 1 -- 172.21.15.45:0/2421445399 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdf5c0a2c20 msgr2=0x7fdf5c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf7964c700 1 --2- 172.21.15.45:0/2421445399 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdf5c0a2c20 0x7fdf5c0a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdf68005950 tx=0x7fdf680058e0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf7964c700 1 -- 172.21.15.45:0/2421445399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf74100ac0 msgr2=0x7fdf7419e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.106+0000 7fdf7964c700 1 --2- 172.21.15.45:0/2421445399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf74100ac0 0x7fdf7419e6e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400d950 tx=0x7fdf6400dd10 comp rx=0 tx=0).stop 2024-06-06T04:31:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.107+0000 7fdf7964c700 1 -- 172.21.15.45:0/2421445399 shutdown_connections 2024-06-06T04:31:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.107+0000 7fdf7964c700 1 --2- 172.21.15.45:0/2421445399 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdf5c0a2c20 0x7fdf5c0a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.107+0000 7fdf7964c700 1 --2- 172.21.15.45:0/2421445399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf74106ac0 0x7fdf741988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.107+0000 7fdf7964c700 1 --2- 172.21.15.45:0/2421445399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf74100ac0 0x7fdf7419e6e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.107+0000 7fdf7964c700 1 --2- 172.21.15.45:0/2421445399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf74101480 0x7fdf7419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.107+0000 7fdf7964c700 1 -- 172.21.15.45:0/2421445399 >> 172.21.15.45:0/2421445399 conn(0x7fdf74077a00 msgr2=0x7fdf740ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.107+0000 7fdf7964c700 1 -- 172.21.15.45:0/2421445399 shutdown_connections 2024-06-06T04:31:12.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:12.107+0000 7fdf7964c700 1 -- 172.21.15.45:0/2421445399 wait complete. 2024-06-06T04:31:12.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:11 smithi086 ceph-mon[36053]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:31:12.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:11 smithi154 ceph-mon[36428]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:31:12.794 INFO:teuthology.run_tasks:Running task sequential... 2024-06-06T04:31:12.809 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-06-06T04:31:12.809 INFO:teuthology.task.sequential:In sequential, running task print... 2024-06-06T04:31:12.810 INFO:teuthology.task.print:Enabling mgr modules 2024-06-06T04:31:12.810 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-06-06T04:31:12.819 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-06-06T04:31:12.820 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-06-06T04:31:12.820 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-06-06T04:31:12.820 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-06T04:31:12.820 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-06T04:31:12.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:31:12.820 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-06-06T04:31:12.838 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-06-06T04:31:12.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:31:12.839 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T04:31:12.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:31:12.908 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-06-06T04:31:13.009 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-06-06T04:31:13.009 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-06T04:31:13.261 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:31:14.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:13 smithi045 ceph-mon[32299]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:31:14.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:13 smithi086 ceph-mon[36053]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:31:14.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:13 smithi154 ceph-mon[36428]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:31:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.988+0000 7f4e87a3c700 1 -- 172.21.15.45:0/29163979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e8006abc0 msgr2=0x7f4e8006afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.988+0000 7f4e87a3c700 1 --2- 172.21.15.45:0/29163979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e8006abc0 0x7f4e8006afa0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009990 tx=0x7f4e7c009ca0 comp rx=0 tx=0).stop 2024-06-06T04:31:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.988+0000 7f4e87a3c700 1 -- 172.21.15.45:0/29163979 shutdown_connections 2024-06-06T04:31:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.988+0000 7f4e87a3c700 1 --2- 172.21.15.45:0/29163979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e8010ef70 0x7f4e80111360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.988+0000 7f4e87a3c700 1 --2- 172.21.15.45:0/29163979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e8006abc0 0x7f4e8006afa0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.988+0000 7f4e87a3c700 1 --2- 172.21.15.45:0/29163979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e8006b4e0 0x7f4e8010ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.988+0000 7f4e87a3c700 1 -- 172.21.15.45:0/29163979 >> 172.21.15.45:0/29163979 conn(0x7f4e80078340 msgr2=0x7f4e80078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.988+0000 7f4e87a3c700 1 -- 172.21.15.45:0/29163979 shutdown_connections 2024-06-06T04:31:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.988+0000 7f4e87a3c700 1 -- 172.21.15.45:0/29163979 wait complete. 2024-06-06T04:31:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.989+0000 7f4e87a3c700 1 Processor -- start 2024-06-06T04:31:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.989+0000 7f4e87a3c700 1 -- start start 2024-06-06T04:31:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e87a3c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e8006abc0 0x7f4e8019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e87a3c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e8006b4e0 0x7f4e8019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e87a3c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e8010ef70 0x7f4e8019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e87a3c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4e80114840 con 0x7f4e8010ef70 2024-06-06T04:31:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e87a3c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801146c0 con 0x7f4e8006b4e0 2024-06-06T04:31:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e87a3c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801149c0 con 0x7f4e8006abc0 2024-06-06T04:31:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e85fd9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e8010ef70 0x7f4e8019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:14.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e85fd9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e8010ef70 0x7f4e8019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55360/0 (socket says 172.21.15.45:55360) 2024-06-06T04:31:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e85fd9700 1 -- 172.21.15.45:0/2036536666 learned_addr learned my addr 172.21.15.45:0/2036536666 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e84fd7700 1 --2- 172.21.15.45:0/2036536666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e8006b4e0 0x7f4e8019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e857d8700 1 --2- 172.21.15.45:0/2036536666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e8006abc0 0x7f4e8019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e85fd9700 1 -- 172.21.15.45:0/2036536666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e8006abc0 msgr2=0x7f4e8019ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e85fd9700 1 --2- 172.21.15.45:0/2036536666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e8006abc0 0x7f4e8019ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e85fd9700 1 -- 172.21.15.45:0/2036536666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e8006b4e0 msgr2=0x7f4e8019b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e85fd9700 1 --2- 172.21.15.45:0/2036536666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e8006b4e0 0x7f4e8019b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e85fd9700 1 -- 172.21.15.45:0/2036536666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8019fd40 con 0x7f4e8010ef70 2024-06-06T04:31:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.990+0000 7f4e84fd7700 1 --2- 172.21.15.45:0/2036536666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e8006b4e0 0x7f4e8019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:31:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.991+0000 7f4e85fd9700 1 --2- 172.21.15.45:0/2036536666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e8010ef70 0x7f4e8019f600 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4e780094f0 tx=0x7f4e7800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.991+0000 7f4e767fc700 1 -- 172.21.15.45:0/2036536666 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78004db0 con 0x7f4e8010ef70 2024-06-06T04:31:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.991+0000 7f4e767fc700 1 -- 172.21.15.45:0/2036536666 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e78009e20 con 0x7f4e8010ef70 2024-06-06T04:31:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.991+0000 7f4e767fc700 1 -- 172.21.15.45:0/2036536666 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78015400 con 0x7f4e8010ef70 2024-06-06T04:31:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.991+0000 7f4e87a3c700 1 -- 172.21.15.45:0/2036536666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e801a0030 con 0x7f4e8010ef70 2024-06-06T04:31:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.991+0000 7f4e87a3c700 1 -- 172.21.15.45:0/2036536666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e801a78b0 con 0x7f4e8010ef70 2024-06-06T04:31:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.992+0000 7f4e87a3c700 1 -- 172.21.15.45:0/2036536666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e80069180 con 0x7f4e8010ef70 2024-06-06T04:31:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.993+0000 7f4e767fc700 1 -- 172.21.15.45:0/2036536666 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4e780052d0 con 0x7f4e8010ef70 2024-06-06T04:31:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.994+0000 7f4e767fc700 1 --2- 172.21.15.45:0/2036536666 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4e6c0a2ca0 0x7f4e6c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.994+0000 7f4e857d8700 1 --2- 172.21.15.45:0/2036536666 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4e6c0a2ca0 0x7f4e6c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.994+0000 7f4e767fc700 1 -- 172.21.15.45:0/2036536666 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f4e780ced00 con 0x7f4e8010ef70 2024-06-06T04:31:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.994+0000 7f4e857d8700 1 --2- 172.21.15.45:0/2036536666 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4e6c0a2ca0 0x7f4e6c0a5160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0098e0 tx=0x7f4e7c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:14.999+0000 7f4e767fc700 1 -- 172.21.15.45:0/2036536666 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4e78010070 con 0x7f4e8010ef70 2024-06-06T04:31:15.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:15.254+0000 7f4e87a3c700 1 -- 172.21.15.45:0/2036536666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4e8002dca0 con 0x7f4e8010ef70 2024-06-06T04:31:15.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:15.254+0000 7f4e767fc700 1 -- 172.21.15.45:0/2036536666 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7f4e7809b8a0 con 0x7f4e8010ef70 2024-06-06T04:31:15.255 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-06T04:31:15.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:15.257+0000 7f4e87a3c700 1 -- 172.21.15.45:0/2036536666 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4e6c0a2ca0 msgr2=0x7f4e6c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:15.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:15.257+0000 7f4e87a3c700 1 --2- 172.21.15.45:0/2036536666 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4e6c0a2ca0 0x7f4e6c0a5160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0098e0 tx=0x7f4e7c005960 comp rx=0 tx=0).stop 2024-06-06T04:31:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:15.257+0000 7f4e87a3c700 1 -- 172.21.15.45:0/2036536666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e8010ef70 msgr2=0x7f4e8019f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:15.257+0000 7f4e87a3c700 1 --2- 172.21.15.45:0/2036536666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e8010ef70 0x7f4e8019f600 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4e780094f0 tx=0x7f4e7800d3b0 comp rx=0 tx=0).stop 2024-06-06T04:31:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:15.257+0000 7f4e87a3c700 1 -- 172.21.15.45:0/2036536666 shutdown_connections 2024-06-06T04:31:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:15.257+0000 7f4e87a3c700 1 --2- 172.21.15.45:0/2036536666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e8010ef70 0x7f4e8019f600 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:15.257+0000 7f4e87a3c700 1 --2- 172.21.15.45:0/2036536666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e8006abc0 0x7f4e8019ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:15.257+0000 7f4e87a3c700 1 --2- 172.21.15.45:0/2036536666 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4e6c0a2ca0 0x7f4e6c0a5160 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:15.257+0000 7f4e87a3c700 1 --2- 172.21.15.45:0/2036536666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e8006b4e0 0x7f4e8019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:15.257+0000 7f4e87a3c700 1 -- 172.21.15.45:0/2036536666 >> 172.21.15.45:0/2036536666 conn(0x7f4e80078340 msgr2=0x7f4e801105a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:15.257+0000 7f4e87a3c700 1 -- 172.21.15.45:0/2036536666 shutdown_connections 2024-06-06T04:31:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:15.257+0000 7f4e87a3c700 1 -- 172.21.15.45:0/2036536666 wait complete. 2024-06-06T04:31:16.026 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-06T04:31:16.027 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-06T04:31:16.027 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-06-06T04:31:16.027 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-06T04:31:16.027 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-06T04:31:16.027 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-06T04:31:16.027 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-06T04:31:16.027 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-06T04:31:16.027 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-06-06T04:31:16.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:31:16.028 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-06-06T04:31:16.047 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-06T04:31:16.047 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-06T04:31:16.047 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-06-06T04:31:16.047 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-06-06T04:31:16.047 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-06-06T04:31:16.047 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-06-06T04:31:16.047 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T04:31:16.047 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-06-06T04:31:16.048 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-06-06T04:31:16.048 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T04:31:16.048 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-06-06T04:31:16.048 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-06-06T04:31:16.048 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-06T04:31:16.048 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-06-06T04:31:16.048 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-06-06T04:31:16.048 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-06-06T04:31:16.048 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 40256sec preferred_lft 40256sec 2024-06-06T04:31:16.048 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-06-06T04:31:16.048 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-06-06T04:31:16.048 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T04:31:16.048 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-06-06T04:31:16.048 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-06-06T04:31:16.049 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-06T04:31:16.049 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T04:31:16.049 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-06-06T04:31:16.049 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2024-06-06T04:31:16.049 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2024-06-06T04:31:16.049 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2024-06-06T04:31:16.049 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-06T04:31:16.049 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-06-06T04:31:16.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T04:31:16.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:15 smithi045 ceph-mon[32299]: pgmap v425: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:31:16.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:15 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2036536666' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T04:31:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T04:31:16.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:31:16.202 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-06T04:31:16.228 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-06-06T04:31:16.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:31:16.230 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-06-06T04:31:16.288 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-06-06T04:31:16.288 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-06T04:31:16.288 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-06T04:31:16.288 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-06T04:31:16.288 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-06T04:31:16.288 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T04:31:16.288 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-06-06T04:31:16.288 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-06T04:31:16.288 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-06T04:31:16.288 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-06T04:31:16.289 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-06-06T04:31:16.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:15 smithi086 ceph-mon[36053]: pgmap v425: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:31:16.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:15 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2036536666' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T04:31:16.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T04:31:16.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T04:31:16.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:31:16.419 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-06-06T04:31:16.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:15 smithi154 ceph-mon[36428]: pgmap v425: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:31:16.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:15 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2036536666' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T04:31:16.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:31:16.435 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-06-06T04:31:16.492 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T04:31:16.492 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-06-06T04:31:16.492 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T04:31:16.492 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-06-06T04:31:16.492 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-06-06T04:31:16.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T04:31:16.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T04:31:16.613 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-06-06T04:31:16.613 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T04:31:16.614 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-06-06T04:31:16.614 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-06-06T04:31:16.614 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-06T04:31:16.614 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-06-06T04:31:16.614 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-06-06T04:31:16.614 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-06-06T04:31:16.614 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-06-06T04:31:16.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T04:31:16.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T04:31:16.733 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T04:31:16.733 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-06-06T04:31:16.733 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-06-06T04:31:16.733 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2024-06-06T04:31:16.733 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-06-06T04:31:16.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T04:31:16.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T04:31:16.812 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-06-06T04:31:16.813 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T04:31:16.813 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-06-06T04:31:16.828 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-06-06T04:31:16.829 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T04:31:16.829 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-06-06T04:31:16.884 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2024-06-06T04:31:16.952 DEBUG:teuthology.orchestra.run.smithi045:> cat /proc/self/mounts | awk '{print $2}' 2024-06-06T04:31:17.009 INFO:teuthology.orchestra.run.smithi045.stdout:/proc 2024-06-06T04:31:17.009 INFO:teuthology.orchestra.run.smithi045.stdout:/sys 2024-06-06T04:31:17.009 INFO:teuthology.orchestra.run.smithi045.stdout:/dev 2024-06-06T04:31:17.009 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/security 2024-06-06T04:31:17.009 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/shm 2024-06-06T04:31:17.009 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/pts 2024-06-06T04:31:17.010 INFO:teuthology.orchestra.run.smithi045.stdout:/run 2024-06-06T04:31:17.010 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/cgroup 2024-06-06T04:31:17.010 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/pstore 2024-06-06T04:31:17.010 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/bpf 2024-06-06T04:31:17.010 INFO:teuthology.orchestra.run.smithi045.stdout:/ 2024-06-06T04:31:17.010 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/selinux 2024-06-06T04:31:17.010 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T04:31:17.010 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/debug 2024-06-06T04:31:17.010 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/mqueue 2024-06-06T04:31:17.010 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/hugepages 2024-06-06T04:31:17.010 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/tracing 2024-06-06T04:31:17.010 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-06T04:31:17.010 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/fuse/connections 2024-06-06T04:31:17.011 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/config 2024-06-06T04:31:17.011 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysctl.service 2024-06-06T04:31:17.011 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-06T04:31:17.011 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/nfs/rpc_pipefs 2024-06-06T04:31:17.011 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/1000 2024-06-06T04:31:17.011 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph 2024-06-06T04:31:17.011 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay 2024-06-06T04:31:17.011 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/57e443f29a64c959e8effc38e4275befcb284fbdb507cfd6aaf0c442915f8721/merged 2024-06-06T04:31:17.011 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/74431be50d65222972207a60a4c2f5a70aedb8a539ae8bef8bdc16892ed09952/merged 2024-06-06T04:31:17.011 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/0 2024-06-06T04:31:17.011 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T04:31:17.011 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/10bafd02f689ba63a05ed448439911fdc03cc7f3deaefc5465ce7072fcd2908b/merged 2024-06-06T04:31:17.011 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/5cdfe5a48f1929f19b88e34c68effb122ae479c8f4c4c1597a7975b5bdddc8f3/merged 2024-06-06T04:31:17.012 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/d3d8ff039eb3956da278e78baf060d983274d705302185cc3f75986841f98316/merged 2024-06-06T04:31:17.012 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/38c3e71ee0536db7c152dfeddf84a6291eb6cf535e4dbcb039208eb439e5b8a2/merged 2024-06-06T04:31:17.012 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/544f6f38c488d03f0f92d0f5428e3eaed2d25358f86ece6e89dbdc3ca6b8d4a2/merged 2024-06-06T04:31:17.012 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/f2f3e1307d03e6ba7f53088299078b6765c4e4d3aa7d67e51e32ef148eeab479/merged 2024-06-06T04:31:17.012 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/dde2a0e46c917da425a686247392814eb317a380d3855a146ba84659038f0cf2/merged 2024-06-06T04:31:17.012 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/3208d4a515a126b6e016421971bd1ac3649cb436a579a29fdc1720ff3d16253b/merged 2024-06-06T04:31:17.012 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns 2024-06-06T04:31:17.012 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T04:31:17.012 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T04:31:17.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:31:17.012 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2024-06-06T04:31:17.065 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-06T04:31:17.065 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/d0b2bf13-bd83-48bd-8e31-022ceebb759d --client_fs=cephfs) 2024-06-06T04:31:17.107 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2024-06-06T04:31:17.133 DEBUG:teuthology.orchestra.run.smithi045:> cat /proc/self/mounts | awk '{print $2}' 2024-06-06T04:31:17.190 INFO:teuthology.orchestra.run.smithi045.stdout:/proc 2024-06-06T04:31:17.191 INFO:teuthology.orchestra.run.smithi045.stdout:/sys 2024-06-06T04:31:17.191 INFO:teuthology.orchestra.run.smithi045.stdout:/dev 2024-06-06T04:31:17.191 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/security 2024-06-06T04:31:17.191 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/shm 2024-06-06T04:31:17.191 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/pts 2024-06-06T04:31:17.191 INFO:teuthology.orchestra.run.smithi045.stdout:/run 2024-06-06T04:31:17.191 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/cgroup 2024-06-06T04:31:17.191 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/pstore 2024-06-06T04:31:17.191 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/bpf 2024-06-06T04:31:17.191 INFO:teuthology.orchestra.run.smithi045.stdout:/ 2024-06-06T04:31:17.191 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/selinux 2024-06-06T04:31:17.191 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T04:31:17.192 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/debug 2024-06-06T04:31:17.192 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/mqueue 2024-06-06T04:31:17.192 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/hugepages 2024-06-06T04:31:17.192 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/tracing 2024-06-06T04:31:17.192 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-06T04:31:17.192 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/fuse/connections 2024-06-06T04:31:17.192 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/config 2024-06-06T04:31:17.192 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysctl.service 2024-06-06T04:31:17.192 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-06T04:31:17.192 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/nfs/rpc_pipefs 2024-06-06T04:31:17.192 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/1000 2024-06-06T04:31:17.192 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph 2024-06-06T04:31:17.192 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay 2024-06-06T04:31:17.192 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/57e443f29a64c959e8effc38e4275befcb284fbdb507cfd6aaf0c442915f8721/merged 2024-06-06T04:31:17.193 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/74431be50d65222972207a60a4c2f5a70aedb8a539ae8bef8bdc16892ed09952/merged 2024-06-06T04:31:17.193 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/0 2024-06-06T04:31:17.193 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T04:31:17.193 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/10bafd02f689ba63a05ed448439911fdc03cc7f3deaefc5465ce7072fcd2908b/merged 2024-06-06T04:31:17.193 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/5cdfe5a48f1929f19b88e34c68effb122ae479c8f4c4c1597a7975b5bdddc8f3/merged 2024-06-06T04:31:17.193 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/d3d8ff039eb3956da278e78baf060d983274d705302185cc3f75986841f98316/merged 2024-06-06T04:31:17.193 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/38c3e71ee0536db7c152dfeddf84a6291eb6cf535e4dbcb039208eb439e5b8a2/merged 2024-06-06T04:31:17.193 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/544f6f38c488d03f0f92d0f5428e3eaed2d25358f86ece6e89dbdc3ca6b8d4a2/merged 2024-06-06T04:31:17.193 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/f2f3e1307d03e6ba7f53088299078b6765c4e4d3aa7d67e51e32ef148eeab479/merged 2024-06-06T04:31:17.193 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/dde2a0e46c917da425a686247392814eb317a380d3855a146ba84659038f0cf2/merged 2024-06-06T04:31:17.193 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/3208d4a515a126b6e016421971bd1ac3649cb436a579a29fdc1720ff3d16253b/merged 2024-06-06T04:31:17.193 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns 2024-06-06T04:31:17.193 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T04:31:17.194 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T04:31:17.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:31:17.194 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2024-06-06T04:31:17.195 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:2024-06-06T04:31:17.193+0000 7fd26524a480 -1 init, newargv = 0x55c1d68cb990 newargc=15 2024-06-06T04:31:17.195 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[91397]: starting ceph client 2024-06-06T04:31:17.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[91397]: starting fuse 2024-06-06T04:31:17.211 INFO:teuthology.orchestra.run.smithi045.stdout:107 2024-06-06T04:31:17.211 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [107] 2024-06-06T04:31:17.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo stdin-killer -- python3 -c ' 2024-06-06T04:31:17.211 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2024-06-06T04:31:17.211 DEBUG:teuthology.orchestra.run.smithi045:> import re 2024-06-06T04:31:17.211 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-06-06T04:31:17.211 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2024-06-06T04:31:17.212 DEBUG:teuthology.orchestra.run.smithi045:> 2024-06-06T04:31:17.212 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2024-06-06T04:31:17.212 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-06-06T04:31:17.212 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2024-06-06T04:31:17.212 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-06-06T04:31:17.212 DEBUG:teuthology.orchestra.run.smithi045:> 2024-06-06T04:31:17.212 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2024-06-06T04:31:17.212 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2024-06-06T04:31:17.212 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2024-06-06T04:31:17.212 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2024-06-06T04:31:17.212 DEBUG:teuthology.orchestra.run.smithi045:> 2024-06-06T04:31:17.212 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2024-06-06T04:31:17.212 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-06T04:31:17.212 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-06T04:31:17.212 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-06T04:31:17.212 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2024-06-06T04:31:17.213 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2024-06-06T04:31:17.213 DEBUG:teuthology.orchestra.run.smithi045:> return f 2024-06-06T04:31:17.213 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-06T04:31:17.213 DEBUG:teuthology.orchestra.run.smithi045:> 2024-06-06T04:31:17.213 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.0")) 2024-06-06T04:31:17.213 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-06-06T04:31:17.332 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.0.91397.asok 2024-06-06T04:31:17.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T04:31:17.343 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.91397.asok 2024-06-06T04:31:17.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:31:17.343 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.91397.asok status 2024-06-06T04:31:17.482 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:31:17.482 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2024-06-06T04:31:17.482 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-06T04:31:17.482 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T04:31:17.482 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "0", 2024-06-06T04:31:17.482 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2024-06-06T04:31:17.482 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-06-06T04:31:17.482 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "91397", 2024-06-06T04:31:17.482 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/volumes/qa/sv_0/d0b2bf13-bd83-48bd-8e31-022ceebb759d" 2024-06-06T04:31:17.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:31:17.482 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2024-06-06T04:31:17.483 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2024-06-06T04:31:17.483 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 15216, 2024-06-06T04:31:17.483 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2024-06-06T04:31:17.483 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2024-06-06T04:31:17.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2024-06-06T04:31:17.483 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 15216 2024-06-06T04:31:17.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:31:17.483 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2024-06-06T04:31:17.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-06-06T04:31:17.483 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2024-06-06T04:31:17.483 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1856403291 2024-06-06T04:31:17.483 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:31:17.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:31:17.484 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2024-06-06T04:31:17.484 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-06-06T04:31:17.484 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2024-06-06T04:31:17.484 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1856403291 2024-06-06T04:31:17.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:31:17.484 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.15216 192.168.0.1:0/1856403291", 2024-06-06T04:31:17.484 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/1856403291", 2024-06-06T04:31:17.484 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 5, 2024-06-06T04:31:17.484 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 26, 2024-06-06T04:31:17.484 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 100, 2024-06-06T04:31:17.484 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2024-06-06T04:31:17.484 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2024-06-06T04:31:17.484 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2024-06-06T04:31:17.484 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:31:17.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:31:17.490 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2024-06-06T04:31:17.491 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.0 2024-06-06T04:31:17.507 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2024-06-06T04:31:17.507 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-06-06T04:31:17.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:31:17.507 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-06-06T04:31:17.576 INFO:teuthology.run_tasks:Running task exec... 2024-06-06T04:31:17.586 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T04:31:17.587 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-06-06T04:31:17.587 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-06-06T04:31:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.721+0000 7fd7821b7640 1 -- 172.21.15.45:0/4195143738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd77c101ba0 msgr2=0x7fd77c102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.721+0000 7fd7821b7640 1 --2- 172.21.15.45:0/4195143738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd77c101ba0 0x7fd77c102020 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd764009900 tx=0x7fd76402a410 comp rx=0 tx=0).stop 2024-06-06T04:31:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.721+0000 7fd7821b7640 1 -- 172.21.15.45:0/4195143738 shutdown_connections 2024-06-06T04:31:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.721+0000 7fd7821b7640 1 --2- 172.21.15.45:0/4195143738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd77c102560 0x7fd77c108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.721+0000 7fd7821b7640 1 --2- 172.21.15.45:0/4195143738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd77c101ba0 0x7fd77c102020 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.721+0000 7fd7821b7640 1 --2- 172.21.15.45:0/4195143738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd77c1009b0 0x7fd77c100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.721+0000 7fd7821b7640 1 -- 172.21.15.45:0/4195143738 >> 172.21.15.45:0/4195143738 conn(0x7fd77c0fbf20 msgr2=0x7fd77c0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.721+0000 7fd7821b7640 1 -- 172.21.15.45:0/4195143738 shutdown_connections 2024-06-06T04:31:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.721+0000 7fd7821b7640 1 -- 172.21.15.45:0/4195143738 wait complete. 2024-06-06T04:31:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.722+0000 7fd7821b7640 1 Processor -- start 2024-06-06T04:31:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.722+0000 7fd7821b7640 1 -- start start 2024-06-06T04:31:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.722+0000 7fd7821b7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd77c1009b0 0x7fd77c06ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.722+0000 7fd7821b7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd77c101ba0 0x7fd77c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.722+0000 7fd7821b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd77c102560 0x7fd77c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.722+0000 7fd7821b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c10c470 con 0x7fd77c102560 2024-06-06T04:31:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.722+0000 7fd7821b7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c10c2f0 con 0x7fd77c1009b0 2024-06-06T04:31:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.722+0000 7fd7821b7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c10c5f0 con 0x7fd77c101ba0 2024-06-06T04:31:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.722+0000 7fd77bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd77c102560 0x7fd77c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.722+0000 7fd77bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd77c102560 0x7fd77c19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55370/0 (socket says 172.21.15.45:55370) 2024-06-06T04:31:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.722+0000 7fd77bfff640 1 -- 172.21.15.45:0/3305299658 learned_addr learned my addr 172.21.15.45:0/3305299658 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.722+0000 7fd77b7fe640 1 --2- 172.21.15.45:0/3305299658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd77c1009b0 0x7fd77c06ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.722+0000 7fd77affd640 1 --2- 172.21.15.45:0/3305299658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd77c101ba0 0x7fd77c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.723+0000 7fd77bfff640 1 -- 172.21.15.45:0/3305299658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd77c101ba0 msgr2=0x7fd77c19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.723+0000 7fd77bfff640 1 --2- 172.21.15.45:0/3305299658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd77c101ba0 0x7fd77c19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.723+0000 7fd77bfff640 1 -- 172.21.15.45:0/3305299658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd77c1009b0 msgr2=0x7fd77c06ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.723+0000 7fd77bfff640 1 --2- 172.21.15.45:0/3305299658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd77c1009b0 0x7fd77c06ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.723+0000 7fd77bfff640 1 -- 172.21.15.45:0/3305299658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77c19f830 con 0x7fd77c102560 2024-06-06T04:31:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.723+0000 7fd77bfff640 1 --2- 172.21.15.45:0/3305299658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd77c102560 0x7fd77c19ef20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd77400cad0 tx=0x7fd77400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.723+0000 7fd778ff9640 1 -- 172.21.15.45:0/3305299658 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77400ae20 con 0x7fd77c102560 2024-06-06T04:31:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.723+0000 7fd778ff9640 1 -- 172.21.15.45:0/3305299658 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd774005020 con 0x7fd77c102560 2024-06-06T04:31:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.723+0000 7fd7821b7640 1 -- 172.21.15.45:0/3305299658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77c19fac0 con 0x7fd77c102560 2024-06-06T04:31:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.723+0000 7fd778ff9640 1 -- 172.21.15.45:0/3305299658 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77400f6b0 con 0x7fd77c102560 2024-06-06T04:31:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.723+0000 7fd7821b7640 1 -- 172.21.15.45:0/3305299658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd77c1a0030 con 0x7fd77c102560 2024-06-06T04:31:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.724+0000 7fd7821b7640 1 -- 172.21.15.45:0/3305299658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd77c107cd0 con 0x7fd77c102560 2024-06-06T04:31:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.726+0000 7fd778ff9640 1 -- 172.21.15.45:0/3305299658 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd7740051d0 con 0x7fd77c102560 2024-06-06T04:31:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.727+0000 7fd778ff9640 1 --2- 172.21.15.45:0/3305299658 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd7680a2cb0 0x7fd7680a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.727+0000 7fd778ff9640 1 -- 172.21.15.45:0/3305299658 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fd7740cf720 con 0x7fd77c102560 2024-06-06T04:31:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.727+0000 7fd77b7fe640 1 --2- 172.21.15.45:0/3305299658 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd7680a2cb0 0x7fd7680a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:17.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.727+0000 7fd77b7fe640 1 --2- 172.21.15.45:0/3305299658 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd7680a2cb0 0x7fd7680a5170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd77c101a00 tx=0x7fd76c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:17.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.731+0000 7fd778ff9640 1 -- 172.21.15.45:0/3305299658 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd774014030 con 0x7fd77c102560 2024-06-06T04:31:17.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:17.955+0000 7fd7821b7640 1 -- 172.21.15.45:0/3305299658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fd77c02d380 con 0x7fd77c102560 2024-06-06T04:31:18.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:17 smithi045 ceph-mon[32299]: pgmap v426: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:31:18.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:17 smithi086 ceph-mon[36053]: pgmap v426: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:31:18.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:17 smithi154 ceph-mon[36428]: pgmap v426: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:31:18.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:18.946+0000 7fd778ff9640 1 -- 172.21.15.45:0/3305299658 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7fd77409bc90 con 0x7fd77c102560 2024-06-06T04:31:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:18.949+0000 7fd7821b7640 1 -- 172.21.15.45:0/3305299658 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd7680a2cb0 msgr2=0x7fd7680a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:18.949+0000 7fd7821b7640 1 --2- 172.21.15.45:0/3305299658 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd7680a2cb0 0x7fd7680a5170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd77c101a00 tx=0x7fd76c00b390 comp rx=0 tx=0).stop 2024-06-06T04:31:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:18.949+0000 7fd7821b7640 1 -- 172.21.15.45:0/3305299658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd77c102560 msgr2=0x7fd77c19ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:18.949+0000 7fd7821b7640 1 --2- 172.21.15.45:0/3305299658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd77c102560 0x7fd77c19ef20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd77400cad0 tx=0x7fd77400cf90 comp rx=0 tx=0).stop 2024-06-06T04:31:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:18.949+0000 7fd7821b7640 1 -- 172.21.15.45:0/3305299658 shutdown_connections 2024-06-06T04:31:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:18.949+0000 7fd7821b7640 1 --2- 172.21.15.45:0/3305299658 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd7680a2cb0 0x7fd7680a5170 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:18.949+0000 7fd7821b7640 1 --2- 172.21.15.45:0/3305299658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd77c102560 0x7fd77c19ef20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:18.949+0000 7fd7821b7640 1 --2- 172.21.15.45:0/3305299658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd77c101ba0 0x7fd77c19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:18.949+0000 7fd7821b7640 1 --2- 172.21.15.45:0/3305299658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd77c1009b0 0x7fd77c06ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:18.949+0000 7fd7821b7640 1 -- 172.21.15.45:0/3305299658 >> 172.21.15.45:0/3305299658 conn(0x7fd77c0fbf20 msgr2=0x7fd77c104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:18.949+0000 7fd7821b7640 1 -- 172.21.15.45:0/3305299658 shutdown_connections 2024-06-06T04:31:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:18.949+0000 7fd7821b7640 1 -- 172.21.15.45:0/3305299658 wait complete. 2024-06-06T04:31:18.963 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-06-06T04:31:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.076+0000 7f3391b14640 1 -- 172.21.15.45:0/3594633961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c107790 msgr2=0x7f338c109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.076+0000 7f3391b14640 1 --2- 172.21.15.45:0/3594633961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c107790 0x7f338c109be0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3384009ca0 tx=0x7f338402a8c0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.076+0000 7f3391b14640 1 -- 172.21.15.45:0/3594633961 shutdown_connections 2024-06-06T04:31:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.076+0000 7f3391b14640 1 --2- 172.21.15.45:0/3594633961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f338c104e60 0x7f338c107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.076+0000 7f3391b14640 1 --2- 172.21.15.45:0/3594633961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c107790 0x7f338c109be0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.076+0000 7f3391b14640 1 --2- 172.21.15.45:0/3594633961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c102530 0x7f338c104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.076+0000 7f3391b14640 1 -- 172.21.15.45:0/3594633961 >> 172.21.15.45:0/3594633961 conn(0x7f338c0fbf20 msgr2=0x7f338c0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.076+0000 7f3391b14640 1 -- 172.21.15.45:0/3594633961 shutdown_connections 2024-06-06T04:31:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.076+0000 7f3391b14640 1 -- 172.21.15.45:0/3594633961 wait complete. 2024-06-06T04:31:19.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.077+0000 7f3391b14640 1 Processor -- start 2024-06-06T04:31:19.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.077+0000 7f3391b14640 1 -- start start 2024-06-06T04:31:19.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.077+0000 7f3391b14640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c102530 0x7f338c199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:19.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.077+0000 7f3391b14640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c104e60 0x7f338c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.077+0000 7f3391b14640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f338c107790 0x7f338c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.077+0000 7f3391b14640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f338c10d660 con 0x7f338c102530 2024-06-06T04:31:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.077+0000 7f3391b14640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f338c10d4e0 con 0x7f338c107790 2024-06-06T04:31:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.077+0000 7f3391b14640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f338c10d7e0 con 0x7f338c104e60 2024-06-06T04:31:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.077+0000 7f338b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c102530 0x7f338c199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.077+0000 7f338b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c102530 0x7f338c199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41020/0 (socket says 172.21.15.45:41020) 2024-06-06T04:31:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.077+0000 7f338b7fe640 1 -- 172.21.15.45:0/2689062312 learned_addr learned my addr 172.21.15.45:0/2689062312 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.077+0000 7f338affd640 1 --2- 172.21.15.45:0/2689062312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c104e60 0x7f338c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.077+0000 7f338b7fe640 1 -- 172.21.15.45:0/2689062312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c104e60 msgr2=0x7f338c19a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.077+0000 7f338b7fe640 1 --2- 172.21.15.45:0/2689062312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c104e60 0x7f338c19a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.078+0000 7f338b7fe640 1 -- 172.21.15.45:0/2689062312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f338c107790 msgr2=0x7f338c1a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.078+0000 7f338b7fe640 1 --2- 172.21.15.45:0/2689062312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f338c107790 0x7f338c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.078+0000 7f338b7fe640 1 -- 172.21.15.45:0/2689062312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338c1a1a30 con 0x7f338c102530 2024-06-06T04:31:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.078+0000 7f338affd640 1 --2- 172.21.15.45:0/2689062312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c104e60 0x7f338c19a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:31:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.078+0000 7f338b7fe640 1 --2- 172.21.15.45:0/2689062312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c102530 0x7f338c199e10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f337c00b4a0 tx=0x7f337c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:19.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.078+0000 7f3388ff9640 1 -- 172.21.15.45:0/2689062312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c014070 con 0x7f338c102530 2024-06-06T04:31:19.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.078+0000 7f3391b14640 1 -- 172.21.15.45:0/2689062312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338c1a1d20 con 0x7f338c102530 2024-06-06T04:31:19.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.078+0000 7f3388ff9640 1 -- 172.21.15.45:0/2689062312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f337c010ac0 con 0x7f338c102530 2024-06-06T04:31:19.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.078+0000 7f3388ff9640 1 -- 172.21.15.45:0/2689062312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c017c30 con 0x7f338c102530 2024-06-06T04:31:19.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.078+0000 7f3391b14640 1 -- 172.21.15.45:0/2689062312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f338c1a2240 con 0x7f338c102530 2024-06-06T04:31:19.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.079+0000 7f3391b14640 1 -- 172.21.15.45:0/2689062312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f338c109280 con 0x7f338c102530 2024-06-06T04:31:19.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.080+0000 7f3388ff9640 1 -- 172.21.15.45:0/2689062312 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f337c017070 con 0x7f338c102530 2024-06-06T04:31:19.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.081+0000 7f3388ff9640 1 --2- 172.21.15.45:0/2689062312 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f33740a2c30 0x7f33740a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:19.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.081+0000 7f338affd640 1 --2- 172.21.15.45:0/2689062312 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f33740a2c30 0x7f33740a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:19.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.081+0000 7f3388ff9640 1 -- 172.21.15.45:0/2689062312 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f337c0ce900 con 0x7f338c102530 2024-06-06T04:31:19.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.081+0000 7f338affd640 1 --2- 172.21.15.45:0/2689062312 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f33740a2c30 0x7f33740a50f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3380006fd0 tx=0x7f3380008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:19.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.085+0000 7f3388ff9640 1 -- 172.21.15.45:0/2689062312 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f337c09b440 con 0x7f338c102530 2024-06-06T04:31:19.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:18 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3305299658' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-06T04:31:19.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.296+0000 7f3391b14640 1 -- 172.21.15.45:0/2689062312 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f338c061e80 con 0x7f33740a2c30 2024-06-06T04:31:19.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.313+0000 7f3388ff9640 1 -- 172.21.15.45:0/2689062312 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f338c061e80 con 0x7f33740a2c30 2024-06-06T04:31:19.314 INFO:teuthology.orchestra.run.smithi045.stdout:{} 2024-06-06T04:31:19.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.315+0000 7f3391b14640 1 -- 172.21.15.45:0/2689062312 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f33740a2c30 msgr2=0x7f33740a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:19.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.315+0000 7f3391b14640 1 --2- 172.21.15.45:0/2689062312 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f33740a2c30 0x7f33740a50f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3380006fd0 tx=0x7f3380008040 comp rx=0 tx=0).stop 2024-06-06T04:31:19.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.315+0000 7f3391b14640 1 -- 172.21.15.45:0/2689062312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c102530 msgr2=0x7f338c199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:19.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.315+0000 7f3391b14640 1 --2- 172.21.15.45:0/2689062312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c102530 0x7f338c199e10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f337c00b4a0 tx=0x7f337c00b960 comp rx=0 tx=0).stop 2024-06-06T04:31:19.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.315+0000 7f3391b14640 1 -- 172.21.15.45:0/2689062312 shutdown_connections 2024-06-06T04:31:19.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.315+0000 7f3391b14640 1 --2- 172.21.15.45:0/2689062312 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f33740a2c30 0x7f33740a50f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.315+0000 7f3391b14640 1 --2- 172.21.15.45:0/2689062312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f338c107790 0x7f338c1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.315+0000 7f3391b14640 1 --2- 172.21.15.45:0/2689062312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c104e60 0x7f338c19a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.315+0000 7f3391b14640 1 --2- 172.21.15.45:0/2689062312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f338c102530 0x7f338c199e10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.315+0000 7f3391b14640 1 -- 172.21.15.45:0/2689062312 >> 172.21.15.45:0/2689062312 conn(0x7f338c0fbf20 msgr2=0x7f338c101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:19.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.315+0000 7f3391b14640 1 -- 172.21.15.45:0/2689062312 shutdown_connections 2024-06-06T04:31:19.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.315+0000 7f3391b14640 1 -- 172.21.15.45:0/2689062312 wait complete. 2024-06-06T04:31:19.327 INFO:teuthology.run_tasks:Running task check-counter... 2024-06-06T04:31:19.337 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-06-06T04:31:19.348 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-06T04:31:19.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:18 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3305299658' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-06T04:31:19.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:18 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3305299658' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-06T04:31:19.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.453+0000 7ff311f1d640 1 -- 172.21.15.45:0/167177385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff30c102480 msgr2=0x7ff30c104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.453+0000 7ff311f1d640 1 --2- 172.21.15.45:0/167177385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff30c102480 0x7ff30c104870 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009900 tx=0x7ff2fc02a410 comp rx=0 tx=0).stop 2024-06-06T04:31:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.453+0000 7ff311f1d640 1 -- 172.21.15.45:0/167177385 shutdown_connections 2024-06-06T04:31:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.453+0000 7ff311f1d640 1 --2- 172.21.15.45:0/167177385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c1076e0 0x7ff30c109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.453+0000 7ff311f1d640 1 --2- 172.21.15.45:0/167177385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff30c104db0 0x7ff30c1071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.453+0000 7ff311f1d640 1 --2- 172.21.15.45:0/167177385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff30c102480 0x7ff30c104870 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.453+0000 7ff311f1d640 1 -- 172.21.15.45:0/167177385 >> 172.21.15.45:0/167177385 conn(0x7ff30c0fbe90 msgr2=0x7ff30c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.453+0000 7ff311f1d640 1 -- 172.21.15.45:0/167177385 shutdown_connections 2024-06-06T04:31:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.453+0000 7ff311f1d640 1 -- 172.21.15.45:0/167177385 wait complete. 2024-06-06T04:31:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.453+0000 7ff311f1d640 1 Processor -- start 2024-06-06T04:31:19.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff311f1d640 1 -- start start 2024-06-06T04:31:19.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff311f1d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff30c102480 0x7ff30c101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:19.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff311f1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c104db0 0x7ff30c100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:19.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff311f1d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff30c1076e0 0x7ff30c100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:19.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff311f1d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c10d530 con 0x7ff30c1076e0 2024-06-06T04:31:19.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff311f1d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c10d3b0 con 0x7ff30c104db0 2024-06-06T04:31:19.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff311f1d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c10d6b0 con 0x7ff30c102480 2024-06-06T04:31:19.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff30bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff30c1076e0 0x7ff30c100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:19.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff30bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff30c1076e0 0x7ff30c100970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41054/0 (socket says 172.21.15.45:41054) 2024-06-06T04:31:19.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff30bfff640 1 -- 172.21.15.45:0/1652135374 learned_addr learned my addr 172.21.15.45:0/1652135374 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:19.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff30b7fe640 1 --2- 172.21.15.45:0/1652135374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff30c102480 0x7ff30c101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:19.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff30affd640 1 --2- 172.21.15.45:0/1652135374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c104db0 0x7ff30c100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:19.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff30bfff640 1 -- 172.21.15.45:0/1652135374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff30c102480 msgr2=0x7ff30c101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:19.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff30bfff640 1 --2- 172.21.15.45:0/1652135374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff30c102480 0x7ff30c101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff30bfff640 1 -- 172.21.15.45:0/1652135374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c104db0 msgr2=0x7ff30c100430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:19.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff30bfff640 1 --2- 172.21.15.45:0/1652135374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c104db0 0x7ff30c100430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff30bfff640 1 -- 172.21.15.45:0/1652135374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c101200 con 0x7ff30c1076e0 2024-06-06T04:31:19.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.454+0000 7ff30bfff640 1 --2- 172.21.15.45:0/1652135374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff30c1076e0 0x7ff30c100970 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff30400eaa0 tx=0x7ff30400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:19.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.455+0000 7ff308ff9640 1 -- 172.21.15.45:0/1652135374 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30400cdb0 con 0x7ff30c1076e0 2024-06-06T04:31:19.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.455+0000 7ff308ff9640 1 -- 172.21.15.45:0/1652135374 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff304005020 con 0x7ff30c1076e0 2024-06-06T04:31:19.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.455+0000 7ff311f1d640 1 -- 172.21.15.45:0/1652135374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c1a1dd0 con 0x7ff30c1076e0 2024-06-06T04:31:19.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.455+0000 7ff308ff9640 1 -- 172.21.15.45:0/1652135374 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3040116d0 con 0x7ff30c1076e0 2024-06-06T04:31:19.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.455+0000 7ff311f1d640 1 -- 172.21.15.45:0/1652135374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c1a2230 con 0x7ff30c1076e0 2024-06-06T04:31:19.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.456+0000 7ff311f1d640 1 -- 172.21.15.45:0/1652135374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff30c0fd980 con 0x7ff30c1076e0 2024-06-06T04:31:19.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.457+0000 7ff308ff9640 1 -- 172.21.15.45:0/1652135374 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff3040051d0 con 0x7ff30c1076e0 2024-06-06T04:31:19.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.458+0000 7ff308ff9640 1 --2- 172.21.15.45:0/1652135374 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff2f40a2c10 0x7ff2f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:19.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.458+0000 7ff30b7fe640 1 --2- 172.21.15.45:0/1652135374 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff2f40a2c10 0x7ff2f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:19.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.458+0000 7ff308ff9640 1 -- 172.21.15.45:0/1652135374 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7ff3040cf4b0 con 0x7ff30c1076e0 2024-06-06T04:31:19.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.458+0000 7ff30b7fe640 1 --2- 172.21.15.45:0/1652135374 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff2f40a2c10 0x7ff2f40a50d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc006010 tx=0x7ff2fc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:19.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.462+0000 7ff308ff9640 1 -- 172.21.15.45:0/1652135374 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff304016030 con 0x7ff30c1076e0 2024-06-06T04:31:19.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.649+0000 7ff311f1d640 1 -- 172.21.15.45:0/1652135374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff30c062640 con 0x7ff30c1076e0 2024-06-06T04:31:19.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.651+0000 7ff308ff9640 1 -- 172.21.15.45:0/1652135374 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24789 (secure 0 0 0) 0x7ff30409b9a0 con 0x7ff30c1076e0 2024-06-06T04:31:19.651 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:31:19.652 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":100,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","created":"2024-06-06T04:19:02.626916+0000","modified":"2024-06-06T04:29:05.901370+0000","last_up_change":"2024-06-06T04:26:17.390429+0000","last_in_change":"2024-06-06T04:25:56.938388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T04:22:28.797137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T04:27:17.750625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T04:27:21.486917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T04:27:31.029818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d3cee4a-5025-45b3-b217-082af37e79db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6803","nonce":2111759250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6805","nonce":2111759250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6809","nonce":2111759250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6807","nonce":2111759250}]},"public_addr":"172.21.15.45:6803/2111759250","cluster_addr":"172.21.15.45:6805/2111759250","heartbeat_back_addr":"172.21.15.45:6809/2111759250","heartbeat_front_addr":"172.21.15.45:6807/2111759250","state":["exists","up"]},{"osd":1,"uuid":"75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6801","nonce":3732225287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6803","nonce":3732225287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6807","nonce":3732225287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6805","nonce":3732225287}]},"public_addr":"172.21.15.86:6801/3732225287","cluster_addr":"172.21.15.86:6803/3732225287","heartbeat_back_addr":"172.21.15.86:6807/3732225287","heartbeat_front_addr":"172.21.15.86:6805/3732225287","state":["exists","up"]},{"osd":2,"uuid":"bb20565f-f9ae-47e3-9604-c69f737cff4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6801","nonce":2898783039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6803","nonce":2898783039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6807","nonce":2898783039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6805","nonce":2898783039}]},"public_addr":"172.21.15.154:6801/2898783039","cluster_addr":"172.21.15.154:6803/2898783039","heartbeat_back_addr":"172.21.15.154:6807/2898783039","heartbeat_front_addr":"172.21.15.154:6805/2898783039","state":["exists","up"]},{"osd":3,"uuid":"e1160799-7811-4326-9792-7e6c7d189c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6811","nonce":3436753135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6813","nonce":3436753135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6817","nonce":3436753135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6815","nonce":3436753135}]},"public_addr":"172.21.15.45:6811/3436753135","cluster_addr":"172.21.15.45:6813/3436753135","heartbeat_back_addr":"172.21.15.45:6817/3436753135","heartbeat_front_addr":"172.21.15.45:6815/3436753135","state":["exists","up"]},{"osd":4,"uuid":"6b389bc8-e19a-47f3-83ff-7b782d146d0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6809","nonce":3030235796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6811","nonce":3030235796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6815","nonce":3030235796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6813","nonce":3030235796}]},"public_addr":"172.21.15.86:6809/3030235796","cluster_addr":"172.21.15.86:6811/3030235796","heartbeat_back_addr":"172.21.15.86:6815/3030235796","heartbeat_front_addr":"172.21.15.86:6813/3030235796","state":["exists","up"]},{"osd":5,"uuid":"069de7b2-89bf-4150-bed7-7b8832573dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6809","nonce":2222268646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6811","nonce":2222268646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6815","nonce":2222268646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6813","nonce":2222268646}]},"public_addr":"172.21.15.154:6809/2222268646","cluster_addr":"172.21.15.154:6811/2222268646","heartbeat_back_addr":"172.21.15.154:6815/2222268646","heartbeat_front_addr":"172.21.15.154:6813/2222268646","state":["exists","up"]},{"osd":6,"uuid":"4c70b627-eafa-4e27-ba2a-4d8e1e335962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6819","nonce":384734939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6821","nonce":384734939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6825","nonce":384734939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6823","nonce":384734939}]},"public_addr":"172.21.15.45:6819/384734939","cluster_addr":"172.21.15.45:6821/384734939","heartbeat_back_addr":"172.21.15.45:6825/384734939","heartbeat_front_addr":"172.21.15.45:6823/384734939","state":["exists","up"]},{"osd":7,"uuid":"3b09b875-be52-4be0-b116-7fd9b0c26705","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6817","nonce":1119791005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6819","nonce":1119791005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6823","nonce":1119791005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6821","nonce":1119791005}]},"public_addr":"172.21.15.86:6817/1119791005","cluster_addr":"172.21.15.86:6819/1119791005","heartbeat_back_addr":"172.21.15.86:6823/1119791005","heartbeat_front_addr":"172.21.15.86:6821/1119791005","state":["exists","up"]},{"osd":8,"uuid":"cdf1dd1b-5392-4836-8e65-4c7c5b688d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6817","nonce":3054812496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6819","nonce":3054812496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6823","nonce":3054812496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6821","nonce":3054812496}]},"public_addr":"172.21.15.154:6817/3054812496","cluster_addr":"172.21.15.154:6819/3054812496","heartbeat_back_addr":"172.21.15.154:6823/3054812496","heartbeat_front_addr":"172.21.15.154:6821/3054812496","state":["exists","up"]},{"osd":9,"uuid":"acede92d-2cc1-4967-a357-7e5e201e99d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6827","nonce":1360655510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6829","nonce":1360655510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6833","nonce":1360655510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6831","nonce":1360655510}]},"public_addr":"172.21.15.45:6827/1360655510","cluster_addr":"172.21.15.45:6829/1360655510","heartbeat_back_addr":"172.21.15.45:6833/1360655510","heartbeat_front_addr":"172.21.15.45:6831/1360655510","state":["exists","up"]},{"osd":10,"uuid":"a2d0c488-1892-4da4-adbf-952c22df0ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6825","nonce":1701691877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6827","nonce":1701691877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6831","nonce":1701691877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6829","nonce":1701691877}]},"public_addr":"172.21.15.86:6825/1701691877","cluster_addr":"172.21.15.86:6827/1701691877","heartbeat_back_addr":"172.21.15.86:6831/1701691877","heartbeat_front_addr":"172.21.15.86:6829/1701691877","state":["exists","up"]},{"osd":11,"uuid":"6b349c4c-b672-480b-ac6a-8594dab0eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6825","nonce":2065648007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6827","nonce":2065648007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6831","nonce":2065648007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6829","nonce":2065648007}]},"public_addr":"172.21.15.154:6825/2065648007","cluster_addr":"172.21.15.154:6827/2065648007","heartbeat_back_addr":"172.21.15.154:6831/2065648007","heartbeat_front_addr":"172.21.15.154:6829/2065648007","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:21:37.147394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:00.776617+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:24.786849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:50.487455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:14.869272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:39.815158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:05.972619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:31.410063+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:56.795119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:23.114884+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:49.256504+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:26:15.251967+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/36732849":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/1500318775":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6801/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6800/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/2041380391":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/2504003341":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/3798552599":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/498952822":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6800/1562511056":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6801/1562511056":"2024-06-07T04:19:37.026199+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T04:31:19.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.653+0000 7ff311f1d640 1 -- 172.21.15.45:0/1652135374 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff2f40a2c10 msgr2=0x7ff2f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:19.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.653+0000 7ff311f1d640 1 --2- 172.21.15.45:0/1652135374 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff2f40a2c10 0x7ff2f40a50d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc006010 tx=0x7ff2fc000bc0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.653+0000 7ff311f1d640 1 -- 172.21.15.45:0/1652135374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff30c1076e0 msgr2=0x7ff30c100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:19.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.653+0000 7ff311f1d640 1 --2- 172.21.15.45:0/1652135374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff30c1076e0 0x7ff30c100970 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff30400eaa0 tx=0x7ff30400ef60 comp rx=0 tx=0).stop 2024-06-06T04:31:19.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.653+0000 7ff311f1d640 1 -- 172.21.15.45:0/1652135374 shutdown_connections 2024-06-06T04:31:19.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.653+0000 7ff311f1d640 1 --2- 172.21.15.45:0/1652135374 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff2f40a2c10 0x7ff2f40a50d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.653+0000 7ff311f1d640 1 --2- 172.21.15.45:0/1652135374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff30c104db0 0x7ff30c100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.653+0000 7ff311f1d640 1 --2- 172.21.15.45:0/1652135374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff30c1076e0 0x7ff30c100970 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.653+0000 7ff311f1d640 1 --2- 172.21.15.45:0/1652135374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff30c102480 0x7ff30c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.653+0000 7ff311f1d640 1 -- 172.21.15.45:0/1652135374 >> 172.21.15.45:0/1652135374 conn(0x7ff30c0fbe90 msgr2=0x7ff30c104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:19.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.653+0000 7ff311f1d640 1 -- 172.21.15.45:0/1652135374 shutdown_connections 2024-06-06T04:31:19.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.653+0000 7ff311f1d640 1 -- 172.21.15.45:0/1652135374 wait complete. 2024-06-06T04:31:19.663 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:31:19 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:31:19.301+0000 7f120b098700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T04:31:19.663 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:31:19 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:31:19.301+0000 7f120b098700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T04:31:19.664 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:31:19 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:31:19.301+0000 7f120b098700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T04:31:19.664 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:31:19 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:31:19.301+0000 7f120b098700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T04:31:19.664 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:31:19 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:31:19.301+0000 7f120b098700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T04:31:19.665 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-06T04:22:28.797137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-06T04:27:17.750625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-06T04:27:21.486917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-06T04:27:31.029818+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-06T04:31:19.665 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-06T04:31:19.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.772+0000 7fbb931ce640 1 -- 172.21.15.45:0/1422565084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb8c101af0 msgr2=0x7fbb8c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:19.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.772+0000 7fbb931ce640 1 --2- 172.21.15.45:0/1422565084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb8c101af0 0x7fbb8c101f70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009900 tx=0x7fbb7402a410 comp rx=0 tx=0).stop 2024-06-06T04:31:19.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.773+0000 7fbb931ce640 1 -- 172.21.15.45:0/1422565084 shutdown_connections 2024-06-06T04:31:19.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.773+0000 7fbb931ce640 1 --2- 172.21.15.45:0/1422565084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb8c1024b0 0x7fbb8c108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.773+0000 7fbb931ce640 1 --2- 172.21.15.45:0/1422565084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb8c101af0 0x7fbb8c101f70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.773+0000 7fbb931ce640 1 --2- 172.21.15.45:0/1422565084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb8c100900 0x7fbb8c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.773+0000 7fbb931ce640 1 -- 172.21.15.45:0/1422565084 >> 172.21.15.45:0/1422565084 conn(0x7fbb8c0fbe90 msgr2=0x7fbb8c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.773+0000 7fbb931ce640 1 -- 172.21.15.45:0/1422565084 shutdown_connections 2024-06-06T04:31:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.773+0000 7fbb931ce640 1 -- 172.21.15.45:0/1422565084 wait complete. 2024-06-06T04:31:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.773+0000 7fbb931ce640 1 Processor -- start 2024-06-06T04:31:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.773+0000 7fbb931ce640 1 -- start start 2024-06-06T04:31:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.773+0000 7fbb931ce640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb8c100900 0x7fbb8c199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.773+0000 7fbb931ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb8c101af0 0x7fbb8c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.773+0000 7fbb931ce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb8c1024b0 0x7fbb8c1a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.773+0000 7fbb931ce640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8c10c490 con 0x7fbb8c101af0 2024-06-06T04:31:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.773+0000 7fbb931ce640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8c10c310 con 0x7fbb8c1024b0 2024-06-06T04:31:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.773+0000 7fbb931ce640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8c10c610 con 0x7fbb8c100900 2024-06-06T04:31:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.773+0000 7fbb8bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb8c101af0 0x7fbb8c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:19.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb8bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb8c101af0 0x7fbb8c19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41080/0 (socket says 172.21.15.45:41080) 2024-06-06T04:31:19.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb8bfff640 1 -- 172.21.15.45:0/2277943285 learned_addr learned my addr 172.21.15.45:0/2277943285 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:19.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb90f6a640 1 --2- 172.21.15.45:0/2277943285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb8c100900 0x7fbb8c199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:19.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb9176b640 1 --2- 172.21.15.45:0/2277943285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb8c1024b0 0x7fbb8c1a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:19.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb8bfff640 1 -- 172.21.15.45:0/2277943285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb8c100900 msgr2=0x7fbb8c199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:19.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb8bfff640 1 --2- 172.21.15.45:0/2277943285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb8c100900 0x7fbb8c199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb8bfff640 1 -- 172.21.15.45:0/2277943285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb8c1024b0 msgr2=0x7fbb8c1a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:19.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb8bfff640 1 --2- 172.21.15.45:0/2277943285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb8c1024b0 0x7fbb8c1a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb8bfff640 1 -- 172.21.15.45:0/2277943285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8c1a1ae0 con 0x7fbb8c101af0 2024-06-06T04:31:19.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb90f6a640 1 --2- 172.21.15.45:0/2277943285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb8c100900 0x7fbb8c199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:31:19.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb9176b640 1 --2- 172.21.15.45:0/2277943285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb8c1024b0 0x7fbb8c1a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:31:19.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb8bfff640 1 --2- 172.21.15.45:0/2277943285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb8c101af0 0x7fbb8c19a400 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005940 tx=0x7fbb74004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:19.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb89ffb640 1 -- 172.21.15.45:0/2277943285 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7403f030 con 0x7fbb8c101af0 2024-06-06T04:31:19.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb931ce640 1 -- 172.21.15.45:0/2277943285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb8c1a1d70 con 0x7fbb8c101af0 2024-06-06T04:31:19.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb89ffb640 1 -- 172.21.15.45:0/2277943285 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb74003cb0 con 0x7fbb8c101af0 2024-06-06T04:31:19.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb89ffb640 1 -- 172.21.15.45:0/2277943285 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74035070 con 0x7fbb8c101af0 2024-06-06T04:31:19.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.774+0000 7fbb931ce640 1 -- 172.21.15.45:0/2277943285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb8c1a2280 con 0x7fbb8c101af0 2024-06-06T04:31:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.775+0000 7fbb931ce640 1 -- 172.21.15.45:0/2277943285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb8c1a24b0 con 0x7fbb8c101af0 2024-06-06T04:31:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.776+0000 7fbb89ffb640 1 -- 172.21.15.45:0/2277943285 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbb74042030 con 0x7fbb8c101af0 2024-06-06T04:31:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.777+0000 7fbb89ffb640 1 --2- 172.21.15.45:0/2277943285 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbb780a2b90 0x7fbb780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.777+0000 7fbb89ffb640 1 -- 172.21.15.45:0/2277943285 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fbb740eff60 con 0x7fbb8c101af0 2024-06-06T04:31:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.777+0000 7fbb90f6a640 1 --2- 172.21.15.45:0/2277943285 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbb780a2b90 0x7fbb780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.778+0000 7fbb90f6a640 1 --2- 172.21.15.45:0/2277943285 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbb780a2b90 0x7fbb780a5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c101950 tx=0x7fbb7c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:19.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.782+0000 7fbb89ffb640 1 -- 172.21.15.45:0/2277943285 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbb740bcb00 con 0x7fbb8c101af0 2024-06-06T04:31:19.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.966+0000 7fbb931ce640 1 -- 172.21.15.45:0/2277943285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbb8c1a2680 con 0x7fbb8c101af0 2024-06-06T04:31:19.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.967+0000 7fbb89ffb640 1 -- 172.21.15.45:0/2277943285 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7fbb740bc400 con 0x7fbb8c101af0 2024-06-06T04:31:19.967 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-06-06T04:31:19.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.969+0000 7fbb931ce640 1 -- 172.21.15.45:0/2277943285 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbb780a2b90 msgr2=0x7fbb780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:19.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.969+0000 7fbb931ce640 1 --2- 172.21.15.45:0/2277943285 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbb780a2b90 0x7fbb780a5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c101950 tx=0x7fbb7c00a560 comp rx=0 tx=0).stop 2024-06-06T04:31:19.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.969+0000 7fbb931ce640 1 -- 172.21.15.45:0/2277943285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb8c101af0 msgr2=0x7fbb8c19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:19.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.969+0000 7fbb931ce640 1 --2- 172.21.15.45:0/2277943285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb8c101af0 0x7fbb8c19a400 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005940 tx=0x7fbb74004430 comp rx=0 tx=0).stop 2024-06-06T04:31:19.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.969+0000 7fbb931ce640 1 -- 172.21.15.45:0/2277943285 shutdown_connections 2024-06-06T04:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.969+0000 7fbb931ce640 1 --2- 172.21.15.45:0/2277943285 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbb780a2b90 0x7fbb780a5050 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.969+0000 7fbb931ce640 1 --2- 172.21.15.45:0/2277943285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb8c1024b0 0x7fbb8c1a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.969+0000 7fbb931ce640 1 --2- 172.21.15.45:0/2277943285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb8c101af0 0x7fbb8c19a400 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.969+0000 7fbb931ce640 1 --2- 172.21.15.45:0/2277943285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb8c100900 0x7fbb8c199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.969+0000 7fbb931ce640 1 -- 172.21.15.45:0/2277943285 >> 172.21.15.45:0/2277943285 conn(0x7fbb8c0fbe90 msgr2=0x7fbb8c069fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.969+0000 7fbb931ce640 1 -- 172.21.15.45:0/2277943285 shutdown_connections 2024-06-06T04:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:19.969+0000 7fbb931ce640 1 -- 172.21.15.45:0/2277943285 wait complete. 2024-06-06T04:31:19.980 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-06T04:31:20.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.087+0000 7f68d1229640 1 -- 172.21.15.45:0/979038515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68cc104db0 msgr2=0x7f68cc1071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:20.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.087+0000 7f68d1229640 1 --2- 172.21.15.45:0/979038515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68cc104db0 0x7f68cc1071a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009900 tx=0x7f68c002a410 comp rx=0 tx=0).stop 2024-06-06T04:31:20.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.087+0000 7f68d1229640 1 -- 172.21.15.45:0/979038515 shutdown_connections 2024-06-06T04:31:20.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.087+0000 7f68d1229640 1 --2- 172.21.15.45:0/979038515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc1076e0 0x7f68cc109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.087+0000 7f68d1229640 1 --2- 172.21.15.45:0/979038515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68cc104db0 0x7f68cc1071a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.087+0000 7f68d1229640 1 --2- 172.21.15.45:0/979038515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68cc102480 0x7f68cc104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.087+0000 7f68d1229640 1 -- 172.21.15.45:0/979038515 >> 172.21.15.45:0/979038515 conn(0x7f68cc0fbe70 msgr2=0x7f68cc0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:20.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.087+0000 7f68d1229640 1 -- 172.21.15.45:0/979038515 shutdown_connections 2024-06-06T04:31:20.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.087+0000 7f68d1229640 1 -- 172.21.15.45:0/979038515 wait complete. 2024-06-06T04:31:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.088+0000 7f68d1229640 1 Processor -- start 2024-06-06T04:31:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.088+0000 7f68d1229640 1 -- start start 2024-06-06T04:31:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.088+0000 7f68d1229640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68cc102480 0x7f68cc197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.088+0000 7f68d1229640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc104db0 0x7f68cc198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.088+0000 7f68d1229640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68cc1076e0 0x7f68cc19f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.088+0000 7f68d1229640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc10d530 con 0x7f68cc102480 2024-06-06T04:31:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.088+0000 7f68d1229640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc10d3b0 con 0x7f68cc1076e0 2024-06-06T04:31:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.088+0000 7f68d1229640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc10d6b0 con 0x7f68cc104db0 2024-06-06T04:31:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.089+0000 7f68cad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68cc102480 0x7f68cc197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.089+0000 7f68cad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68cc102480 0x7f68cc197cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41090/0 (socket says 172.21.15.45:41090) 2024-06-06T04:31:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.089+0000 7f68cad9d640 1 -- 172.21.15.45:0/25487736 learned_addr learned my addr 172.21.15.45:0/25487736 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.089+0000 7f68cb59e640 1 --2- 172.21.15.45:0/25487736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68cc1076e0 0x7f68cc19f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.089+0000 7f68ca59c640 1 --2- 172.21.15.45:0/25487736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc104db0 0x7f68cc198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.089+0000 7f68cad9d640 1 -- 172.21.15.45:0/25487736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc104db0 msgr2=0x7f68cc198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.089+0000 7f68cad9d640 1 --2- 172.21.15.45:0/25487736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc104db0 0x7f68cc198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.089+0000 7f68cad9d640 1 -- 172.21.15.45:0/25487736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68cc1076e0 msgr2=0x7f68cc19f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.089+0000 7f68cad9d640 1 --2- 172.21.15.45:0/25487736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68cc1076e0 0x7f68cc19f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.089+0000 7f68cad9d640 1 -- 172.21.15.45:0/25487736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68cc19f8c0 con 0x7f68cc102480 2024-06-06T04:31:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.089+0000 7f68cad9d640 1 --2- 172.21.15.45:0/25487736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68cc102480 0x7f68cc197cd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f68bc00c890 tx=0x7f68bc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.089+0000 7f68b3fff640 1 -- 172.21.15.45:0/25487736 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc004220 con 0x7f68cc102480 2024-06-06T04:31:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.089+0000 7f68b3fff640 1 -- 172.21.15.45:0/25487736 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68bc0043c0 con 0x7f68cc102480 2024-06-06T04:31:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.089+0000 7f68d1229640 1 -- 172.21.15.45:0/25487736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68cc19fbb0 con 0x7f68cc102480 2024-06-06T04:31:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.089+0000 7f68b3fff640 1 -- 172.21.15.45:0/25487736 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc004a00 con 0x7f68cc102480 2024-06-06T04:31:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.089+0000 7f68d1229640 1 -- 172.21.15.45:0/25487736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68cc1a00d0 con 0x7f68cc102480 2024-06-06T04:31:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.090+0000 7f68d1229640 1 -- 172.21.15.45:0/25487736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68cc0fd960 con 0x7f68cc102480 2024-06-06T04:31:20.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.092+0000 7f68b3fff640 1 -- 172.21.15.45:0/25487736 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f68bc004ba0 con 0x7f68cc102480 2024-06-06T04:31:20.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.092+0000 7f68b3fff640 1 --2- 172.21.15.45:0/25487736 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f68b40a2b90 0x7f68b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:20.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.092+0000 7f68ca59c640 1 --2- 172.21.15.45:0/25487736 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f68b40a2b90 0x7f68b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:20.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.092+0000 7f68b3fff640 1 -- 172.21.15.45:0/25487736 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f68bc0cf010 con 0x7f68cc102480 2024-06-06T04:31:20.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.093+0000 7f68ca59c640 1 --2- 172.21.15.45:0/25487736 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f68b40a2b90 0x7f68b40a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f68c0004770 tx=0x7f68c0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.097+0000 7f68b3fff640 1 -- 172.21.15.45:0/25487736 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f68bc014030 con 0x7f68cc102480 2024-06-06T04:31:20.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:19 smithi045 ceph-mon[32299]: pgmap v427: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T04:31:20.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:19 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3305299658' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-06T04:31:20.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:19 smithi045 ceph-mon[32299]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:31:20.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:19 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1652135374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:31:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.283+0000 7f68d1229640 1 -- 172.21.15.45:0/25487736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f68cc062640 con 0x7f68cc102480 2024-06-06T04:31:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.283+0000 7f68b3fff640 1 -- 172.21.15.45:0/25487736 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f68bc09b560 con 0x7f68cc102480 2024-06-06T04:31:20.284 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-06-06T04:31:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.285+0000 7f68d1229640 1 -- 172.21.15.45:0/25487736 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f68b40a2b90 msgr2=0x7f68b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.285+0000 7f68d1229640 1 --2- 172.21.15.45:0/25487736 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f68b40a2b90 0x7f68b40a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f68c0004770 tx=0x7f68c0000bc0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.285+0000 7f68d1229640 1 -- 172.21.15.45:0/25487736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68cc102480 msgr2=0x7f68cc197cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.285+0000 7f68d1229640 1 --2- 172.21.15.45:0/25487736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68cc102480 0x7f68cc197cd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f68bc00c890 tx=0x7f68bc00cd50 comp rx=0 tx=0).stop 2024-06-06T04:31:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.285+0000 7f68d1229640 1 -- 172.21.15.45:0/25487736 shutdown_connections 2024-06-06T04:31:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.285+0000 7f68d1229640 1 --2- 172.21.15.45:0/25487736 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f68b40a2b90 0x7f68b40a5050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.285+0000 7f68d1229640 1 --2- 172.21.15.45:0/25487736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68cc1076e0 0x7f68cc19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.285+0000 7f68d1229640 1 --2- 172.21.15.45:0/25487736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc104db0 0x7f68cc198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.285+0000 7f68d1229640 1 --2- 172.21.15.45:0/25487736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68cc102480 0x7f68cc197cd0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.285+0000 7f68d1229640 1 -- 172.21.15.45:0/25487736 >> 172.21.15.45:0/25487736 conn(0x7f68cc0fbe70 msgr2=0x7f68cc102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.286+0000 7f68d1229640 1 -- 172.21.15.45:0/25487736 shutdown_connections 2024-06-06T04:31:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.286+0000 7f68d1229640 1 -- 172.21.15.45:0/25487736 wait complete. 2024-06-06T04:31:20.297 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-06T04:31:20.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:19 smithi086 ceph-mon[36053]: pgmap v427: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T04:31:20.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:19 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3305299658' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-06T04:31:20.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:19 smithi086 ceph-mon[36053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:31:20.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:19 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1652135374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:31:20.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.405+0000 7f4b57061640 1 -- 172.21.15.45:0/941448113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b501024b0 msgr2=0x7f4b50108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:20.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.405+0000 7f4b57061640 1 --2- 172.21.15.45:0/941448113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b501024b0 0x7f4b50108d40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009ca0 tx=0x7f4b4802a8c0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.405+0000 7f4b57061640 1 -- 172.21.15.45:0/941448113 shutdown_connections 2024-06-06T04:31:20.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.405+0000 7f4b57061640 1 --2- 172.21.15.45:0/941448113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b501024b0 0x7f4b50108d40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.405+0000 7f4b57061640 1 --2- 172.21.15.45:0/941448113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b50101af0 0x7f4b50101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.405+0000 7f4b57061640 1 --2- 172.21.15.45:0/941448113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b50100900 0x7f4b50100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.405+0000 7f4b57061640 1 -- 172.21.15.45:0/941448113 >> 172.21.15.45:0/941448113 conn(0x7f4b500fbe90 msgr2=0x7f4b500fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:20.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.405+0000 7f4b57061640 1 -- 172.21.15.45:0/941448113 shutdown_connections 2024-06-06T04:31:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.405+0000 7f4b57061640 1 -- 172.21.15.45:0/941448113 wait complete. 2024-06-06T04:31:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b57061640 1 Processor -- start 2024-06-06T04:31:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b57061640 1 -- start start 2024-06-06T04:31:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b57061640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b50100900 0x7f4b5006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b57061640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b50101af0 0x7f4b5019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b57061640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b501024b0 0x7f4b5019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b57061640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5010c430 con 0x7f4b50101af0 2024-06-06T04:31:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b57061640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5010c2b0 con 0x7f4b501024b0 2024-06-06T04:31:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b57061640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5010c5b0 con 0x7f4b50100900 2024-06-06T04:31:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b4ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b50101af0 0x7f4b5019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b4ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b50101af0 0x7f4b5019c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41112/0 (socket says 172.21.15.45:41112) 2024-06-06T04:31:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b4ffff640 1 -- 172.21.15.45:0/3531421738 learned_addr learned my addr 172.21.15.45:0/3531421738 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b54dfd640 1 --2- 172.21.15.45:0/3531421738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b50100900 0x7f4b5006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b555fe640 1 --2- 172.21.15.45:0/3531421738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b501024b0 0x7f4b5019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b4ffff640 1 -- 172.21.15.45:0/3531421738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b50100900 msgr2=0x7f4b5006ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b4ffff640 1 --2- 172.21.15.45:0/3531421738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b50100900 0x7f4b5006ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b4ffff640 1 -- 172.21.15.45:0/3531421738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b501024b0 msgr2=0x7f4b5019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b4ffff640 1 --2- 172.21.15.45:0/3531421738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b501024b0 0x7f4b5019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b4ffff640 1 -- 172.21.15.45:0/3531421738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5019f880 con 0x7f4b50101af0 2024-06-06T04:31:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.406+0000 7f4b4ffff640 1 --2- 172.21.15.45:0/3531421738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b50101af0 0x7f4b5019c950 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000e9f0 tx=0x7f4b4000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.407+0000 7f4b4dffb640 1 -- 172.21.15.45:0/3531421738 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4000cd80 con 0x7f4b50101af0 2024-06-06T04:31:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.407+0000 7f4b4dffb640 1 -- 172.21.15.45:0/3531421738 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b400054d0 con 0x7f4b50101af0 2024-06-06T04:31:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.407+0000 7f4b4dffb640 1 -- 172.21.15.45:0/3531421738 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40010490 con 0x7f4b50101af0 2024-06-06T04:31:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.407+0000 7f4b57061640 1 -- 172.21.15.45:0/3531421738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5019fb10 con 0x7f4b50101af0 2024-06-06T04:31:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.407+0000 7f4b57061640 1 -- 172.21.15.45:0/3531421738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b501a0060 con 0x7f4b50101af0 2024-06-06T04:31:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.408+0000 7f4b57061640 1 -- 172.21.15.45:0/3531421738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b50107ca0 con 0x7f4b50101af0 2024-06-06T04:31:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.409+0000 7f4b4dffb640 1 -- 172.21.15.45:0/3531421738 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4b40005020 con 0x7f4b50101af0 2024-06-06T04:31:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.410+0000 7f4b4dffb640 1 --2- 172.21.15.45:0/3531421738 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4b3c0a2c10 0x7f4b3c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.410+0000 7f4b4dffb640 1 -- 172.21.15.45:0/3531421738 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f4b400cf830 con 0x7f4b50101af0 2024-06-06T04:31:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.410+0000 7f4b54dfd640 1 --2- 172.21.15.45:0/3531421738 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4b3c0a2c10 0x7f4b3c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.410+0000 7f4b54dfd640 1 --2- 172.21.15.45:0/3531421738 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4b3c0a2c10 0x7f4b3c0a50d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4b38005fd0 tx=0x7f4b38005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:20.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.414+0000 7f4b4dffb640 1 -- 172.21.15.45:0/3531421738 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4b4009c3d0 con 0x7f4b50101af0 2024-06-06T04:31:20.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:19 smithi154 ceph-mon[36428]: pgmap v427: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-06-06T04:31:20.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:19 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3305299658' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-06T04:31:20.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:19 smithi154 ceph-mon[36428]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:31:20.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:19 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1652135374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:31:20.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.598+0000 7f4b57061640 1 -- 172.21.15.45:0/3531421738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4b500625a0 con 0x7f4b50101af0 2024-06-06T04:31:20.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.598+0000 7f4b4dffb640 1 -- 172.21.15.45:0/3531421738 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7f4b4009bcd0 con 0x7f4b50101af0 2024-06-06T04:31:20.599 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-06-06T04:31:20.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.600+0000 7f4b57061640 1 -- 172.21.15.45:0/3531421738 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4b3c0a2c10 msgr2=0x7f4b3c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:20.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.600+0000 7f4b57061640 1 --2- 172.21.15.45:0/3531421738 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4b3c0a2c10 0x7f4b3c0a50d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4b38005fd0 tx=0x7f4b38005850 comp rx=0 tx=0).stop 2024-06-06T04:31:20.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.600+0000 7f4b57061640 1 -- 172.21.15.45:0/3531421738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b50101af0 msgr2=0x7f4b5019c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:20.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.600+0000 7f4b57061640 1 --2- 172.21.15.45:0/3531421738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b50101af0 0x7f4b5019c950 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000e9f0 tx=0x7f4b4000edf0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.600+0000 7f4b57061640 1 -- 172.21.15.45:0/3531421738 shutdown_connections 2024-06-06T04:31:20.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.600+0000 7f4b57061640 1 --2- 172.21.15.45:0/3531421738 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4b3c0a2c10 0x7f4b3c0a50d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.600+0000 7f4b57061640 1 --2- 172.21.15.45:0/3531421738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b501024b0 0x7f4b5019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.600+0000 7f4b57061640 1 --2- 172.21.15.45:0/3531421738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b50101af0 0x7f4b5019c950 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.600+0000 7f4b57061640 1 --2- 172.21.15.45:0/3531421738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b50100900 0x7f4b5006ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.600+0000 7f4b57061640 1 -- 172.21.15.45:0/3531421738 >> 172.21.15.45:0/3531421738 conn(0x7f4b500fbe90 msgr2=0x7f4b50104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:20.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.600+0000 7f4b57061640 1 -- 172.21.15.45:0/3531421738 shutdown_connections 2024-06-06T04:31:20.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.600+0000 7f4b57061640 1 -- 172.21.15.45:0/3531421738 wait complete. 2024-06-06T04:31:20.612 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-06T04:31:20.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.719+0000 7ff1b9eda640 1 -- 172.21.15.45:0/331529180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1b41008e0 msgr2=0x7ff1b4100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:20.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.719+0000 7ff1b9eda640 1 --2- 172.21.15.45:0/331529180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1b41008e0 0x7ff1b4100d00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009900 tx=0x7ff1a402a410 comp rx=0 tx=0).stop 2024-06-06T04:31:20.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.719+0000 7ff1b9eda640 1 -- 172.21.15.45:0/331529180 shutdown_connections 2024-06-06T04:31:20.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.719+0000 7ff1b9eda640 1 --2- 172.21.15.45:0/331529180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b4101ad0 0x7ff1b4101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.719+0000 7ff1b9eda640 1 --2- 172.21.15.45:0/331529180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1b4102490 0x7ff1b4108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.719+0000 7ff1b9eda640 1 --2- 172.21.15.45:0/331529180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1b41008e0 0x7ff1b4100d00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.719+0000 7ff1b9eda640 1 -- 172.21.15.45:0/331529180 >> 172.21.15.45:0/331529180 conn(0x7ff1b40fbe50 msgr2=0x7ff1b40fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.719+0000 7ff1b9eda640 1 -- 172.21.15.45:0/331529180 shutdown_connections 2024-06-06T04:31:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.719+0000 7ff1b9eda640 1 -- 172.21.15.45:0/331529180 wait complete. 2024-06-06T04:31:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.719+0000 7ff1b9eda640 1 Processor -- start 2024-06-06T04:31:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.719+0000 7ff1b9eda640 1 -- start start 2024-06-06T04:31:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b9eda640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1b41008e0 0x7ff1b40696a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b9eda640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b4101ad0 0x7ff1b419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b9eda640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1b4102490 0x7ff1b41a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b9eda640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b410c3e0 con 0x7ff1b4102490 2024-06-06T04:31:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b9eda640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b410c260 con 0x7ff1b4101ad0 2024-06-06T04:31:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b9eda640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b410c560 con 0x7ff1b41008e0 2024-06-06T04:31:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1b4102490 0x7ff1b41a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1b4102490 0x7ff1b41a12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41132/0 (socket says 172.21.15.45:41132) 2024-06-06T04:31:20.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b3fff640 1 -- 172.21.15.45:0/628097438 learned_addr learned my addr 172.21.15.45:0/628097438 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:20.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1aaffd640 1 --2- 172.21.15.45:0/628097438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b4101ad0 0x7ff1b419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:20.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b37fe640 1 --2- 172.21.15.45:0/628097438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1b41008e0 0x7ff1b40696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:20.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b3fff640 1 -- 172.21.15.45:0/628097438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1b41008e0 msgr2=0x7ff1b40696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:20.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b3fff640 1 --2- 172.21.15.45:0/628097438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1b41008e0 0x7ff1b40696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b3fff640 1 -- 172.21.15.45:0/628097438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b4101ad0 msgr2=0x7ff1b419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:20.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b3fff640 1 --2- 172.21.15.45:0/628097438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b4101ad0 0x7ff1b419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b3fff640 1 -- 172.21.15.45:0/628097438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b41a1aa0 con 0x7ff1b4102490 2024-06-06T04:31:20.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b3fff640 1 --2- 172.21.15.45:0/628097438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1b4102490 0x7ff1b41a12b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00cce0 tx=0x7ff1ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:20.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.720+0000 7ff1b17fa640 1 -- 172.21.15.45:0/628097438 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac00f450 con 0x7ff1b4102490 2024-06-06T04:31:20.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.721+0000 7ff1b17fa640 1 -- 172.21.15.45:0/628097438 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1ac00faa0 con 0x7ff1b4102490 2024-06-06T04:31:20.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.721+0000 7ff1b9eda640 1 -- 172.21.15.45:0/628097438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b41a1d30 con 0x7ff1b4102490 2024-06-06T04:31:20.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.721+0000 7ff1b17fa640 1 -- 172.21.15.45:0/628097438 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac018b70 con 0x7ff1b4102490 2024-06-06T04:31:20.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.721+0000 7ff1b9eda640 1 -- 172.21.15.45:0/628097438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b41a2220 con 0x7ff1b4102490 2024-06-06T04:31:20.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.722+0000 7ff1b9eda640 1 -- 172.21.15.45:0/628097438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b4107c70 con 0x7ff1b4102490 2024-06-06T04:31:20.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.723+0000 7ff1b17fa640 1 -- 172.21.15.45:0/628097438 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff1ac018d10 con 0x7ff1b4102490 2024-06-06T04:31:20.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.724+0000 7ff1b17fa640 1 --2- 172.21.15.45:0/628097438 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff1a00a2c40 0x7ff1a00a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:20.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.724+0000 7ff1b37fe640 1 --2- 172.21.15.45:0/628097438 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff1a00a2c40 0x7ff1a00a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:20.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.724+0000 7ff1b17fa640 1 -- 172.21.15.45:0/628097438 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7ff1ac0cf7a0 con 0x7ff1b4102490 2024-06-06T04:31:20.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.724+0000 7ff1b37fe640 1 --2- 172.21.15.45:0/628097438 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff1a00a2c40 0x7ff1a00a5100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4000c00 tx=0x7ff1a40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:20.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.728+0000 7ff1b17fa640 1 -- 172.21.15.45:0/628097438 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff1ac014030 con 0x7ff1b4102490 2024-06-06T04:31:20.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.912+0000 7ff1b9eda640 1 -- 172.21.15.45:0/628097438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff1b40626a0 con 0x7ff1b4102490 2024-06-06T04:31:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.912+0000 7ff1b17fa640 1 -- 172.21.15.45:0/628097438 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v100) ==== 103+0+11 (secure 0 0 0) 0x7ff1ac09c340 con 0x7ff1b4102490 2024-06-06T04:31:20.936 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-06-06T04:31:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.914+0000 7ff1b9eda640 1 -- 172.21.15.45:0/628097438 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff1a00a2c40 msgr2=0x7ff1a00a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.914+0000 7ff1b9eda640 1 --2- 172.21.15.45:0/628097438 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff1a00a2c40 0x7ff1a00a5100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4000c00 tx=0x7ff1a40314a0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.914+0000 7ff1b9eda640 1 -- 172.21.15.45:0/628097438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1b4102490 msgr2=0x7ff1b41a12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.914+0000 7ff1b9eda640 1 --2- 172.21.15.45:0/628097438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1b4102490 0x7ff1b41a12b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00cce0 tx=0x7ff1ac00a620 comp rx=0 tx=0).stop 2024-06-06T04:31:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.914+0000 7ff1b9eda640 1 -- 172.21.15.45:0/628097438 shutdown_connections 2024-06-06T04:31:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.914+0000 7ff1b9eda640 1 --2- 172.21.15.45:0/628097438 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff1a00a2c40 0x7ff1a00a5100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.914+0000 7ff1b9eda640 1 --2- 172.21.15.45:0/628097438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b4101ad0 0x7ff1b419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.914+0000 7ff1b9eda640 1 --2- 172.21.15.45:0/628097438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1b4102490 0x7ff1b41a12b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.914+0000 7ff1b9eda640 1 --2- 172.21.15.45:0/628097438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1b41008e0 0x7ff1b40696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.914+0000 7ff1b9eda640 1 -- 172.21.15.45:0/628097438 >> 172.21.15.45:0/628097438 conn(0x7ff1b40fbe50 msgr2=0x7ff1b4104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.915+0000 7ff1b9eda640 1 -- 172.21.15.45:0/628097438 shutdown_connections 2024-06-06T04:31:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:20.915+0000 7ff1b9eda640 1 -- 172.21.15.45:0/628097438 wait complete. 2024-06-06T04:31:20.938 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-06-06T04:31:20.938 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:31:21.193 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:31:21.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:20 smithi045 ceph-mon[32299]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:31:21.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:20 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2277943285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T04:31:21.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:20 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/25487736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T04:31:21.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:20 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3531421738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T04:31:21.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:20 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/628097438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T04:31:21.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:20 smithi086 ceph-mon[36053]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:31:21.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:20 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2277943285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T04:31:21.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:20 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/25487736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T04:31:21.260 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:20 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3531421738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T04:31:21.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:20 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/628097438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T04:31:21.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:20 smithi154 ceph-mon[36428]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T04:31:21.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:20 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2277943285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T04:31:21.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:20 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/25487736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T04:31:21.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:20 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3531421738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T04:31:21.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:20 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/628097438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T04:31:22.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:21 smithi045 ceph-mon[32299]: pgmap v428: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-06-06T04:31:22.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:21 smithi045 ceph-mon[32299]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-06-06T04:31:22.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:21 smithi086 ceph-mon[36053]: pgmap v428: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-06-06T04:31:22.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:21 smithi086 ceph-mon[36053]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-06-06T04:31:22.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:21 smithi154 ceph-mon[36428]: pgmap v428: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-06-06T04:31:22.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:21 smithi154 ceph-mon[36428]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-06-06T04:31:23.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.003+0000 7f8ffd95a700 1 -- 172.21.15.45:0/3794576658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff8106720 msgr2=0x7f8ff810e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:23.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.003+0000 7f8ffd95a700 1 --2- 172.21.15.45:0/3794576658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff8106720 0x7f8ff810e2c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000b0d0 tx=0x7f8ff000b3e0 comp rx=0 tx=0).stop 2024-06-06T04:31:23.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.003+0000 7f8ffd95a700 1 -- 172.21.15.45:0/3794576658 shutdown_connections 2024-06-06T04:31:23.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.003+0000 7f8ffd95a700 1 --2- 172.21.15.45:0/3794576658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff8106720 0x7f8ff810e2c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:23.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.003+0000 7f8ffd95a700 1 --2- 172.21.15.45:0/3794576658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff81019d0 0x7f8ff8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:23.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.003+0000 7f8ffd95a700 1 --2- 172.21.15.45:0/3794576658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff8102380 0x7f8ff8105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:23.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.003+0000 7f8ffd95a700 1 -- 172.21.15.45:0/3794576658 >> 172.21.15.45:0/3794576658 conn(0x7f8ff8078340 msgr2=0x7f8ff8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:23.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.003+0000 7f8ffd95a700 1 -- 172.21.15.45:0/3794576658 shutdown_connections 2024-06-06T04:31:23.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.003+0000 7f8ffd95a700 1 -- 172.21.15.45:0/3794576658 wait complete. 2024-06-06T04:31:23.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.003+0000 7f8ffd95a700 1 Processor -- start 2024-06-06T04:31:23.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ffd95a700 1 -- start start 2024-06-06T04:31:23.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ffd95a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff81019d0 0x7f8ff8103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:23.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ffd95a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff8102380 0x7f8ff81035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:23.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ffd95a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff8106720 0x7f8ff8103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:23.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ffd95a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff81115b0 con 0x7f8ff8102380 2024-06-06T04:31:23.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ffd95a700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff8111430 con 0x7f8ff81019d0 2024-06-06T04:31:23.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ffd95a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff8111730 con 0x7f8ff8106720 2024-06-06T04:31:23.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ff67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff8102380 0x7f8ff81035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:23.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ff67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff8102380 0x7f8ff81035d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41146/0 (socket says 172.21.15.45:41146) 2024-06-06T04:31:23.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ff67fc700 1 -- 172.21.15.45:0/3615005037 learned_addr learned my addr 172.21.15.45:0/3615005037 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:23.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ff77fe700 1 --2- 172.21.15.45:0/3615005037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff8106720 0x7f8ff8103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:23.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ff6ffd700 1 --2- 172.21.15.45:0/3615005037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff81019d0 0x7f8ff8103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:23.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ff67fc700 1 -- 172.21.15.45:0/3615005037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff8106720 msgr2=0x7f8ff8103b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:23.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ff67fc700 1 --2- 172.21.15.45:0/3615005037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff8106720 0x7f8ff8103b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:23.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ff67fc700 1 -- 172.21.15.45:0/3615005037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff81019d0 msgr2=0x7f8ff8103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:23.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ff67fc700 1 --2- 172.21.15.45:0/3615005037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff81019d0 0x7f8ff8103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:23.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ff67fc700 1 -- 172.21.15.45:0/3615005037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff81a70b0 con 0x7f8ff8102380 2024-06-06T04:31:23.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.004+0000 7f8ff6ffd700 1 --2- 172.21.15.45:0/3615005037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff81019d0 0x7f8ff8103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:31:23.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.005+0000 7f8ff67fc700 1 --2- 172.21.15.45:0/3615005037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff8102380 0x7f8ff81035d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00d980 tx=0x7f8fec00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:23.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.005+0000 7f8ffc958700 1 -- 172.21.15.45:0/3615005037 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec017070 con 0x7f8ff8102380 2024-06-06T04:31:23.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.005+0000 7f8ffc958700 1 -- 172.21.15.45:0/3615005037 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fec00be20 con 0x7f8ff8102380 2024-06-06T04:31:23.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.005+0000 7f8ffd95a700 1 -- 172.21.15.45:0/3615005037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff81a73a0 con 0x7f8ff8102380 2024-06-06T04:31:23.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.005+0000 7f8ffc958700 1 -- 172.21.15.45:0/3615005037 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0053a0 con 0x7f8ff8102380 2024-06-06T04:31:23.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.005+0000 7f8ffd95a700 1 -- 172.21.15.45:0/3615005037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff81a78e0 con 0x7f8ff8102380 2024-06-06T04:31:23.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.006+0000 7f8ffd95a700 1 -- 172.21.15.45:0/3615005037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff8102ad0 con 0x7f8ff8102380 2024-06-06T04:31:23.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.007+0000 7f8ffc958700 1 -- 172.21.15.45:0/3615005037 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f8fec005540 con 0x7f8ff8102380 2024-06-06T04:31:23.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.008+0000 7f8ffc958700 1 --2- 172.21.15.45:0/3615005037 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8fe00a2d30 0x7f8fe00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:23.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.008+0000 7f8ff6ffd700 1 --2- 172.21.15.45:0/3615005037 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8fe00a2d30 0x7f8fe00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:23.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.008+0000 7f8ffc958700 1 -- 172.21.15.45:0/3615005037 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f8fec0cf1c0 con 0x7f8ff8102380 2024-06-06T04:31:23.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.008+0000 7f8ff6ffd700 1 --2- 172.21.15.45:0/3615005037 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8fe00a2d30 0x7f8fe00a51f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8005fd0 tx=0x7f8fe8005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:23.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.011+0000 7f8ffc958700 1 -- 172.21.15.45:0/3615005037 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8fec09bc80 con 0x7f8ff8102380 2024-06-06T04:31:23.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.266+0000 7f8ffd95a700 1 -- 172.21.15.45:0/3615005037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ff802dca0 con 0x7f8ff8102380 2024-06-06T04:31:23.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.267+0000 7f8ffc958700 1 -- 172.21.15.45:0/3615005037 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11858 (secure 0 0 0) 0x7f8fec09b580 con 0x7f8ff8102380 2024-06-06T04:31:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:31:23.268 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T04:27:24.462897+0000","modified":"2024-06-06T04:31:17.956589+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24451,24463,24473]},"id":1}]} 2024-06-06T04:31:23.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.269+0000 7f8ffd95a700 1 -- 172.21.15.45:0/3615005037 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8fe00a2d30 msgr2=0x7f8fe00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:23.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.269+0000 7f8ffd95a700 1 --2- 172.21.15.45:0/3615005037 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8fe00a2d30 0x7f8fe00a51f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8005fd0 tx=0x7f8fe8005ee0 comp rx=0 tx=0).stop 2024-06-06T04:31:23.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.269+0000 7f8ffd95a700 1 -- 172.21.15.45:0/3615005037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff8102380 msgr2=0x7f8ff81035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:23.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.269+0000 7f8ffd95a700 1 --2- 172.21.15.45:0/3615005037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff8102380 0x7f8ff81035d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00d980 tx=0x7f8fec00dd40 comp rx=0 tx=0).stop 2024-06-06T04:31:23.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.269+0000 7f8ffd95a700 1 -- 172.21.15.45:0/3615005037 shutdown_connections 2024-06-06T04:31:23.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.269+0000 7f8ffd95a700 1 --2- 172.21.15.45:0/3615005037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff8106720 0x7f8ff8103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:23.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.269+0000 7f8ffd95a700 1 --2- 172.21.15.45:0/3615005037 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8fe00a2d30 0x7f8fe00a51f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:23.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.269+0000 7f8ffd95a700 1 --2- 172.21.15.45:0/3615005037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff81019d0 0x7f8ff8103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:23.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.269+0000 7f8ffd95a700 1 --2- 172.21.15.45:0/3615005037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff8102380 0x7f8ff81035d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:23.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.269+0000 7f8ffd95a700 1 -- 172.21.15.45:0/3615005037 >> 172.21.15.45:0/3615005037 conn(0x7f8ff8078340 msgr2=0x7f8ff806aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:23.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.269+0000 7f8ffd95a700 1 -- 172.21.15.45:0/3615005037 shutdown_connections 2024-06-06T04:31:23.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:23.269+0000 7f8ffd95a700 1 -- 172.21.15.45:0/3615005037 wait complete. 2024-06-06T04:31:23.272 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-06-06T04:31:23.939 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-06-06T04:31:23.939 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-06-06T04:31:23.940 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-06T04:31:24.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.049+0000 7fa18e5ee640 1 -- 172.21.15.45:0/355085326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1881024b0 msgr2=0x7fa188108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:24.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.049+0000 7fa18e5ee640 1 --2- 172.21.15.45:0/355085326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1881024b0 0x7fa188108d40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa180009ca0 tx=0x7fa18002a8c0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.049+0000 7fa18e5ee640 1 -- 172.21.15.45:0/355085326 shutdown_connections 2024-06-06T04:31:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.049+0000 7fa18e5ee640 1 --2- 172.21.15.45:0/355085326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1881024b0 0x7fa188108d40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.049+0000 7fa18e5ee640 1 --2- 172.21.15.45:0/355085326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa188101af0 0x7fa188101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.049+0000 7fa18e5ee640 1 --2- 172.21.15.45:0/355085326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa188100900 0x7fa188100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.049+0000 7fa18e5ee640 1 -- 172.21.15.45:0/355085326 >> 172.21.15.45:0/355085326 conn(0x7fa1880fbe90 msgr2=0x7fa1880fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.049+0000 7fa18e5ee640 1 -- 172.21.15.45:0/355085326 shutdown_connections 2024-06-06T04:31:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.049+0000 7fa18e5ee640 1 -- 172.21.15.45:0/355085326 wait complete. 2024-06-06T04:31:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.049+0000 7fa18e5ee640 1 Processor -- start 2024-06-06T04:31:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.049+0000 7fa18e5ee640 1 -- start start 2024-06-06T04:31:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa18e5ee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa188100900 0x7fa188197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa18e5ee640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa188101af0 0x7fa188198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa18e5ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1881024b0 0x7fa18819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa18e5ee640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa18810d630 con 0x7fa1881024b0 2024-06-06T04:31:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa18e5ee640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa18810d4b0 con 0x7fa188100900 2024-06-06T04:31:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa18e5ee640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa18810d7b0 con 0x7fa188101af0 2024-06-06T04:31:24.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa18cb8b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1881024b0 0x7fa18819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa18cb8b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1881024b0 0x7fa18819f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41158/0 (socket says 172.21.15.45:41158) 2024-06-06T04:31:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa18cb8b640 1 -- 172.21.15.45:0/3394063337 learned_addr learned my addr 172.21.15.45:0/3394063337 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa1877fe640 1 --2- 172.21.15.45:0/3394063337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa188101af0 0x7fa188198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa187fff640 1 --2- 172.21.15.45:0/3394063337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa188100900 0x7fa188197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa18cb8b640 1 -- 172.21.15.45:0/3394063337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa188101af0 msgr2=0x7fa188198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa18cb8b640 1 --2- 172.21.15.45:0/3394063337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa188101af0 0x7fa188198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa18cb8b640 1 -- 172.21.15.45:0/3394063337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa188100900 msgr2=0x7fa188197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa18cb8b640 1 --2- 172.21.15.45:0/3394063337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa188100900 0x7fa188197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa18cb8b640 1 -- 172.21.15.45:0/3394063337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18819f900 con 0x7fa1881024b0 2024-06-06T04:31:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.050+0000 7fa18cb8b640 1 --2- 172.21.15.45:0/3394063337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1881024b0 0x7fa18819f260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa180009dc0 tx=0x7fa180003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.051+0000 7fa1857fa640 1 -- 172.21.15.45:0/3394063337 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180004040 con 0x7fa1881024b0 2024-06-06T04:31:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.051+0000 7fa1857fa640 1 -- 172.21.15.45:0/3394063337 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa180004690 con 0x7fa1881024b0 2024-06-06T04:31:24.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.051+0000 7fa18e5ee640 1 -- 172.21.15.45:0/3394063337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa18819fb90 con 0x7fa1881024b0 2024-06-06T04:31:24.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.051+0000 7fa1857fa640 1 -- 172.21.15.45:0/3394063337 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18002f4c0 con 0x7fa1881024b0 2024-06-06T04:31:24.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.051+0000 7fa18e5ee640 1 -- 172.21.15.45:0/3394063337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1881a0130 con 0x7fa1881024b0 2024-06-06T04:31:24.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.052+0000 7fa18e5ee640 1 -- 172.21.15.45:0/3394063337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa188107c20 con 0x7fa1881024b0 2024-06-06T04:31:24.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.053+0000 7fa1857fa640 1 -- 172.21.15.45:0/3394063337 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa1800305c0 con 0x7fa1881024b0 2024-06-06T04:31:24.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.054+0000 7fa1857fa640 1 --2- 172.21.15.45:0/3394063337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa1700a2ca0 0x7fa1700a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:24.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.054+0000 7fa187fff640 1 --2- 172.21.15.45:0/3394063337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa1700a2ca0 0x7fa1700a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:24.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.054+0000 7fa1857fa640 1 -- 172.21.15.45:0/3394063337 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fa1800f0cc0 con 0x7fa1881024b0 2024-06-06T04:31:24.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.054+0000 7fa187fff640 1 --2- 172.21.15.45:0/3394063337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa1700a2ca0 0x7fa1700a5160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa1881991c0 tx=0x7fa178008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:24.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.058+0000 7fa1857fa640 1 -- 172.21.15.45:0/3394063337 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa18004d3a0 con 0x7fa1881024b0 2024-06-06T04:31:24.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:23 smithi045 ceph-mon[32299]: pgmap v429: 129 pgs: 129 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-06-06T04:31:24.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:23 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3615005037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:31:24.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.240+0000 7fa18e5ee640 1 -- 172.21.15.45:0/3394063337 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa188061de0 con 0x7fa1700a2ca0 2024-06-06T04:31:24.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.250+0000 7fa1857fa640 1 -- 172.21.15.45:0/3394063337 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340940 (secure 0 0 0) 0x7fa188061de0 con 0x7fa1700a2ca0 2024-06-06T04:31:24.256 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:31:24.258 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-06-06T04:31:24.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.254+0000 7fa18e5ee640 1 -- 172.21.15.45:0/3394063337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa1700a2ca0 msgr2=0x7fa1700a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:24.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.254+0000 7fa18e5ee640 1 --2- 172.21.15.45:0/3394063337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa1700a2ca0 0x7fa1700a5160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa1881991c0 tx=0x7fa178008040 comp rx=0 tx=0).stop 2024-06-06T04:31:24.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.254+0000 7fa18e5ee640 1 -- 172.21.15.45:0/3394063337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1881024b0 msgr2=0x7fa18819f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:24.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.254+0000 7fa18e5ee640 1 --2- 172.21.15.45:0/3394063337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1881024b0 0x7fa18819f260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa180009dc0 tx=0x7fa180003ba0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.254+0000 7fa18e5ee640 1 -- 172.21.15.45:0/3394063337 shutdown_connections 2024-06-06T04:31:24.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.254+0000 7fa18e5ee640 1 --2- 172.21.15.45:0/3394063337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa1700a2ca0 0x7fa1700a5160 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.254+0000 7fa18e5ee640 1 --2- 172.21.15.45:0/3394063337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1881024b0 0x7fa18819f260 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.254+0000 7fa18e5ee640 1 --2- 172.21.15.45:0/3394063337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa188101af0 0x7fa188198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.254+0000 7fa18e5ee640 1 --2- 172.21.15.45:0/3394063337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa188100900 0x7fa188197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.254+0000 7fa18e5ee640 1 -- 172.21.15.45:0/3394063337 >> 172.21.15.45:0/3394063337 conn(0x7fa1880fbe90 msgr2=0x7fa188104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:24.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.254+0000 7fa18e5ee640 1 -- 172.21.15.45:0/3394063337 shutdown_connections 2024-06-06T04:31:24.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.254+0000 7fa18e5ee640 1 -- 172.21.15.45:0/3394063337 wait complete. 2024-06-06T04:31:24.266 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-06-06T04:31:22.425007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668698,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1185,"num_read_kb":722,"num_write":372,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":801,"ondisk_log_size":801,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375052,"kb_used_data":13420,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566772,"statfs":{"total":1151940427776,"available":1151556374528,"internally_reserved":0,"allocated":13742080,"data_stored":8840720,"data_compressed":497595,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9090,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":51,"num_write":13,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003512"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.386822+0000","last_change":"2024-06-06T04:27:22.574900+0000","last_active":"2024-06-06T04:30:04.386822+0000","last_peered":"2024-06-06T04:30:04.386822+0000","last_clean":"2024-06-06T04:30:04.386822+0000","last_became_active":"2024-06-06T04:27:22.574389+0000","last_became_peered":"2024-06-06T04:27:22.574389+0000","last_unstale":"2024-06-06T04:30:04.386822+0000","last_undegraded":"2024-06-06T04:30:04.386822+0000","last_fullsized":"2024-06-06T04:30:04.386822+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:39.246463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.382712+0000","last_change":"2024-06-06T04:27:22.565543+0000","last_active":"2024-06-06T04:30:04.382712+0000","last_peered":"2024-06-06T04:30:04.382712+0000","last_clean":"2024-06-06T04:30:04.382712+0000","last_became_active":"2024-06-06T04:27:22.563833+0000","last_became_peered":"2024-06-06T04:27:22.563833+0000","last_unstale":"2024-06-06T04:30:04.382712+0000","last_undegraded":"2024-06-06T04:30:04.382712+0000","last_fullsized":"2024-06-06T04:30:04.382712+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:26:17.810782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388761+0000","last_change":"2024-06-06T04:27:22.570107+0000","last_active":"2024-06-06T04:30:04.388761+0000","last_peered":"2024-06-06T04:30:04.388761+0000","last_clean":"2024-06-06T04:30:04.388761+0000","last_became_active":"2024-06-06T04:27:22.569788+0000","last_became_peered":"2024-06-06T04:27:22.569788+0000","last_unstale":"2024-06-06T04:30:04.388761+0000","last_undegraded":"2024-06-06T04:30:04.388761+0000","last_fullsized":"2024-06-06T04:30:04.388761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:48:20.454782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385114+0000","last_change":"2024-06-06T04:27:22.559431+0000","last_active":"2024-06-06T04:30:04.385114+0000","last_peered":"2024-06-06T04:30:04.385114+0000","last_clean":"2024-06-06T04:30:04.385114+0000","last_became_active":"2024-06-06T04:27:22.558961+0000","last_became_peered":"2024-06-06T04:27:22.558961+0000","last_unstale":"2024-06-06T04:30:04.385114+0000","last_undegraded":"2024-06-06T04:30:04.385114+0000","last_fullsized":"2024-06-06T04:30:04.385114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:59.445283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.359959+0000","last_change":"2024-06-06T04:27:22.561598+0000","last_active":"2024-06-06T04:30:04.359959+0000","last_peered":"2024-06-06T04:30:04.359959+0000","last_clean":"2024-06-06T04:30:04.359959+0000","last_became_active":"2024-06-06T04:27:22.561198+0000","last_became_peered":"2024-06-06T04:27:22.561198+0000","last_unstale":"2024-06-06T04:30:04.359959+0000","last_undegraded":"2024-06-06T04:30:04.359959+0000","last_fullsized":"2024-06-06T04:30:04.359959+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:20:28.314941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380904+0000","last_change":"2024-06-06T04:27:22.569395+0000","last_active":"2024-06-06T04:30:04.380904+0000","last_peered":"2024-06-06T04:30:04.380904+0000","last_clean":"2024-06-06T04:30:04.380904+0000","last_became_active":"2024-06-06T04:27:22.569054+0000","last_became_peered":"2024-06-06T04:27:22.569054+0000","last_unstale":"2024-06-06T04:30:04.380904+0000","last_undegraded":"2024-06-06T04:30:04.380904+0000","last_fullsized":"2024-06-06T04:30:04.380904+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:03:56.630677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392227+0000","last_change":"2024-06-06T04:27:22.577121+0000","last_active":"2024-06-06T04:30:04.392227+0000","last_peered":"2024-06-06T04:30:04.392227+0000","last_clean":"2024-06-06T04:30:04.392227+0000","last_became_active":"2024-06-06T04:27:22.575574+0000","last_became_peered":"2024-06-06T04:27:22.575574+0000","last_unstale":"2024-06-06T04:30:04.392227+0000","last_undegraded":"2024-06-06T04:30:04.392227+0000","last_fullsized":"2024-06-06T04:30:04.392227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:05:16.799335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.383226+0000","last_change":"2024-06-06T04:27:22.570051+0000","last_active":"2024-06-06T04:30:04.383226+0000","last_peered":"2024-06-06T04:30:04.383226+0000","last_clean":"2024-06-06T04:30:04.383226+0000","last_became_active":"2024-06-06T04:27:22.569640+0000","last_became_peered":"2024-06-06T04:27:22.569640+0000","last_unstale":"2024-06-06T04:30:04.383226+0000","last_undegraded":"2024-06-06T04:30:04.383226+0000","last_fullsized":"2024-06-06T04:30:04.383226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:48:00.744167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.396353+0000","last_change":"2024-06-06T04:27:22.574900+0000","last_active":"2024-06-06T04:30:04.396353+0000","last_peered":"2024-06-06T04:30:04.396353+0000","last_clean":"2024-06-06T04:30:04.396353+0000","last_became_active":"2024-06-06T04:27:22.574570+0000","last_became_peered":"2024-06-06T04:27:22.574570+0000","last_unstale":"2024-06-06T04:30:04.396353+0000","last_undegraded":"2024-06-06T04:30:04.396353+0000","last_fullsized":"2024-06-06T04:30:04.396353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:54.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380728+0000","last_change":"2024-06-06T04:27:22.570581+0000","last_active":"2024-06-06T04:30:04.380728+0000","last_peered":"2024-06-06T04:30:04.380728+0000","last_clean":"2024-06-06T04:30:04.380728+0000","last_became_active":"2024-06-06T04:27:22.570243+0000","last_became_peered":"2024-06-06T04:27:22.570243+0000","last_unstale":"2024-06-06T04:30:04.380728+0000","last_undegraded":"2024-06-06T04:30:04.380728+0000","last_fullsized":"2024-06-06T04:30:04.380728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:27.087527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.395227+0000","last_change":"2024-06-06T04:27:22.571162+0000","last_active":"2024-06-06T04:30:04.395227+0000","last_peered":"2024-06-06T04:30:04.395227+0000","last_clean":"2024-06-06T04:30:04.395227+0000","last_became_active":"2024-06-06T04:27:22.570843+0000","last_became_peered":"2024-06-06T04:27:22.570843+0000","last_unstale":"2024-06-06T04:30:04.395227+0000","last_undegraded":"2024-06-06T04:30:04.395227+0000","last_fullsized":"2024-06-06T04:30:04.395227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:43:05.590240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.384953+0000","last_change":"2024-06-06T04:27:22.567420+0000","last_active":"2024-06-06T04:30:04.384953+0000","last_peered":"2024-06-06T04:30:04.384953+0000","last_clean":"2024-06-06T04:30:04.384953+0000","last_became_active":"2024-06-06T04:27:22.566750+0000","last_became_peered":"2024-06-06T04:27:22.566750+0000","last_unstale":"2024-06-06T04:30:04.384953+0000","last_undegraded":"2024-06-06T04:30:04.384953+0000","last_fullsized":"2024-06-06T04:30:04.384953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:21.669965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.364827+0000","last_change":"2024-06-06T04:27:22.566831+0000","last_active":"2024-06-06T04:30:04.364827+0000","last_peered":"2024-06-06T04:30:04.364827+0000","last_clean":"2024-06-06T04:30:04.364827+0000","last_became_active":"2024-06-06T04:27:22.566255+0000","last_became_peered":"2024-06-06T04:27:22.566255+0000","last_unstale":"2024-06-06T04:30:04.364827+0000","last_undegraded":"2024-06-06T04:30:04.364827+0000","last_fullsized":"2024-06-06T04:30:04.364827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:40:22.486674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.384669+0000","last_change":"2024-06-06T04:27:22.564596+0000","last_active":"2024-06-06T04:30:04.384669+0000","last_peered":"2024-06-06T04:30:04.384669+0000","last_clean":"2024-06-06T04:30:04.384669+0000","last_became_active":"2024-06-06T04:27:22.563694+0000","last_became_peered":"2024-06-06T04:27:22.563694+0000","last_unstale":"2024-06-06T04:30:04.384669+0000","last_undegraded":"2024-06-06T04:30:04.384669+0000","last_fullsized":"2024-06-06T04:30:04.384669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:18.568266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385298+0000","last_change":"2024-06-06T04:27:22.559395+0000","last_active":"2024-06-06T04:30:04.385298+0000","last_peered":"2024-06-06T04:30:04.385298+0000","last_clean":"2024-06-06T04:30:04.385298+0000","last_became_active":"2024-06-06T04:27:22.559030+0000","last_became_peered":"2024-06-06T04:27:22.559030+0000","last_unstale":"2024-06-06T04:30:04.385298+0000","last_undegraded":"2024-06-06T04:30:04.385298+0000","last_fullsized":"2024-06-06T04:30:04.385298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:08.562184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394517+0000","last_change":"2024-06-06T04:27:22.569657+0000","last_active":"2024-06-06T04:30:04.394517+0000","last_peered":"2024-06-06T04:30:04.394517+0000","last_clean":"2024-06-06T04:30:04.394517+0000","last_became_active":"2024-06-06T04:27:22.569359+0000","last_became_peered":"2024-06-06T04:27:22.569359+0000","last_unstale":"2024-06-06T04:30:04.394517+0000","last_undegraded":"2024-06-06T04:30:04.394517+0000","last_fullsized":"2024-06-06T04:30:04.394517+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:42:20.197715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394347+0000","last_change":"2024-06-06T04:27:22.574661+0000","last_active":"2024-06-06T04:30:04.394347+0000","last_peered":"2024-06-06T04:30:04.394347+0000","last_clean":"2024-06-06T04:30:04.394347+0000","last_became_active":"2024-06-06T04:27:22.574310+0000","last_became_peered":"2024-06-06T04:27:22.574310+0000","last_unstale":"2024-06-06T04:30:04.394347+0000","last_undegraded":"2024-06-06T04:30:04.394347+0000","last_fullsized":"2024-06-06T04:30:04.394347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:45:18.772270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388153+0000","last_change":"2024-06-06T04:27:22.565575+0000","last_active":"2024-06-06T04:30:04.388153+0000","last_peered":"2024-06-06T04:30:04.388153+0000","last_clean":"2024-06-06T04:30:04.388153+0000","last_became_active":"2024-06-06T04:27:22.565256+0000","last_became_peered":"2024-06-06T04:27:22.565256+0000","last_unstale":"2024-06-06T04:30:04.388153+0000","last_undegraded":"2024-06-06T04:30:04.388153+0000","last_fullsized":"2024-06-06T04:30:04.388153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:49.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.365082+0000","last_change":"2024-06-06T04:27:22.559606+0000","last_active":"2024-06-06T04:30:04.365082+0000","last_peered":"2024-06-06T04:30:04.365082+0000","last_clean":"2024-06-06T04:30:04.365082+0000","last_became_active":"2024-06-06T04:27:22.557881+0000","last_became_peered":"2024-06-06T04:27:22.557881+0000","last_unstale":"2024-06-06T04:30:04.365082+0000","last_undegraded":"2024-06-06T04:30:04.365082+0000","last_fullsized":"2024-06-06T04:30:04.365082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:31:06.269785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905476+0000","last_change":"2024-06-06T04:27:32.626612+0000","last_active":"2024-06-06T04:29:05.905476+0000","last_peered":"2024-06-06T04:29:05.905476+0000","last_clean":"2024-06-06T04:29:05.905476+0000","last_became_active":"2024-06-06T04:27:32.626113+0000","last_became_peered":"2024-06-06T04:27:32.626113+0000","last_unstale":"2024-06-06T04:29:05.905476+0000","last_undegraded":"2024-06-06T04:29:05.905476+0000","last_fullsized":"2024-06-06T04:29:05.905476+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.318738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908330+0000","last_change":"2024-06-06T04:27:19.302744+0000","last_active":"2024-06-06T04:29:05.908330+0000","last_peered":"2024-06-06T04:29:05.908330+0000","last_clean":"2024-06-06T04:29:05.908330+0000","last_became_active":"2024-06-06T04:27:19.302389+0000","last_became_peered":"2024-06-06T04:27:19.302389+0000","last_unstale":"2024-06-06T04:29:05.908330+0000","last_undegraded":"2024-06-06T04:29:05.908330+0000","last_fullsized":"2024-06-06T04:29:05.908330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:49.872618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381219+0000","last_change":"2024-06-06T04:27:22.570220+0000","last_active":"2024-06-06T04:30:04.381219+0000","last_peered":"2024-06-06T04:30:04.381219+0000","last_clean":"2024-06-06T04:30:04.381219+0000","last_became_active":"2024-06-06T04:27:22.569835+0000","last_became_peered":"2024-06-06T04:27:22.569835+0000","last_unstale":"2024-06-06T04:30:04.381219+0000","last_undegraded":"2024-06-06T04:30:04.381219+0000","last_fullsized":"2024-06-06T04:30:04.381219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:54:57.372561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401940+0000","last_change":"2024-06-06T04:27:32.622407+0000","last_active":"2024-06-06T04:29:06.401940+0000","last_peered":"2024-06-06T04:29:06.401940+0000","last_clean":"2024-06-06T04:29:06.401940+0000","last_became_active":"2024-06-06T04:27:32.621868+0000","last_became_peered":"2024-06-06T04:27:32.621868+0000","last_unstale":"2024-06-06T04:29:06.401940+0000","last_undegraded":"2024-06-06T04:29:06.401940+0000","last_fullsized":"2024-06-06T04:29:06.401940+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:50:05.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:21.905209+0000","last_change":"2024-06-06T04:27:19.516235+0000","last_active":"2024-06-06T04:29:21.905209+0000","last_peered":"2024-06-06T04:29:21.905209+0000","last_clean":"2024-06-06T04:29:21.905209+0000","last_became_active":"2024-06-06T04:27:19.515849+0000","last_became_peered":"2024-06-06T04:27:19.515849+0000","last_unstale":"2024-06-06T04:29:21.905209+0000","last_undegraded":"2024-06-06T04:29:21.905209+0000","last_fullsized":"2024-06-06T04:29:21.905209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:52:42.722103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388234+0000","last_change":"2024-06-06T04:27:22.570797+0000","last_active":"2024-06-06T04:30:04.388234+0000","last_peered":"2024-06-06T04:30:04.388234+0000","last_clean":"2024-06-06T04:30:04.388234+0000","last_became_active":"2024-06-06T04:27:22.570305+0000","last_became_peered":"2024-06-06T04:27:22.570305+0000","last_unstale":"2024-06-06T04:30:04.388234+0000","last_undegraded":"2024-06-06T04:30:04.388234+0000","last_fullsized":"2024-06-06T04:30:04.388234+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:04:14.545306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401666+0000","last_change":"2024-06-06T04:27:32.623241+0000","last_active":"2024-06-06T04:29:06.401666+0000","last_peered":"2024-06-06T04:29:06.401666+0000","last_clean":"2024-06-06T04:29:06.401666+0000","last_became_active":"2024-06-06T04:27:32.622751+0000","last_became_peered":"2024-06-06T04:27:32.622751+0000","last_unstale":"2024-06-06T04:29:06.401666+0000","last_undegraded":"2024-06-06T04:29:06.401666+0000","last_fullsized":"2024-06-06T04:29:06.401666+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:24:53.446329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.909112+0000","last_change":"2024-06-06T04:28:05.733350+0000","last_active":"2024-06-06T04:29:05.909112+0000","last_peered":"2024-06-06T04:29:05.909112+0000","last_clean":"2024-06-06T04:29:05.909112+0000","last_became_active":"2024-06-06T04:28:05.726556+0000","last_became_peered":"2024-06-06T04:28:05.726556+0000","last_unstale":"2024-06-06T04:29:05.909112+0000","last_undegraded":"2024-06-06T04:29:05.909112+0000","last_fullsized":"2024-06-06T04:29:05.909112+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:26.636625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.396094+0000","last_change":"2024-06-06T04:27:22.571731+0000","last_active":"2024-06-06T04:30:04.396094+0000","last_peered":"2024-06-06T04:30:04.396094+0000","last_clean":"2024-06-06T04:30:04.396094+0000","last_became_active":"2024-06-06T04:27:22.571398+0000","last_became_peered":"2024-06-06T04:27:22.571398+0000","last_unstale":"2024-06-06T04:30:04.396094+0000","last_undegraded":"2024-06-06T04:30:04.396094+0000","last_fullsized":"2024-06-06T04:30:04.396094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:22:45.144427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401661+0000","last_change":"2024-06-06T04:27:32.618465+0000","last_active":"2024-06-06T04:29:06.401661+0000","last_peered":"2024-06-06T04:29:06.401661+0000","last_clean":"2024-06-06T04:29:06.401661+0000","last_became_active":"2024-06-06T04:27:32.618066+0000","last_became_peered":"2024-06-06T04:27:32.618066+0000","last_unstale":"2024-06-06T04:29:06.401661+0000","last_undegraded":"2024-06-06T04:29:06.401661+0000","last_fullsized":"2024-06-06T04:29:06.401661+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:54:23.988687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908783+0000","last_change":"2024-06-06T04:28:05.730381+0000","last_active":"2024-06-06T04:29:05.908783+0000","last_peered":"2024-06-06T04:29:05.908783+0000","last_clean":"2024-06-06T04:29:05.908783+0000","last_became_active":"2024-06-06T04:28:05.726552+0000","last_became_peered":"2024-06-06T04:28:05.726552+0000","last_unstale":"2024-06-06T04:29:05.908783+0000","last_undegraded":"2024-06-06T04:29:05.908783+0000","last_fullsized":"2024-06-06T04:29:05.908783+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:52.903042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394789+0000","last_change":"2024-06-06T04:27:22.573769+0000","last_active":"2024-06-06T04:30:04.394789+0000","last_peered":"2024-06-06T04:30:04.394789+0000","last_clean":"2024-06-06T04:30:04.394789+0000","last_became_active":"2024-06-06T04:27:22.572674+0000","last_became_peered":"2024-06-06T04:27:22.572674+0000","last_unstale":"2024-06-06T04:30:04.394789+0000","last_undegraded":"2024-06-06T04:30:04.394789+0000","last_fullsized":"2024-06-06T04:30:04.394789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:57:22.653052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905675+0000","last_change":"2024-06-06T04:27:32.617934+0000","last_active":"2024-06-06T04:29:05.905675+0000","last_peered":"2024-06-06T04:29:05.905675+0000","last_clean":"2024-06-06T04:29:05.905675+0000","last_became_active":"2024-06-06T04:27:32.617537+0000","last_became_peered":"2024-06-06T04:27:32.617537+0000","last_unstale":"2024-06-06T04:29:05.905675+0000","last_undegraded":"2024-06-06T04:29:05.905675+0000","last_fullsized":"2024-06-06T04:29:05.905675+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:21:48.319434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'7","reported_seq":228,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:19.968167+0000","last_change":"2024-06-06T04:27:19.306236+0000","last_active":"2024-06-06T04:31:19.968167+0000","last_peered":"2024-06-06T04:31:19.968167+0000","last_clean":"2024-06-06T04:31:19.968167+0000","last_became_active":"2024-06-06T04:27:19.305850+0000","last_became_peered":"2024-06-06T04:27:19.305850+0000","last_unstale":"2024-06-06T04:31:19.968167+0000","last_undegraded":"2024-06-06T04:31:19.968167+0000","last_fullsized":"2024-06-06T04:31:19.968167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:58:47.310040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1504,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.365431+0000","last_change":"2024-06-06T04:27:22.567440+0000","last_active":"2024-06-06T04:30:04.365431+0000","last_peered":"2024-06-06T04:30:04.365431+0000","last_clean":"2024-06-06T04:30:04.365431+0000","last_became_active":"2024-06-06T04:27:22.566732+0000","last_became_peered":"2024-06-06T04:27:22.566732+0000","last_unstale":"2024-06-06T04:30:04.365431+0000","last_undegraded":"2024-06-06T04:30:04.365431+0000","last_fullsized":"2024-06-06T04:30:04.365431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:48:46.270103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905710+0000","last_change":"2024-06-06T04:27:32.626554+0000","last_active":"2024-06-06T04:29:05.905710+0000","last_peered":"2024-06-06T04:29:05.905710+0000","last_clean":"2024-06-06T04:29:05.905710+0000","last_became_active":"2024-06-06T04:27:32.626108+0000","last_became_peered":"2024-06-06T04:27:32.626108+0000","last_unstale":"2024-06-06T04:29:05.905710+0000","last_undegraded":"2024-06-06T04:29:05.905710+0000","last_fullsized":"2024-06-06T04:29:05.905710+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:37:43.811005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.262695+0000","last_change":"2024-06-06T04:27:19.306821+0000","last_active":"2024-06-06T04:29:06.262695+0000","last_peered":"2024-06-06T04:29:06.262695+0000","last_clean":"2024-06-06T04:29:06.262695+0000","last_became_active":"2024-06-06T04:27:19.306489+0000","last_became_peered":"2024-06-06T04:27:19.306489+0000","last_unstale":"2024-06-06T04:29:06.262695+0000","last_undegraded":"2024-06-06T04:29:06.262695+0000","last_fullsized":"2024-06-06T04:29:06.262695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:31:28.973112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.378929+0000","last_change":"2024-06-06T04:27:22.570617+0000","last_active":"2024-06-06T04:30:04.378929+0000","last_peered":"2024-06-06T04:30:04.378929+0000","last_clean":"2024-06-06T04:30:04.378929+0000","last_became_active":"2024-06-06T04:27:22.570271+0000","last_became_peered":"2024-06-06T04:27:22.570271+0000","last_unstale":"2024-06-06T04:30:04.378929+0000","last_undegraded":"2024-06-06T04:30:04.378929+0000","last_fullsized":"2024-06-06T04:30:04.378929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:45:33.890164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.909010+0000","last_change":"2024-06-06T04:27:32.628132+0000","last_active":"2024-06-06T04:29:05.909010+0000","last_peered":"2024-06-06T04:29:05.909010+0000","last_clean":"2024-06-06T04:29:05.909010+0000","last_became_active":"2024-06-06T04:27:32.627658+0000","last_became_peered":"2024-06-06T04:27:32.627658+0000","last_unstale":"2024-06-06T04:29:05.909010+0000","last_undegraded":"2024-06-06T04:29:05.909010+0000","last_fullsized":"2024-06-06T04:29:05.909010+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:24.817079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904125+0000","last_change":"2024-06-06T04:27:19.300510+0000","last_active":"2024-06-06T04:29:05.904125+0000","last_peered":"2024-06-06T04:29:05.904125+0000","last_clean":"2024-06-06T04:29:05.904125+0000","last_became_active":"2024-06-06T04:27:19.300150+0000","last_became_peered":"2024-06-06T04:27:19.300150+0000","last_unstale":"2024-06-06T04:29:05.904125+0000","last_undegraded":"2024-06-06T04:29:05.904125+0000","last_fullsized":"2024-06-06T04:29:05.904125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:38:25.084690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381098+0000","last_change":"2024-06-06T04:27:22.563864+0000","last_active":"2024-06-06T04:30:04.381098+0000","last_peered":"2024-06-06T04:30:04.381098+0000","last_clean":"2024-06-06T04:30:04.381098+0000","last_became_active":"2024-06-06T04:27:22.563555+0000","last_became_peered":"2024-06-06T04:27:22.563555+0000","last_unstale":"2024-06-06T04:30:04.381098+0000","last_undegraded":"2024-06-06T04:30:04.381098+0000","last_fullsized":"2024-06-06T04:30:04.381098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:39:33.707404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.165015+0000","last_change":"2024-06-06T04:27:32.617627+0000","last_active":"2024-06-06T04:29:06.165015+0000","last_peered":"2024-06-06T04:29:06.165015+0000","last_clean":"2024-06-06T04:29:06.165015+0000","last_became_active":"2024-06-06T04:27:32.617187+0000","last_became_peered":"2024-06-06T04:27:32.617187+0000","last_unstale":"2024-06-06T04:29:06.165015+0000","last_undegraded":"2024-06-06T04:29:06.165015+0000","last_fullsized":"2024-06-06T04:29:06.165015+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:24:52.554176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790234+0000","last_change":"2024-06-06T04:27:19.304316+0000","last_active":"2024-06-06T04:29:06.790234+0000","last_peered":"2024-06-06T04:29:06.790234+0000","last_clean":"2024-06-06T04:29:06.790234+0000","last_became_active":"2024-06-06T04:27:19.303952+0000","last_became_peered":"2024-06-06T04:27:19.303952+0000","last_unstale":"2024-06-06T04:29:06.790234+0000","last_undegraded":"2024-06-06T04:29:06.790234+0000","last_fullsized":"2024-06-06T04:29:06.790234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:26.886916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.371820+0000","last_change":"2024-06-06T04:27:22.561824+0000","last_active":"2024-06-06T04:30:04.371820+0000","last_peered":"2024-06-06T04:30:04.371820+0000","last_clean":"2024-06-06T04:30:04.371820+0000","last_became_active":"2024-06-06T04:27:22.561495+0000","last_became_peered":"2024-06-06T04:27:22.561495+0000","last_unstale":"2024-06-06T04:30:04.371820+0000","last_undegraded":"2024-06-06T04:30:04.371820+0000","last_fullsized":"2024-06-06T04:30:04.371820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:40:31.314627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560380+0000","last_change":"2024-06-06T04:27:32.624540+0000","last_active":"2024-06-06T04:29:06.560380+0000","last_peered":"2024-06-06T04:29:06.560380+0000","last_clean":"2024-06-06T04:29:06.560380+0000","last_became_active":"2024-06-06T04:27:32.624036+0000","last_became_peered":"2024-06-06T04:27:32.624036+0000","last_unstale":"2024-06-06T04:29:06.560380+0000","last_undegraded":"2024-06-06T04:29:06.560380+0000","last_fullsized":"2024-06-06T04:29:06.560380+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:16.983576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'4","reported_seq":219,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:19.967188+0000","last_change":"2024-06-06T04:27:19.304614+0000","last_active":"2024-06-06T04:31:19.967188+0000","last_peered":"2024-06-06T04:31:19.967188+0000","last_clean":"2024-06-06T04:31:19.967188+0000","last_became_active":"2024-06-06T04:27:19.304126+0000","last_became_peered":"2024-06-06T04:27:19.304126+0000","last_unstale":"2024-06-06T04:31:19.967188+0000","last_undegraded":"2024-06-06T04:31:19.967188+0000","last_fullsized":"2024-06-06T04:31:19.967188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:07:39.908630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":172,"num_read_kb":172,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.396136+0000","last_change":"2024-06-06T04:27:22.572534+0000","last_active":"2024-06-06T04:30:04.396136+0000","last_peered":"2024-06-06T04:30:04.396136+0000","last_clean":"2024-06-06T04:30:04.396136+0000","last_became_active":"2024-06-06T04:27:22.572214+0000","last_became_peered":"2024-06-06T04:27:22.572214+0000","last_unstale":"2024-06-06T04:30:04.396136+0000","last_undegraded":"2024-06-06T04:30:04.396136+0000","last_fullsized":"2024-06-06T04:30:04.396136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:13.176045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.262366+0000","last_change":"2024-06-06T04:28:05.723450+0000","last_active":"2024-06-06T04:29:06.262366+0000","last_peered":"2024-06-06T04:29:06.262366+0000","last_clean":"2024-06-06T04:29:06.262366+0000","last_became_active":"2024-06-06T04:28:05.722928+0000","last_became_peered":"2024-06-06T04:28:05.722928+0000","last_unstale":"2024-06-06T04:29:06.262366+0000","last_undegraded":"2024-06-06T04:29:06.262366+0000","last_fullsized":"2024-06-06T04:29:06.262366+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:53.093276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905918+0000","last_change":"2024-06-06T04:27:19.308465+0000","last_active":"2024-06-06T04:29:05.905918+0000","last_peered":"2024-06-06T04:29:05.905918+0000","last_clean":"2024-06-06T04:29:05.905918+0000","last_became_active":"2024-06-06T04:27:19.308077+0000","last_became_peered":"2024-06-06T04:27:19.308077+0000","last_unstale":"2024-06-06T04:29:05.905918+0000","last_undegraded":"2024-06-06T04:29:05.905918+0000","last_fullsized":"2024-06-06T04:29:05.905918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:37:48.282613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385739+0000","last_change":"2024-06-06T04:27:22.574141+0000","last_active":"2024-06-06T04:30:04.385739+0000","last_peered":"2024-06-06T04:30:04.385739+0000","last_clean":"2024-06-06T04:30:04.385739+0000","last_became_active":"2024-06-06T04:27:22.573679+0000","last_became_peered":"2024-06-06T04:27:22.573679+0000","last_unstale":"2024-06-06T04:30:04.385739+0000","last_undegraded":"2024-06-06T04:30:04.385739+0000","last_fullsized":"2024-06-06T04:30:04.385739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:27:25.177990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401856+0000","last_change":"2024-06-06T04:27:32.624148+0000","last_active":"2024-06-06T04:29:06.401856+0000","last_peered":"2024-06-06T04:29:06.401856+0000","last_clean":"2024-06-06T04:29:06.401856+0000","last_became_active":"2024-06-06T04:27:32.623448+0000","last_became_peered":"2024-06-06T04:27:32.623448+0000","last_unstale":"2024-06-06T04:29:06.401856+0000","last_undegraded":"2024-06-06T04:29:06.401856+0000","last_fullsized":"2024-06-06T04:29:06.401856+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:02.037296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790253+0000","last_change":"2024-06-06T04:27:19.516875+0000","last_active":"2024-06-06T04:29:06.790253+0000","last_peered":"2024-06-06T04:29:06.790253+0000","last_clean":"2024-06-06T04:29:06.790253+0000","last_became_active":"2024-06-06T04:27:19.516393+0000","last_became_peered":"2024-06-06T04:27:19.516393+0000","last_unstale":"2024-06-06T04:29:06.790253+0000","last_undegraded":"2024-06-06T04:29:06.790253+0000","last_fullsized":"2024-06-06T04:29:06.790253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:20.079218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392596+0000","last_change":"2024-06-06T04:28:05.724043+0000","last_active":"2024-06-06T04:30:04.392596+0000","last_peered":"2024-06-06T04:30:04.392596+0000","last_clean":"2024-06-06T04:30:04.392596+0000","last_became_active":"2024-06-06T04:28:05.723622+0000","last_became_peered":"2024-06-06T04:28:05.723622+0000","last_unstale":"2024-06-06T04:30:04.392596+0000","last_undegraded":"2024-06-06T04:30:04.392596+0000","last_fullsized":"2024-06-06T04:30:04.392596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:42.162400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.907660+0000","last_change":"2024-06-06T04:27:32.626366+0000","last_active":"2024-06-06T04:29:05.907660+0000","last_peered":"2024-06-06T04:29:05.907660+0000","last_clean":"2024-06-06T04:29:05.907660+0000","last_became_active":"2024-06-06T04:27:32.625847+0000","last_became_peered":"2024-06-06T04:27:32.625847+0000","last_unstale":"2024-06-06T04:29:05.907660+0000","last_undegraded":"2024-06-06T04:29:05.907660+0000","last_fullsized":"2024-06-06T04:29:05.907660+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:47:44.167531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'42","reported_seq":441,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:20.968878+0000","last_change":"2024-06-06T04:27:19.303581+0000","last_active":"2024-06-06T04:31:20.968878+0000","last_peered":"2024-06-06T04:31:20.968878+0000","last_clean":"2024-06-06T04:31:20.968878+0000","last_became_active":"2024-06-06T04:27:19.303234+0000","last_became_peered":"2024-06-06T04:27:19.303234+0000","last_unstale":"2024-06-06T04:31:20.968878+0000","last_undegraded":"2024-06-06T04:31:20.968878+0000","last_fullsized":"2024-06-06T04:31:20.968878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:37.721588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72581,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":356,"num_read_kb":250,"num_write":43,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.391077+0000","last_change":"2024-06-06T04:27:22.567177+0000","last_active":"2024-06-06T04:30:04.391077+0000","last_peered":"2024-06-06T04:30:04.391077+0000","last_clean":"2024-06-06T04:30:04.391077+0000","last_became_active":"2024-06-06T04:27:22.566751+0000","last_became_peered":"2024-06-06T04:27:22.566751+0000","last_unstale":"2024-06-06T04:30:04.391077+0000","last_undegraded":"2024-06-06T04:30:04.391077+0000","last_fullsized":"2024-06-06T04:30:04.391077+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:37:15.914505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790613+0000","last_change":"2024-06-06T04:27:32.619319+0000","last_active":"2024-06-06T04:29:06.790613+0000","last_peered":"2024-06-06T04:29:06.790613+0000","last_clean":"2024-06-06T04:29:06.790613+0000","last_became_active":"2024-06-06T04:27:32.618843+0000","last_became_peered":"2024-06-06T04:27:32.618843+0000","last_unstale":"2024-06-06T04:29:06.790613+0000","last_undegraded":"2024-06-06T04:29:06.790613+0000","last_fullsized":"2024-06-06T04:29:06.790613+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:32:20.071512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905924+0000","last_change":"2024-06-06T04:27:19.304731+0000","last_active":"2024-06-06T04:29:05.905924+0000","last_peered":"2024-06-06T04:29:05.905924+0000","last_clean":"2024-06-06T04:29:05.905924+0000","last_became_active":"2024-06-06T04:27:19.304351+0000","last_became_peered":"2024-06-06T04:27:19.304351+0000","last_unstale":"2024-06-06T04:29:05.905924+0000","last_undegraded":"2024-06-06T04:29:05.905924+0000","last_fullsized":"2024-06-06T04:29:05.905924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:16:14.136633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.391311+0000","last_change":"2024-06-06T04:29:05.909041+0000","last_active":"2024-06-06T04:30:04.391311+0000","last_peered":"2024-06-06T04:30:04.391311+0000","last_clean":"2024-06-06T04:30:04.391311+0000","last_became_active":"2024-06-06T04:29:05.908666+0000","last_became_peered":"2024-06-06T04:29:05.908666+0000","last_unstale":"2024-06-06T04:30:04.391311+0000","last_undegraded":"2024-06-06T04:30:04.391311+0000","last_fullsized":"2024-06-06T04:30:04.391311+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:47:55.225572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908861+0000","last_change":"2024-06-06T04:27:32.628645+0000","last_active":"2024-06-06T04:29:05.908861+0000","last_peered":"2024-06-06T04:29:05.908861+0000","last_clean":"2024-06-06T04:29:05.908861+0000","last_became_active":"2024-06-06T04:27:32.627586+0000","last_became_peered":"2024-06-06T04:27:32.627586+0000","last_unstale":"2024-06-06T04:29:05.908861+0000","last_undegraded":"2024-06-06T04:29:05.908861+0000","last_fullsized":"2024-06-06T04:29:05.908861+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:46:53.094511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790029+0000","last_change":"2024-06-06T04:27:19.303805+0000","last_active":"2024-06-06T04:29:06.790029+0000","last_peered":"2024-06-06T04:29:06.790029+0000","last_clean":"2024-06-06T04:29:06.790029+0000","last_became_active":"2024-06-06T04:27:19.303391+0000","last_became_peered":"2024-06-06T04:27:19.303391+0000","last_unstale":"2024-06-06T04:29:06.790029+0000","last_undegraded":"2024-06-06T04:29:06.790029+0000","last_fullsized":"2024-06-06T04:29:06.790029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:30:27.336567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.387613+0000","last_change":"2024-06-06T04:27:22.575148+0000","last_active":"2024-06-06T04:30:04.387613+0000","last_peered":"2024-06-06T04:30:04.387613+0000","last_clean":"2024-06-06T04:30:04.387613+0000","last_became_active":"2024-06-06T04:27:22.574591+0000","last_became_peered":"2024-06-06T04:27:22.574591+0000","last_unstale":"2024-06-06T04:30:04.387613+0000","last_undegraded":"2024-06-06T04:30:04.387613+0000","last_fullsized":"2024-06-06T04:30:04.387613+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:35:23.404644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214440+0000","last_change":"2024-06-06T04:28:05.726734+0000","last_active":"2024-06-06T04:29:06.214440+0000","last_peered":"2024-06-06T04:29:06.214440+0000","last_clean":"2024-06-06T04:29:06.214440+0000","last_became_active":"2024-06-06T04:28:05.726238+0000","last_became_peered":"2024-06-06T04:28:05.726238+0000","last_unstale":"2024-06-06T04:29:06.214440+0000","last_undegraded":"2024-06-06T04:29:06.214440+0000","last_fullsized":"2024-06-06T04:29:06.214440+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:47.841475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"100'6","reported_seq":222,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:16.962391+0000","last_change":"2024-06-06T04:27:19.305571+0000","last_active":"2024-06-06T04:31:16.962391+0000","last_peered":"2024-06-06T04:31:16.962391+0000","last_clean":"2024-06-06T04:31:16.962391+0000","last_became_active":"2024-06-06T04:27:19.305183+0000","last_became_peered":"2024-06-06T04:27:19.305183+0000","last_unstale":"2024-06-06T04:31:16.962391+0000","last_undegraded":"2024-06-06T04:31:16.962391+0000","last_fullsized":"2024-06-06T04:31:16.962391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:51.205209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1611,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392588+0000","last_change":"2024-06-06T04:29:06.263872+0000","last_active":"2024-06-06T04:30:04.392588+0000","last_peered":"2024-06-06T04:30:04.392588+0000","last_clean":"2024-06-06T04:30:04.392588+0000","last_became_active":"2024-06-06T04:29:06.263487+0000","last_became_peered":"2024-06-06T04:29:06.263487+0000","last_unstale":"2024-06-06T04:30:04.392588+0000","last_undegraded":"2024-06-06T04:30:04.392588+0000","last_fullsized":"2024-06-06T04:30:04.392588+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:15.965926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904639+0000","last_change":"2024-06-06T04:28:05.724892+0000","last_active":"2024-06-06T04:29:05.904639+0000","last_peered":"2024-06-06T04:29:05.904639+0000","last_clean":"2024-06-06T04:29:05.904639+0000","last_became_active":"2024-06-06T04:28:05.724420+0000","last_became_peered":"2024-06-06T04:28:05.724420+0000","last_unstale":"2024-06-06T04:29:05.904639+0000","last_undegraded":"2024-06-06T04:29:05.904639+0000","last_fullsized":"2024-06-06T04:29:05.904639+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:39.879481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":60,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.213846+0000","last_change":"2024-06-06T04:27:19.516687+0000","last_active":"2024-06-06T04:29:06.213846+0000","last_peered":"2024-06-06T04:29:06.213846+0000","last_clean":"2024-06-06T04:29:06.213846+0000","last_became_active":"2024-06-06T04:27:19.516271+0000","last_became_peered":"2024-06-06T04:27:19.516271+0000","last_unstale":"2024-06-06T04:29:06.213846+0000","last_undegraded":"2024-06-06T04:29:06.213846+0000","last_fullsized":"2024-06-06T04:29:06.213846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:33:46.817727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394832+0000","last_change":"2024-06-06T04:27:22.572671+0000","last_active":"2024-06-06T04:30:04.394832+0000","last_peered":"2024-06-06T04:30:04.394832+0000","last_clean":"2024-06-06T04:30:04.394832+0000","last_became_active":"2024-06-06T04:27:22.572230+0000","last_became_peered":"2024-06-06T04:27:22.572230+0000","last_unstale":"2024-06-06T04:30:04.394832+0000","last_undegraded":"2024-06-06T04:30:04.394832+0000","last_fullsized":"2024-06-06T04:30:04.394832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:36:48.393092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.789710+0000","last_change":"2024-06-06T04:27:32.620131+0000","last_active":"2024-06-06T04:29:06.789710+0000","last_peered":"2024-06-06T04:29:06.789710+0000","last_clean":"2024-06-06T04:29:06.789710+0000","last_became_active":"2024-06-06T04:27:32.619686+0000","last_became_peered":"2024-06-06T04:27:32.619686+0000","last_unstale":"2024-06-06T04:29:06.789710+0000","last_undegraded":"2024-06-06T04:29:06.789710+0000","last_fullsized":"2024-06-06T04:29:06.789710+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:19:17.130410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214545+0000","last_change":"2024-06-06T04:27:19.304682+0000","last_active":"2024-06-06T04:29:06.214545+0000","last_peered":"2024-06-06T04:29:06.214545+0000","last_clean":"2024-06-06T04:29:06.214545+0000","last_became_active":"2024-06-06T04:27:19.304262+0000","last_became_peered":"2024-06-06T04:27:19.304262+0000","last_unstale":"2024-06-06T04:29:06.214545+0000","last_undegraded":"2024-06-06T04:29:06.214545+0000","last_fullsized":"2024-06-06T04:29:06.214545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:45.819520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.377096+0000","last_change":"2024-06-06T04:29:06.791555+0000","last_active":"2024-06-06T04:30:04.377096+0000","last_peered":"2024-06-06T04:30:04.377096+0000","last_clean":"2024-06-06T04:30:04.377096+0000","last_became_active":"2024-06-06T04:29:06.790925+0000","last_became_peered":"2024-06-06T04:29:06.790925+0000","last_unstale":"2024-06-06T04:30:04.377096+0000","last_undegraded":"2024-06-06T04:30:04.377096+0000","last_fullsized":"2024-06-06T04:30:04.377096+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:23:39.549238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.213893+0000","last_change":"2024-06-06T04:27:32.622092+0000","last_active":"2024-06-06T04:29:06.213893+0000","last_peered":"2024-06-06T04:29:06.213893+0000","last_clean":"2024-06-06T04:29:06.213893+0000","last_became_active":"2024-06-06T04:27:32.621618+0000","last_became_peered":"2024-06-06T04:27:32.621618+0000","last_unstale":"2024-06-06T04:29:06.213893+0000","last_undegraded":"2024-06-06T04:29:06.213893+0000","last_fullsized":"2024-06-06T04:29:06.213893+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:03.067292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.907896+0000","last_change":"2024-06-06T04:27:19.516803+0000","last_active":"2024-06-06T04:29:05.907896+0000","last_peered":"2024-06-06T04:29:05.907896+0000","last_clean":"2024-06-06T04:29:05.907896+0000","last_became_active":"2024-06-06T04:27:19.516458+0000","last_became_peered":"2024-06-06T04:27:19.516458+0000","last_unstale":"2024-06-06T04:29:05.907896+0000","last_undegraded":"2024-06-06T04:29:05.907896+0000","last_fullsized":"2024-06-06T04:29:05.907896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:34:53.593825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380334+0000","last_change":"2024-06-06T04:27:22.564306+0000","last_active":"2024-06-06T04:30:04.380334+0000","last_peered":"2024-06-06T04:30:04.380334+0000","last_clean":"2024-06-06T04:30:04.380334+0000","last_became_active":"2024-06-06T04:27:22.563922+0000","last_became_peered":"2024-06-06T04:27:22.563922+0000","last_unstale":"2024-06-06T04:30:04.380334+0000","last_undegraded":"2024-06-06T04:30:04.380334+0000","last_fullsized":"2024-06-06T04:30:04.380334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:42:23.189047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904530+0000","last_change":"2024-06-06T04:27:32.617014+0000","last_active":"2024-06-06T04:29:05.904530+0000","last_peered":"2024-06-06T04:29:05.904530+0000","last_clean":"2024-06-06T04:29:05.904530+0000","last_became_active":"2024-06-06T04:27:32.616639+0000","last_became_peered":"2024-06-06T04:27:32.616639+0000","last_unstale":"2024-06-06T04:29:05.904530+0000","last_undegraded":"2024-06-06T04:29:05.904530+0000","last_fullsized":"2024-06-06T04:29:05.904530+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:54:52.034711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905627+0000","last_change":"2024-06-06T04:27:19.305000+0000","last_active":"2024-06-06T04:29:05.905627+0000","last_peered":"2024-06-06T04:29:05.905627+0000","last_clean":"2024-06-06T04:29:05.905627+0000","last_became_active":"2024-06-06T04:27:19.304676+0000","last_became_peered":"2024-06-06T04:27:19.304676+0000","last_unstale":"2024-06-06T04:29:05.905627+0000","last_undegraded":"2024-06-06T04:29:05.905627+0000","last_fullsized":"2024-06-06T04:29:05.905627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:19:34.054455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.382734+0000","last_change":"2024-06-06T04:27:22.572680+0000","last_active":"2024-06-06T04:30:04.382734+0000","last_peered":"2024-06-06T04:30:04.382734+0000","last_clean":"2024-06-06T04:30:04.382734+0000","last_became_active":"2024-06-06T04:27:22.572348+0000","last_became_peered":"2024-06-06T04:27:22.572348+0000","last_unstale":"2024-06-06T04:30:04.382734+0000","last_undegraded":"2024-06-06T04:30:04.382734+0000","last_fullsized":"2024-06-06T04:30:04.382734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:05:26.215771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904503+0000","last_change":"2024-06-06T04:27:32.611587+0000","last_active":"2024-06-06T04:29:05.904503+0000","last_peered":"2024-06-06T04:29:05.904503+0000","last_clean":"2024-06-06T04:29:05.904503+0000","last_became_active":"2024-06-06T04:27:32.611183+0000","last_became_peered":"2024-06-06T04:27:32.611183+0000","last_unstale":"2024-06-06T04:29:05.904503+0000","last_undegraded":"2024-06-06T04:29:05.904503+0000","last_fullsized":"2024-06-06T04:29:05.904503+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:36.773254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.371730+0000","last_change":"2024-06-06T04:29:06.403321+0000","last_active":"2024-06-06T04:30:04.371730+0000","last_peered":"2024-06-06T04:30:04.371730+0000","last_clean":"2024-06-06T04:30:04.371730+0000","last_became_active":"2024-06-06T04:29:06.402909+0000","last_became_peered":"2024-06-06T04:29:06.402909+0000","last_unstale":"2024-06-06T04:30:04.371730+0000","last_undegraded":"2024-06-06T04:30:04.371730+0000","last_fullsized":"2024-06-06T04:30:04.371730+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:29:38.963372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":220,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:17.206953+0000","last_change":"2024-06-06T04:27:19.516361+0000","last_active":"2024-06-06T04:31:17.206953+0000","last_peered":"2024-06-06T04:31:17.206953+0000","last_clean":"2024-06-06T04:31:17.206953+0000","last_became_active":"2024-06-06T04:27:19.515973+0000","last_became_peered":"2024-06-06T04:27:19.515973+0000","last_unstale":"2024-06-06T04:31:17.206953+0000","last_undegraded":"2024-06-06T04:31:17.206953+0000","last_fullsized":"2024-06-06T04:31:17.206953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:37.298819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":169,"num_read_kb":169,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905945+0000","last_change":"2024-06-06T04:27:32.630528+0000","last_active":"2024-06-06T04:29:05.905945+0000","last_peered":"2024-06-06T04:29:05.905945+0000","last_clean":"2024-06-06T04:29:05.905945+0000","last_became_active":"2024-06-06T04:27:32.630215+0000","last_became_peered":"2024-06-06T04:27:32.630215+0000","last_unstale":"2024-06-06T04:29:05.905945+0000","last_undegraded":"2024-06-06T04:29:05.905945+0000","last_fullsized":"2024-06-06T04:29:05.905945+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:34:29.980193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392719+0000","last_change":"2024-06-06T04:27:22.567466+0000","last_active":"2024-06-06T04:30:04.392719+0000","last_peered":"2024-06-06T04:30:04.392719+0000","last_clean":"2024-06-06T04:30:04.392719+0000","last_became_active":"2024-06-06T04:27:22.567145+0000","last_became_peered":"2024-06-06T04:27:22.567145+0000","last_unstale":"2024-06-06T04:30:04.392719+0000","last_undegraded":"2024-06-06T04:30:04.392719+0000","last_fullsized":"2024-06-06T04:30:04.392719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:56:56.203027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401783+0000","last_change":"2024-06-06T04:27:19.302726+0000","last_active":"2024-06-06T04:29:06.401783+0000","last_peered":"2024-06-06T04:29:06.401783+0000","last_clean":"2024-06-06T04:29:06.401783+0000","last_became_active":"2024-06-06T04:27:19.302329+0000","last_became_peered":"2024-06-06T04:27:19.302329+0000","last_unstale":"2024-06-06T04:29:06.401783+0000","last_undegraded":"2024-06-06T04:29:06.401783+0000","last_fullsized":"2024-06-06T04:29:06.401783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:55:15.010632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":367,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905237+0000","last_change":"2024-06-06T04:25:53.854188+0000","last_active":"2024-06-06T04:29:05.905237+0000","last_peered":"2024-06-06T04:29:05.905237+0000","last_clean":"2024-06-06T04:29:05.905237+0000","last_became_active":"2024-06-06T04:25:53.853700+0000","last_became_peered":"2024-06-06T04:25:53.853700+0000","last_unstale":"2024-06-06T04:29:05.905237+0000","last_undegraded":"2024-06-06T04:29:05.905237+0000","last_fullsized":"2024-06-06T04:29:05.905237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:22:28.847333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:22:28.847333+0000","last_clean_scrub_stamp":"2024-06-06T04:22:28.847333+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:59:10.246373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790675+0000","last_change":"2024-06-06T04:27:32.623300+0000","last_active":"2024-06-06T04:29:06.790675+0000","last_peered":"2024-06-06T04:29:06.790675+0000","last_clean":"2024-06-06T04:29:06.790675+0000","last_became_active":"2024-06-06T04:27:32.622811+0000","last_became_peered":"2024-06-06T04:27:32.622811+0000","last_unstale":"2024-06-06T04:29:06.790675+0000","last_undegraded":"2024-06-06T04:29:06.790675+0000","last_fullsized":"2024-06-06T04:29:06.790675+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:28.383079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.367134+0000","last_change":"2024-06-06T04:27:22.566617+0000","last_active":"2024-06-06T04:30:04.367134+0000","last_peered":"2024-06-06T04:30:04.367134+0000","last_clean":"2024-06-06T04:30:04.367134+0000","last_became_active":"2024-06-06T04:27:22.566256+0000","last_became_peered":"2024-06-06T04:27:22.566256+0000","last_unstale":"2024-06-06T04:30:04.367134+0000","last_undegraded":"2024-06-06T04:30:04.367134+0000","last_fullsized":"2024-06-06T04:30:04.367134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:50:41.318475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:18.419034+0000","last_change":"2024-06-06T04:27:19.305941+0000","last_active":"2024-06-06T04:29:18.419034+0000","last_peered":"2024-06-06T04:29:18.419034+0000","last_clean":"2024-06-06T04:29:18.419034+0000","last_became_active":"2024-06-06T04:27:19.305037+0000","last_became_peered":"2024-06-06T04:27:19.305037+0000","last_unstale":"2024-06-06T04:29:18.419034+0000","last_undegraded":"2024-06-06T04:29:18.419034+0000","last_fullsized":"2024-06-06T04:29:18.419034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:54.084427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908935+0000","last_change":"2024-06-06T04:27:32.621182+0000","last_active":"2024-06-06T04:29:05.908935+0000","last_peered":"2024-06-06T04:29:05.908935+0000","last_clean":"2024-06-06T04:29:05.908935+0000","last_became_active":"2024-06-06T04:27:32.620710+0000","last_became_peered":"2024-06-06T04:27:32.620710+0000","last_unstale":"2024-06-06T04:29:05.908935+0000","last_undegraded":"2024-06-06T04:29:05.908935+0000","last_fullsized":"2024-06-06T04:29:05.908935+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:05.512309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214246+0000","last_change":"2024-06-06T04:27:19.517033+0000","last_active":"2024-06-06T04:29:06.214246+0000","last_peered":"2024-06-06T04:29:06.214246+0000","last_clean":"2024-06-06T04:29:06.214246+0000","last_became_active":"2024-06-06T04:27:19.516278+0000","last_became_peered":"2024-06-06T04:27:19.516278+0000","last_unstale":"2024-06-06T04:29:06.214246+0000","last_undegraded":"2024-06-06T04:29:06.214246+0000","last_fullsized":"2024-06-06T04:29:06.214246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:11:20.336818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.368634+0000","last_change":"2024-06-06T04:27:22.566261+0000","last_active":"2024-06-06T04:30:04.368634+0000","last_peered":"2024-06-06T04:30:04.368634+0000","last_clean":"2024-06-06T04:30:04.368634+0000","last_became_active":"2024-06-06T04:27:22.565875+0000","last_became_peered":"2024-06-06T04:27:22.565875+0000","last_unstale":"2024-06-06T04:30:04.368634+0000","last_undegraded":"2024-06-06T04:30:04.368634+0000","last_fullsized":"2024-06-06T04:30:04.368634+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:49:55.923950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560350+0000","last_change":"2024-06-06T04:27:32.619854+0000","last_active":"2024-06-06T04:29:06.560350+0000","last_peered":"2024-06-06T04:29:06.560350+0000","last_clean":"2024-06-06T04:29:06.560350+0000","last_became_active":"2024-06-06T04:27:32.618765+0000","last_became_peered":"2024-06-06T04:27:32.618765+0000","last_unstale":"2024-06-06T04:29:06.560350+0000","last_undegraded":"2024-06-06T04:29:06.560350+0000","last_fullsized":"2024-06-06T04:29:06.560350+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:47:05.283751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'14","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:17.603468+0000","last_change":"2024-06-06T04:27:19.303057+0000","last_active":"2024-06-06T04:31:17.603468+0000","last_peered":"2024-06-06T04:31:17.603468+0000","last_clean":"2024-06-06T04:31:17.603468+0000","last_became_active":"2024-06-06T04:27:19.302716+0000","last_became_peered":"2024-06-06T04:27:19.302716+0000","last_unstale":"2024-06-06T04:31:17.603468+0000","last_undegraded":"2024-06-06T04:31:17.603468+0000","last_fullsized":"2024-06-06T04:31:17.603468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:35.273663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381463+0000","last_change":"2024-06-06T04:27:22.575274+0000","last_active":"2024-06-06T04:30:04.381463+0000","last_peered":"2024-06-06T04:30:04.381463+0000","last_clean":"2024-06-06T04:30:04.381463+0000","last_became_active":"2024-06-06T04:27:22.574979+0000","last_became_peered":"2024-06-06T04:27:22.574979+0000","last_unstale":"2024-06-06T04:30:04.381463+0000","last_undegraded":"2024-06-06T04:30:04.381463+0000","last_fullsized":"2024-06-06T04:30:04.381463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:30.718398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905413+0000","last_change":"2024-06-06T04:27:32.619939+0000","last_active":"2024-06-06T04:29:05.905413+0000","last_peered":"2024-06-06T04:29:05.905413+0000","last_clean":"2024-06-06T04:29:05.905413+0000","last_became_active":"2024-06-06T04:27:32.619499+0000","last_became_peered":"2024-06-06T04:27:32.619499+0000","last_unstale":"2024-06-06T04:29:05.905413+0000","last_undegraded":"2024-06-06T04:29:05.905413+0000","last_fullsized":"2024-06-06T04:29:05.905413+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:41:13.933546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402340+0000","last_change":"2024-06-06T04:27:19.397552+0000","last_active":"2024-06-06T04:29:06.402340+0000","last_peered":"2024-06-06T04:29:06.402340+0000","last_clean":"2024-06-06T04:29:06.402340+0000","last_became_active":"2024-06-06T04:27:19.397186+0000","last_became_peered":"2024-06-06T04:27:19.397186+0000","last_unstale":"2024-06-06T04:29:06.402340+0000","last_undegraded":"2024-06-06T04:29:06.402340+0000","last_fullsized":"2024-06-06T04:29:06.402340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:24:13.403860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.384537+0000","last_change":"2024-06-06T04:28:05.728212+0000","last_active":"2024-06-06T04:30:04.384537+0000","last_peered":"2024-06-06T04:30:04.384537+0000","last_clean":"2024-06-06T04:30:04.384537+0000","last_became_active":"2024-06-06T04:28:05.727748+0000","last_became_peered":"2024-06-06T04:28:05.727748+0000","last_unstale":"2024-06-06T04:30:04.384537+0000","last_undegraded":"2024-06-06T04:30:04.384537+0000","last_fullsized":"2024-06-06T04:30:04.384537+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:50.583716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402083+0000","last_change":"2024-06-06T04:28:05.896480+0000","last_active":"2024-06-06T04:29:06.402083+0000","last_peered":"2024-06-06T04:29:06.402083+0000","last_clean":"2024-06-06T04:29:06.402083+0000","last_became_active":"2024-06-06T04:28:05.895967+0000","last_became_peered":"2024-06-06T04:28:05.895967+0000","last_unstale":"2024-06-06T04:29:06.402083+0000","last_undegraded":"2024-06-06T04:29:06.402083+0000","last_fullsized":"2024-06-06T04:29:06.402083+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:26:19.846884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.907843+0000","last_change":"2024-06-06T04:27:19.516515+0000","last_active":"2024-06-06T04:29:06.907843+0000","last_peered":"2024-06-06T04:29:06.907843+0000","last_clean":"2024-06-06T04:29:06.907843+0000","last_became_active":"2024-06-06T04:27:19.516087+0000","last_became_peered":"2024-06-06T04:27:19.516087+0000","last_unstale":"2024-06-06T04:29:06.907843+0000","last_undegraded":"2024-06-06T04:29:06.907843+0000","last_fullsized":"2024-06-06T04:29:06.907843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:58.440226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":40,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.342173+0000","last_change":"2024-06-06T04:27:22.562367+0000","last_active":"2024-06-06T04:30:04.342173+0000","last_peered":"2024-06-06T04:30:04.342173+0000","last_clean":"2024-06-06T04:30:04.342173+0000","last_became_active":"2024-06-06T04:27:22.561947+0000","last_became_peered":"2024-06-06T04:27:22.561947+0000","last_unstale":"2024-06-06T04:30:04.342173+0000","last_undegraded":"2024-06-06T04:30:04.342173+0000","last_fullsized":"2024-06-06T04:30:04.342173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:28:06.200748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908183+0000","last_change":"2024-06-06T04:27:32.631577+0000","last_active":"2024-06-06T04:29:05.908183+0000","last_peered":"2024-06-06T04:29:05.908183+0000","last_clean":"2024-06-06T04:29:05.908183+0000","last_became_active":"2024-06-06T04:27:32.629555+0000","last_became_peered":"2024-06-06T04:27:32.629555+0000","last_unstale":"2024-06-06T04:29:05.908183+0000","last_undegraded":"2024-06-06T04:29:05.908183+0000","last_fullsized":"2024-06-06T04:29:05.908183+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:52:56.197281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560477+0000","last_change":"2024-06-06T04:27:19.298880+0000","last_active":"2024-06-06T04:29:06.560477+0000","last_peered":"2024-06-06T04:29:06.560477+0000","last_clean":"2024-06-06T04:29:06.560477+0000","last_became_active":"2024-06-06T04:27:19.298551+0000","last_became_peered":"2024-06-06T04:27:19.298551+0000","last_unstale":"2024-06-06T04:29:06.560477+0000","last_undegraded":"2024-06-06T04:29:06.560477+0000","last_fullsized":"2024-06-06T04:29:06.560477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:11:32.145401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.383001+0000","last_change":"2024-06-06T04:27:22.575065+0000","last_active":"2024-06-06T04:30:04.383001+0000","last_peered":"2024-06-06T04:30:04.383001+0000","last_clean":"2024-06-06T04:30:04.383001+0000","last_became_active":"2024-06-06T04:27:22.574469+0000","last_became_peered":"2024-06-06T04:27:22.574469+0000","last_unstale":"2024-06-06T04:30:04.383001+0000","last_undegraded":"2024-06-06T04:30:04.383001+0000","last_fullsized":"2024-06-06T04:30:04.383001+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:29:44.943833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402041+0000","last_change":"2024-06-06T04:27:32.626678+0000","last_active":"2024-06-06T04:29:06.402041+0000","last_peered":"2024-06-06T04:29:06.402041+0000","last_clean":"2024-06-06T04:29:06.402041+0000","last_became_active":"2024-06-06T04:27:32.626167+0000","last_became_peered":"2024-06-06T04:27:32.626167+0000","last_unstale":"2024-06-06T04:29:06.402041+0000","last_undegraded":"2024-06-06T04:29:06.402041+0000","last_fullsized":"2024-06-06T04:29:06.402041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:56:26.363195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905544+0000","last_change":"2024-06-06T04:27:19.306466+0000","last_active":"2024-06-06T04:29:05.905544+0000","last_peered":"2024-06-06T04:29:05.905544+0000","last_clean":"2024-06-06T04:29:05.905544+0000","last_became_active":"2024-06-06T04:27:19.303821+0000","last_became_peered":"2024-06-06T04:27:19.303821+0000","last_unstale":"2024-06-06T04:29:05.905544+0000","last_undegraded":"2024-06-06T04:29:05.905544+0000","last_fullsized":"2024-06-06T04:29:05.905544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:23:53.593998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.386917+0000","last_change":"2024-06-06T04:27:22.562662+0000","last_active":"2024-06-06T04:30:04.386917+0000","last_peered":"2024-06-06T04:30:04.386917+0000","last_clean":"2024-06-06T04:30:04.386917+0000","last_became_active":"2024-06-06T04:27:22.562318+0000","last_became_peered":"2024-06-06T04:27:22.562318+0000","last_unstale":"2024-06-06T04:30:04.386917+0000","last_undegraded":"2024-06-06T04:30:04.386917+0000","last_fullsized":"2024-06-06T04:30:04.386917+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:31:26.639275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402218+0000","last_change":"2024-06-06T04:27:32.619970+0000","last_active":"2024-06-06T04:29:06.402218+0000","last_peered":"2024-06-06T04:29:06.402218+0000","last_clean":"2024-06-06T04:29:06.402218+0000","last_became_active":"2024-06-06T04:27:32.619464+0000","last_became_peered":"2024-06-06T04:27:32.619464+0000","last_unstale":"2024-06-06T04:29:06.402218+0000","last_undegraded":"2024-06-06T04:29:06.402218+0000","last_fullsized":"2024-06-06T04:29:06.402218+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:44.844018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214056+0000","last_change":"2024-06-06T04:27:19.308172+0000","last_active":"2024-06-06T04:29:06.214056+0000","last_peered":"2024-06-06T04:29:06.214056+0000","last_clean":"2024-06-06T04:29:06.214056+0000","last_became_active":"2024-06-06T04:27:19.307780+0000","last_became_peered":"2024-06-06T04:27:19.307780+0000","last_unstale":"2024-06-06T04:29:06.214056+0000","last_undegraded":"2024-06-06T04:29:06.214056+0000","last_fullsized":"2024-06-06T04:29:06.214056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:39:11.653649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394812+0000","last_change":"2024-06-06T04:27:22.568881+0000","last_active":"2024-06-06T04:30:04.394812+0000","last_peered":"2024-06-06T04:30:04.394812+0000","last_clean":"2024-06-06T04:30:04.394812+0000","last_became_active":"2024-06-06T04:27:22.568489+0000","last_became_peered":"2024-06-06T04:27:22.568489+0000","last_unstale":"2024-06-06T04:30:04.394812+0000","last_undegraded":"2024-06-06T04:30:04.394812+0000","last_fullsized":"2024-06-06T04:30:04.394812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:00:22.339313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.907971+0000","last_change":"2024-06-06T04:27:32.622357+0000","last_active":"2024-06-06T04:29:06.907971+0000","last_peered":"2024-06-06T04:29:06.907971+0000","last_clean":"2024-06-06T04:29:06.907971+0000","last_became_active":"2024-06-06T04:27:32.621680+0000","last_became_peered":"2024-06-06T04:27:32.621680+0000","last_unstale":"2024-06-06T04:29:06.907971+0000","last_undegraded":"2024-06-06T04:29:06.907971+0000","last_fullsized":"2024-06-06T04:29:06.907971+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:00.677721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.906141+0000","last_change":"2024-06-06T04:27:19.516890+0000","last_active":"2024-06-06T04:29:05.906141+0000","last_peered":"2024-06-06T04:29:05.906141+0000","last_clean":"2024-06-06T04:29:05.906141+0000","last_became_active":"2024-06-06T04:27:19.516490+0000","last_became_peered":"2024-06-06T04:27:19.516490+0000","last_unstale":"2024-06-06T04:29:05.906141+0000","last_undegraded":"2024-06-06T04:29:05.906141+0000","last_fullsized":"2024-06-06T04:29:05.906141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:11:59.336250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.395341+0000","last_change":"2024-06-06T04:27:22.571003+0000","last_active":"2024-06-06T04:30:04.395341+0000","last_peered":"2024-06-06T04:30:04.395341+0000","last_clean":"2024-06-06T04:30:04.395341+0000","last_became_active":"2024-06-06T04:27:22.570442+0000","last_became_peered":"2024-06-06T04:27:22.570442+0000","last_unstale":"2024-06-06T04:30:04.395341+0000","last_undegraded":"2024-06-06T04:30:04.395341+0000","last_fullsized":"2024-06-06T04:30:04.395341+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:28:26.870429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.164730+0000","last_change":"2024-06-06T04:27:32.624424+0000","last_active":"2024-06-06T04:29:06.164730+0000","last_peered":"2024-06-06T04:29:06.164730+0000","last_clean":"2024-06-06T04:29:06.164730+0000","last_became_active":"2024-06-06T04:27:32.624026+0000","last_became_peered":"2024-06-06T04:27:32.624026+0000","last_unstale":"2024-06-06T04:29:06.164730+0000","last_undegraded":"2024-06-06T04:29:06.164730+0000","last_fullsized":"2024-06-06T04:29:06.164730+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:30:08.814284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790341+0000","last_change":"2024-06-06T04:27:19.301382+0000","last_active":"2024-06-06T04:29:06.790341+0000","last_peered":"2024-06-06T04:29:06.790341+0000","last_clean":"2024-06-06T04:29:06.790341+0000","last_became_active":"2024-06-06T04:27:19.300882+0000","last_became_peered":"2024-06-06T04:27:19.300882+0000","last_unstale":"2024-06-06T04:29:06.790341+0000","last_undegraded":"2024-06-06T04:29:06.790341+0000","last_fullsized":"2024-06-06T04:29:06.790341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:50.514103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392499+0000","last_change":"2024-06-06T04:27:22.570661+0000","last_active":"2024-06-06T04:30:04.392499+0000","last_peered":"2024-06-06T04:30:04.392499+0000","last_clean":"2024-06-06T04:30:04.392499+0000","last_became_active":"2024-06-06T04:27:22.570287+0000","last_became_peered":"2024-06-06T04:27:22.570287+0000","last_unstale":"2024-06-06T04:30:04.392499+0000","last_undegraded":"2024-06-06T04:30:04.392499+0000","last_fullsized":"2024-06-06T04:30:04.392499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:48.790492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.789984+0000","last_change":"2024-06-06T04:27:32.618385+0000","last_active":"2024-06-06T04:29:06.789984+0000","last_peered":"2024-06-06T04:29:06.789984+0000","last_clean":"2024-06-06T04:29:06.789984+0000","last_became_active":"2024-06-06T04:27:32.617956+0000","last_became_peered":"2024-06-06T04:27:32.617956+0000","last_unstale":"2024-06-06T04:29:06.789984+0000","last_undegraded":"2024-06-06T04:29:06.789984+0000","last_fullsized":"2024-06-06T04:29:06.789984+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:26.308657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904541+0000","last_change":"2024-06-06T04:28:05.728633+0000","last_active":"2024-06-06T04:29:05.904541+0000","last_peered":"2024-06-06T04:29:05.904541+0000","last_clean":"2024-06-06T04:29:05.904541+0000","last_became_active":"2024-06-06T04:28:05.722427+0000","last_became_peered":"2024-06-06T04:28:05.722427+0000","last_unstale":"2024-06-06T04:29:05.904541+0000","last_undegraded":"2024-06-06T04:29:05.904541+0000","last_fullsized":"2024-06-06T04:29:05.904541+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:13:22.061798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388201+0000","last_change":"2024-06-06T04:27:22.574950+0000","last_active":"2024-06-06T04:30:04.388201+0000","last_peered":"2024-06-06T04:30:04.388201+0000","last_clean":"2024-06-06T04:30:04.388201+0000","last_became_active":"2024-06-06T04:27:22.574604+0000","last_became_peered":"2024-06-06T04:27:22.574604+0000","last_unstale":"2024-06-06T04:30:04.388201+0000","last_undegraded":"2024-06-06T04:30:04.388201+0000","last_fullsized":"2024-06-06T04:30:04.388201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:17:29.395137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380448+0000","last_change":"2024-06-06T04:27:22.568554+0000","last_active":"2024-06-06T04:30:04.380448+0000","last_peered":"2024-06-06T04:30:04.380448+0000","last_clean":"2024-06-06T04:30:04.380448+0000","last_became_active":"2024-06-06T04:27:22.568196+0000","last_became_peered":"2024-06-06T04:27:22.568196+0000","last_unstale":"2024-06-06T04:30:04.380448+0000","last_undegraded":"2024-06-06T04:30:04.380448+0000","last_fullsized":"2024-06-06T04:30:04.380448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:11.564067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.372439+0000","last_change":"2024-06-06T04:27:22.565472+0000","last_active":"2024-06-06T04:30:04.372439+0000","last_peered":"2024-06-06T04:30:04.372439+0000","last_clean":"2024-06-06T04:30:04.372439+0000","last_became_active":"2024-06-06T04:27:22.565116+0000","last_became_peered":"2024-06-06T04:27:22.565116+0000","last_unstale":"2024-06-06T04:30:04.372439+0000","last_undegraded":"2024-06-06T04:30:04.372439+0000","last_fullsized":"2024-06-06T04:30:04.372439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:03:58.232707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.377900+0000","last_change":"2024-06-06T04:27:22.560210+0000","last_active":"2024-06-06T04:30:04.377900+0000","last_peered":"2024-06-06T04:30:04.377900+0000","last_clean":"2024-06-06T04:30:04.377900+0000","last_became_active":"2024-06-06T04:27:22.559853+0000","last_became_peered":"2024-06-06T04:27:22.559853+0000","last_unstale":"2024-06-06T04:30:04.377900+0000","last_undegraded":"2024-06-06T04:30:04.377900+0000","last_fullsized":"2024-06-06T04:30:04.377900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:56:18.650427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385809+0000","last_change":"2024-06-06T04:27:22.565071+0000","last_active":"2024-06-06T04:30:04.385809+0000","last_peered":"2024-06-06T04:30:04.385809+0000","last_clean":"2024-06-06T04:30:04.385809+0000","last_became_active":"2024-06-06T04:27:22.561411+0000","last_became_peered":"2024-06-06T04:27:22.561411+0000","last_unstale":"2024-06-06T04:30:04.385809+0000","last_undegraded":"2024-06-06T04:30:04.385809+0000","last_fullsized":"2024-06-06T04:30:04.385809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:41.948481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.382771+0000","last_change":"2024-06-06T04:27:22.563280+0000","last_active":"2024-06-06T04:30:04.382771+0000","last_peered":"2024-06-06T04:30:04.382771+0000","last_clean":"2024-06-06T04:30:04.382771+0000","last_became_active":"2024-06-06T04:27:22.563049+0000","last_became_peered":"2024-06-06T04:27:22.563049+0000","last_unstale":"2024-06-06T04:30:04.382771+0000","last_undegraded":"2024-06-06T04:30:04.382771+0000","last_fullsized":"2024-06-06T04:30:04.382771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:41:48.287969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.377210+0000","last_change":"2024-06-06T04:27:22.567691+0000","last_active":"2024-06-06T04:30:04.377210+0000","last_peered":"2024-06-06T04:30:04.377210+0000","last_clean":"2024-06-06T04:30:04.377210+0000","last_became_active":"2024-06-06T04:27:22.567331+0000","last_became_peered":"2024-06-06T04:27:22.567331+0000","last_unstale":"2024-06-06T04:30:04.377210+0000","last_undegraded":"2024-06-06T04:30:04.377210+0000","last_fullsized":"2024-06-06T04:30:04.377210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:03.886084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.393172+0000","last_change":"2024-06-06T04:27:22.563853+0000","last_active":"2024-06-06T04:30:04.393172+0000","last_peered":"2024-06-06T04:30:04.393172+0000","last_clean":"2024-06-06T04:30:04.393172+0000","last_became_active":"2024-06-06T04:27:22.563436+0000","last_became_peered":"2024-06-06T04:27:22.563436+0000","last_unstale":"2024-06-06T04:30:04.393172+0000","last_undegraded":"2024-06-06T04:30:04.393172+0000","last_fullsized":"2024-06-06T04:30:04.393172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:26.201308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.387060+0000","last_change":"2024-06-06T04:27:22.568189+0000","last_active":"2024-06-06T04:30:04.387060+0000","last_peered":"2024-06-06T04:30:04.387060+0000","last_clean":"2024-06-06T04:30:04.387060+0000","last_became_active":"2024-06-06T04:27:22.567815+0000","last_became_peered":"2024-06-06T04:27:22.567815+0000","last_unstale":"2024-06-06T04:30:04.387060+0000","last_undegraded":"2024-06-06T04:30:04.387060+0000","last_fullsized":"2024-06-06T04:30:04.387060+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:13:39.259012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.359906+0000","last_change":"2024-06-06T04:27:22.569698+0000","last_active":"2024-06-06T04:30:04.359906+0000","last_peered":"2024-06-06T04:30:04.359906+0000","last_clean":"2024-06-06T04:30:04.359906+0000","last_became_active":"2024-06-06T04:27:22.569347+0000","last_became_peered":"2024-06-06T04:27:22.569347+0000","last_unstale":"2024-06-06T04:30:04.359906+0000","last_undegraded":"2024-06-06T04:30:04.359906+0000","last_fullsized":"2024-06-06T04:30:04.359906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:06:34.722798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381456+0000","last_change":"2024-06-06T04:27:22.561149+0000","last_active":"2024-06-06T04:30:04.381456+0000","last_peered":"2024-06-06T04:30:04.381456+0000","last_clean":"2024-06-06T04:30:04.381456+0000","last_became_active":"2024-06-06T04:27:22.560793+0000","last_became_peered":"2024-06-06T04:27:22.560793+0000","last_unstale":"2024-06-06T04:30:04.381456+0000","last_undegraded":"2024-06-06T04:30:04.381456+0000","last_fullsized":"2024-06-06T04:30:04.381456+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:00:41.563443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394791+0000","last_change":"2024-06-06T04:27:22.569331+0000","last_active":"2024-06-06T04:30:04.394791+0000","last_peered":"2024-06-06T04:30:04.394791+0000","last_clean":"2024-06-06T04:30:04.394791+0000","last_became_active":"2024-06-06T04:27:22.569021+0000","last_became_peered":"2024-06-06T04:27:22.569021+0000","last_unstale":"2024-06-06T04:30:04.394791+0000","last_undegraded":"2024-06-06T04:30:04.394791+0000","last_fullsized":"2024-06-06T04:30:04.394791+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:52:37.064317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.391532+0000","last_change":"2024-06-06T04:27:22.574558+0000","last_active":"2024-06-06T04:30:04.391532+0000","last_peered":"2024-06-06T04:30:04.391532+0000","last_clean":"2024-06-06T04:30:04.391532+0000","last_became_active":"2024-06-06T04:27:22.574215+0000","last_became_peered":"2024-06-06T04:27:22.574215+0000","last_unstale":"2024-06-06T04:30:04.391532+0000","last_undegraded":"2024-06-06T04:30:04.391532+0000","last_fullsized":"2024-06-06T04:30:04.391532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:40:52.496322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385148+0000","last_change":"2024-06-06T04:27:22.565320+0000","last_active":"2024-06-06T04:30:04.385148+0000","last_peered":"2024-06-06T04:30:04.385148+0000","last_clean":"2024-06-06T04:30:04.385148+0000","last_became_active":"2024-06-06T04:27:22.564786+0000","last_became_peered":"2024-06-06T04:27:22.564786+0000","last_unstale":"2024-06-06T04:30:04.385148+0000","last_undegraded":"2024-06-06T04:30:04.385148+0000","last_fullsized":"2024-06-06T04:30:04.385148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:19:42.057666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249851,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78096,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1059,"num_read_kb":613,"num_write":189,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":219060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351040,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":572909,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612678,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28244,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716908,"statfs":{"total":95995035648,"available":95966113792,"internally_reserved":0,"allocated":1257472,"data_stored":1233781,"data_compressed":68587,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776203,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":577241,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037839,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573065,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299476,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":1170098,"data_compressed":68587,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463002,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":568321,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":633588,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":569709,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1191936,"data_stored":634446,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149801,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":568515,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888163,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":573994,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724639,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1175552,"data_stored":1165053,"data_compressed":68587,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34523,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34523,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T04:31:24.273 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-06T04:31:24.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:23 smithi086 ceph-mon[36053]: pgmap v429: 129 pgs: 129 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-06-06T04:31:24.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:23 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3615005037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:31:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.411+0000 7fbc835e9640 1 -- 172.21.15.45:0/1972795399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc7c1024d0 msgr2=0x7fbc7c1048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.411+0000 7fbc835e9640 1 --2- 172.21.15.45:0/1972795399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc7c1024d0 0x7fbc7c1048c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009900 tx=0x7fbc7802a410 comp rx=0 tx=0).stop 2024-06-06T04:31:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.411+0000 7fbc835e9640 1 -- 172.21.15.45:0/1972795399 shutdown_connections 2024-06-06T04:31:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.411+0000 7fbc835e9640 1 --2- 172.21.15.45:0/1972795399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc7c107730 0x7fbc7c109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.411+0000 7fbc835e9640 1 --2- 172.21.15.45:0/1972795399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc7c104e00 0x7fbc7c1071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.411+0000 7fbc835e9640 1 --2- 172.21.15.45:0/1972795399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc7c1024d0 0x7fbc7c1048c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.411+0000 7fbc835e9640 1 -- 172.21.15.45:0/1972795399 >> 172.21.15.45:0/1972795399 conn(0x7fbc7c0fbe80 msgr2=0x7fbc7c0fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.411+0000 7fbc835e9640 1 -- 172.21.15.45:0/1972795399 shutdown_connections 2024-06-06T04:31:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.411+0000 7fbc835e9640 1 -- 172.21.15.45:0/1972795399 wait complete. 2024-06-06T04:31:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc835e9640 1 Processor -- start 2024-06-06T04:31:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc835e9640 1 -- start start 2024-06-06T04:31:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc835e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc7c1024d0 0x7fbc7c197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc835e9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc7c104e00 0x7fbc7c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc835e9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc7c107730 0x7fbc7c19f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc835e9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c10d540 con 0x7fbc7c1024d0 2024-06-06T04:31:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc835e9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c10d3c0 con 0x7fbc7c107730 2024-06-06T04:31:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc835e9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c10d6c0 con 0x7fbc7c104e00 2024-06-06T04:31:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc81385640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc7c1024d0 0x7fbc7c197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc81385640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc7c1024d0 0x7fbc7c197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41188/0 (socket says 172.21.15.45:41188) 2024-06-06T04:31:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc81385640 1 -- 172.21.15.45:0/1663474316 learned_addr learned my addr 172.21.15.45:0/1663474316 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc80b84640 1 --2- 172.21.15.45:0/1663474316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc7c104e00 0x7fbc7c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc81b86640 1 --2- 172.21.15.45:0/1663474316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc7c107730 0x7fbc7c19f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc81385640 1 -- 172.21.15.45:0/1663474316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc7c104e00 msgr2=0x7fbc7c198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc81385640 1 --2- 172.21.15.45:0/1663474316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc7c104e00 0x7fbc7c198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc81385640 1 -- 172.21.15.45:0/1663474316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc7c107730 msgr2=0x7fbc7c19f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc81385640 1 --2- 172.21.15.45:0/1663474316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc7c107730 0x7fbc7c19f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc81385640 1 -- 172.21.15.45:0/1663474316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7c19f8b0 con 0x7fbc7c1024d0 2024-06-06T04:31:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.412+0000 7fbc81385640 1 --2- 172.21.15.45:0/1663474316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc7c1024d0 0x7fbc7c197cc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbc78004770 tx=0x7fbc780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.413+0000 7fbc727fc640 1 -- 172.21.15.45:0/1663474316 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7803f030 con 0x7fbc7c1024d0 2024-06-06T04:31:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.413+0000 7fbc727fc640 1 -- 172.21.15.45:0/1663474316 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc78003cb0 con 0x7fbc7c1024d0 2024-06-06T04:31:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.413+0000 7fbc835e9640 1 -- 172.21.15.45:0/1663474316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7c19fb40 con 0x7fbc7c1024d0 2024-06-06T04:31:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.413+0000 7fbc727fc640 1 -- 172.21.15.45:0/1663474316 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc78035070 con 0x7fbc7c1024d0 2024-06-06T04:31:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.413+0000 7fbc835e9640 1 -- 172.21.15.45:0/1663474316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc7c1a0030 con 0x7fbc7c1024d0 2024-06-06T04:31:24.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.414+0000 7fbc835e9640 1 -- 172.21.15.45:0/1663474316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc7c0fd970 con 0x7fbc7c1024d0 2024-06-06T04:31:24.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.415+0000 7fbc727fc640 1 -- 172.21.15.45:0/1663474316 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fbc78042030 con 0x7fbc7c1024d0 2024-06-06T04:31:24.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.415+0000 7fbc727fc640 1 --2- 172.21.15.45:0/1663474316 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbc680a2c20 0x7fbc680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:24.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.416+0000 7fbc727fc640 1 -- 172.21.15.45:0/1663474316 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fbc780f0130 con 0x7fbc7c1024d0 2024-06-06T04:31:24.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.416+0000 7fbc80b84640 1 --2- 172.21.15.45:0/1663474316 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbc680a2c20 0x7fbc680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:24.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.416+0000 7fbc80b84640 1 --2- 172.21.15.45:0/1663474316 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbc680a2c20 0x7fbc680a50e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c199170 tx=0x7fbc6c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:24.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.419+0000 7fbc727fc640 1 -- 172.21.15.45:0/1663474316 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbc780bcb70 con 0x7fbc7c1024d0 2024-06-06T04:31:24.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:23 smithi154 ceph-mon[36428]: pgmap v429: 129 pgs: 129 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-06-06T04:31:24.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:23 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3615005037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:31:24.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.602+0000 7fbc835e9640 1 -- 172.21.15.45:0/1663474316 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbc7c061de0 con 0x7fbc680a2c20 2024-06-06T04:31:24.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.611+0000 7fbc727fc640 1 -- 172.21.15.45:0/1663474316 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340942 (secure 0 0 0) 0x7fbc7c061de0 con 0x7fbc680a2c20 2024-06-06T04:31:24.616 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:31:24.618 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-06-06T04:31:24.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.615+0000 7fbc835e9640 1 -- 172.21.15.45:0/1663474316 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbc680a2c20 msgr2=0x7fbc680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:24.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.615+0000 7fbc835e9640 1 --2- 172.21.15.45:0/1663474316 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbc680a2c20 0x7fbc680a50e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c199170 tx=0x7fbc6c00b390 comp rx=0 tx=0).stop 2024-06-06T04:31:24.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.615+0000 7fbc835e9640 1 -- 172.21.15.45:0/1663474316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc7c1024d0 msgr2=0x7fbc7c197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:24.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.615+0000 7fbc835e9640 1 --2- 172.21.15.45:0/1663474316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc7c1024d0 0x7fbc7c197cc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbc78004770 tx=0x7fbc780042f0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.615+0000 7fbc835e9640 1 -- 172.21.15.45:0/1663474316 shutdown_connections 2024-06-06T04:31:24.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.615+0000 7fbc835e9640 1 --2- 172.21.15.45:0/1663474316 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbc680a2c20 0x7fbc680a50e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.615+0000 7fbc835e9640 1 --2- 172.21.15.45:0/1663474316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc7c107730 0x7fbc7c19f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.615+0000 7fbc835e9640 1 --2- 172.21.15.45:0/1663474316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc7c104e00 0x7fbc7c198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.615+0000 7fbc835e9640 1 --2- 172.21.15.45:0/1663474316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc7c1024d0 0x7fbc7c197cc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.615+0000 7fbc835e9640 1 -- 172.21.15.45:0/1663474316 >> 172.21.15.45:0/1663474316 conn(0x7fbc7c0fbe80 msgr2=0x7fbc7c1032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:24.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.615+0000 7fbc835e9640 1 -- 172.21.15.45:0/1663474316 shutdown_connections 2024-06-06T04:31:24.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.615+0000 7fbc835e9640 1 -- 172.21.15.45:0/1663474316 wait complete. 2024-06-06T04:31:24.628 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-06-06T04:31:24.425531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668698,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1196,"num_read_kb":728,"num_write":374,"num_write_kb":3083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":802,"ondisk_log_size":802,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375068,"kb_used_data":13436,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566756,"statfs":{"total":1151940427776,"available":1151556358144,"internally_reserved":0,"allocated":13758464,"data_stored":8856932,"data_compressed":497595,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9090,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":47,"num_write":15,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003490"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.386822+0000","last_change":"2024-06-06T04:27:22.574900+0000","last_active":"2024-06-06T04:30:04.386822+0000","last_peered":"2024-06-06T04:30:04.386822+0000","last_clean":"2024-06-06T04:30:04.386822+0000","last_became_active":"2024-06-06T04:27:22.574389+0000","last_became_peered":"2024-06-06T04:27:22.574389+0000","last_unstale":"2024-06-06T04:30:04.386822+0000","last_undegraded":"2024-06-06T04:30:04.386822+0000","last_fullsized":"2024-06-06T04:30:04.386822+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:39.246463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.382712+0000","last_change":"2024-06-06T04:27:22.565543+0000","last_active":"2024-06-06T04:30:04.382712+0000","last_peered":"2024-06-06T04:30:04.382712+0000","last_clean":"2024-06-06T04:30:04.382712+0000","last_became_active":"2024-06-06T04:27:22.563833+0000","last_became_peered":"2024-06-06T04:27:22.563833+0000","last_unstale":"2024-06-06T04:30:04.382712+0000","last_undegraded":"2024-06-06T04:30:04.382712+0000","last_fullsized":"2024-06-06T04:30:04.382712+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:26:17.810782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388761+0000","last_change":"2024-06-06T04:27:22.570107+0000","last_active":"2024-06-06T04:30:04.388761+0000","last_peered":"2024-06-06T04:30:04.388761+0000","last_clean":"2024-06-06T04:30:04.388761+0000","last_became_active":"2024-06-06T04:27:22.569788+0000","last_became_peered":"2024-06-06T04:27:22.569788+0000","last_unstale":"2024-06-06T04:30:04.388761+0000","last_undegraded":"2024-06-06T04:30:04.388761+0000","last_fullsized":"2024-06-06T04:30:04.388761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:48:20.454782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385267+0000","last_change":"2024-06-06T04:27:22.559431+0000","last_active":"2024-06-06T04:30:04.385267+0000","last_peered":"2024-06-06T04:30:04.385267+0000","last_clean":"2024-06-06T04:30:04.385267+0000","last_became_active":"2024-06-06T04:27:22.558961+0000","last_became_peered":"2024-06-06T04:27:22.558961+0000","last_unstale":"2024-06-06T04:30:04.385267+0000","last_undegraded":"2024-06-06T04:30:04.385267+0000","last_fullsized":"2024-06-06T04:30:04.385267+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:59.445283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.359959+0000","last_change":"2024-06-06T04:27:22.561598+0000","last_active":"2024-06-06T04:30:04.359959+0000","last_peered":"2024-06-06T04:30:04.359959+0000","last_clean":"2024-06-06T04:30:04.359959+0000","last_became_active":"2024-06-06T04:27:22.561198+0000","last_became_peered":"2024-06-06T04:27:22.561198+0000","last_unstale":"2024-06-06T04:30:04.359959+0000","last_undegraded":"2024-06-06T04:30:04.359959+0000","last_fullsized":"2024-06-06T04:30:04.359959+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:20:28.314941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380904+0000","last_change":"2024-06-06T04:27:22.569395+0000","last_active":"2024-06-06T04:30:04.380904+0000","last_peered":"2024-06-06T04:30:04.380904+0000","last_clean":"2024-06-06T04:30:04.380904+0000","last_became_active":"2024-06-06T04:27:22.569054+0000","last_became_peered":"2024-06-06T04:27:22.569054+0000","last_unstale":"2024-06-06T04:30:04.380904+0000","last_undegraded":"2024-06-06T04:30:04.380904+0000","last_fullsized":"2024-06-06T04:30:04.380904+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:03:56.630677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392227+0000","last_change":"2024-06-06T04:27:22.577121+0000","last_active":"2024-06-06T04:30:04.392227+0000","last_peered":"2024-06-06T04:30:04.392227+0000","last_clean":"2024-06-06T04:30:04.392227+0000","last_became_active":"2024-06-06T04:27:22.575574+0000","last_became_peered":"2024-06-06T04:27:22.575574+0000","last_unstale":"2024-06-06T04:30:04.392227+0000","last_undegraded":"2024-06-06T04:30:04.392227+0000","last_fullsized":"2024-06-06T04:30:04.392227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:05:16.799335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.383226+0000","last_change":"2024-06-06T04:27:22.570051+0000","last_active":"2024-06-06T04:30:04.383226+0000","last_peered":"2024-06-06T04:30:04.383226+0000","last_clean":"2024-06-06T04:30:04.383226+0000","last_became_active":"2024-06-06T04:27:22.569640+0000","last_became_peered":"2024-06-06T04:27:22.569640+0000","last_unstale":"2024-06-06T04:30:04.383226+0000","last_undegraded":"2024-06-06T04:30:04.383226+0000","last_fullsized":"2024-06-06T04:30:04.383226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:48:00.744167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.396353+0000","last_change":"2024-06-06T04:27:22.574900+0000","last_active":"2024-06-06T04:30:04.396353+0000","last_peered":"2024-06-06T04:30:04.396353+0000","last_clean":"2024-06-06T04:30:04.396353+0000","last_became_active":"2024-06-06T04:27:22.574570+0000","last_became_peered":"2024-06-06T04:27:22.574570+0000","last_unstale":"2024-06-06T04:30:04.396353+0000","last_undegraded":"2024-06-06T04:30:04.396353+0000","last_fullsized":"2024-06-06T04:30:04.396353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:54.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380728+0000","last_change":"2024-06-06T04:27:22.570581+0000","last_active":"2024-06-06T04:30:04.380728+0000","last_peered":"2024-06-06T04:30:04.380728+0000","last_clean":"2024-06-06T04:30:04.380728+0000","last_became_active":"2024-06-06T04:27:22.570243+0000","last_became_peered":"2024-06-06T04:27:22.570243+0000","last_unstale":"2024-06-06T04:30:04.380728+0000","last_undegraded":"2024-06-06T04:30:04.380728+0000","last_fullsized":"2024-06-06T04:30:04.380728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:27.087527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.395227+0000","last_change":"2024-06-06T04:27:22.571162+0000","last_active":"2024-06-06T04:30:04.395227+0000","last_peered":"2024-06-06T04:30:04.395227+0000","last_clean":"2024-06-06T04:30:04.395227+0000","last_became_active":"2024-06-06T04:27:22.570843+0000","last_became_peered":"2024-06-06T04:27:22.570843+0000","last_unstale":"2024-06-06T04:30:04.395227+0000","last_undegraded":"2024-06-06T04:30:04.395227+0000","last_fullsized":"2024-06-06T04:30:04.395227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:43:05.590240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.384953+0000","last_change":"2024-06-06T04:27:22.567420+0000","last_active":"2024-06-06T04:30:04.384953+0000","last_peered":"2024-06-06T04:30:04.384953+0000","last_clean":"2024-06-06T04:30:04.384953+0000","last_became_active":"2024-06-06T04:27:22.566750+0000","last_became_peered":"2024-06-06T04:27:22.566750+0000","last_unstale":"2024-06-06T04:30:04.384953+0000","last_undegraded":"2024-06-06T04:30:04.384953+0000","last_fullsized":"2024-06-06T04:30:04.384953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:21.669965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.364827+0000","last_change":"2024-06-06T04:27:22.566831+0000","last_active":"2024-06-06T04:30:04.364827+0000","last_peered":"2024-06-06T04:30:04.364827+0000","last_clean":"2024-06-06T04:30:04.364827+0000","last_became_active":"2024-06-06T04:27:22.566255+0000","last_became_peered":"2024-06-06T04:27:22.566255+0000","last_unstale":"2024-06-06T04:30:04.364827+0000","last_undegraded":"2024-06-06T04:30:04.364827+0000","last_fullsized":"2024-06-06T04:30:04.364827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:40:22.486674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.384669+0000","last_change":"2024-06-06T04:27:22.564596+0000","last_active":"2024-06-06T04:30:04.384669+0000","last_peered":"2024-06-06T04:30:04.384669+0000","last_clean":"2024-06-06T04:30:04.384669+0000","last_became_active":"2024-06-06T04:27:22.563694+0000","last_became_peered":"2024-06-06T04:27:22.563694+0000","last_unstale":"2024-06-06T04:30:04.384669+0000","last_undegraded":"2024-06-06T04:30:04.384669+0000","last_fullsized":"2024-06-06T04:30:04.384669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:18.568266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385298+0000","last_change":"2024-06-06T04:27:22.559395+0000","last_active":"2024-06-06T04:30:04.385298+0000","last_peered":"2024-06-06T04:30:04.385298+0000","last_clean":"2024-06-06T04:30:04.385298+0000","last_became_active":"2024-06-06T04:27:22.559030+0000","last_became_peered":"2024-06-06T04:27:22.559030+0000","last_unstale":"2024-06-06T04:30:04.385298+0000","last_undegraded":"2024-06-06T04:30:04.385298+0000","last_fullsized":"2024-06-06T04:30:04.385298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:08.562184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394517+0000","last_change":"2024-06-06T04:27:22.569657+0000","last_active":"2024-06-06T04:30:04.394517+0000","last_peered":"2024-06-06T04:30:04.394517+0000","last_clean":"2024-06-06T04:30:04.394517+0000","last_became_active":"2024-06-06T04:27:22.569359+0000","last_became_peered":"2024-06-06T04:27:22.569359+0000","last_unstale":"2024-06-06T04:30:04.394517+0000","last_undegraded":"2024-06-06T04:30:04.394517+0000","last_fullsized":"2024-06-06T04:30:04.394517+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:42:20.197715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394347+0000","last_change":"2024-06-06T04:27:22.574661+0000","last_active":"2024-06-06T04:30:04.394347+0000","last_peered":"2024-06-06T04:30:04.394347+0000","last_clean":"2024-06-06T04:30:04.394347+0000","last_became_active":"2024-06-06T04:27:22.574310+0000","last_became_peered":"2024-06-06T04:27:22.574310+0000","last_unstale":"2024-06-06T04:30:04.394347+0000","last_undegraded":"2024-06-06T04:30:04.394347+0000","last_fullsized":"2024-06-06T04:30:04.394347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:45:18.772270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388153+0000","last_change":"2024-06-06T04:27:22.565575+0000","last_active":"2024-06-06T04:30:04.388153+0000","last_peered":"2024-06-06T04:30:04.388153+0000","last_clean":"2024-06-06T04:30:04.388153+0000","last_became_active":"2024-06-06T04:27:22.565256+0000","last_became_peered":"2024-06-06T04:27:22.565256+0000","last_unstale":"2024-06-06T04:30:04.388153+0000","last_undegraded":"2024-06-06T04:30:04.388153+0000","last_fullsized":"2024-06-06T04:30:04.388153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:49.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.365082+0000","last_change":"2024-06-06T04:27:22.559606+0000","last_active":"2024-06-06T04:30:04.365082+0000","last_peered":"2024-06-06T04:30:04.365082+0000","last_clean":"2024-06-06T04:30:04.365082+0000","last_became_active":"2024-06-06T04:27:22.557881+0000","last_became_peered":"2024-06-06T04:27:22.557881+0000","last_unstale":"2024-06-06T04:30:04.365082+0000","last_undegraded":"2024-06-06T04:30:04.365082+0000","last_fullsized":"2024-06-06T04:30:04.365082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:31:06.269785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905476+0000","last_change":"2024-06-06T04:27:32.626612+0000","last_active":"2024-06-06T04:29:05.905476+0000","last_peered":"2024-06-06T04:29:05.905476+0000","last_clean":"2024-06-06T04:29:05.905476+0000","last_became_active":"2024-06-06T04:27:32.626113+0000","last_became_peered":"2024-06-06T04:27:32.626113+0000","last_unstale":"2024-06-06T04:29:05.905476+0000","last_undegraded":"2024-06-06T04:29:05.905476+0000","last_fullsized":"2024-06-06T04:29:05.905476+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.318738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908330+0000","last_change":"2024-06-06T04:27:19.302744+0000","last_active":"2024-06-06T04:29:05.908330+0000","last_peered":"2024-06-06T04:29:05.908330+0000","last_clean":"2024-06-06T04:29:05.908330+0000","last_became_active":"2024-06-06T04:27:19.302389+0000","last_became_peered":"2024-06-06T04:27:19.302389+0000","last_unstale":"2024-06-06T04:29:05.908330+0000","last_undegraded":"2024-06-06T04:29:05.908330+0000","last_fullsized":"2024-06-06T04:29:05.908330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:49.872618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381219+0000","last_change":"2024-06-06T04:27:22.570220+0000","last_active":"2024-06-06T04:30:04.381219+0000","last_peered":"2024-06-06T04:30:04.381219+0000","last_clean":"2024-06-06T04:30:04.381219+0000","last_became_active":"2024-06-06T04:27:22.569835+0000","last_became_peered":"2024-06-06T04:27:22.569835+0000","last_unstale":"2024-06-06T04:30:04.381219+0000","last_undegraded":"2024-06-06T04:30:04.381219+0000","last_fullsized":"2024-06-06T04:30:04.381219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:54:57.372561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401940+0000","last_change":"2024-06-06T04:27:32.622407+0000","last_active":"2024-06-06T04:29:06.401940+0000","last_peered":"2024-06-06T04:29:06.401940+0000","last_clean":"2024-06-06T04:29:06.401940+0000","last_became_active":"2024-06-06T04:27:32.621868+0000","last_became_peered":"2024-06-06T04:27:32.621868+0000","last_unstale":"2024-06-06T04:29:06.401940+0000","last_undegraded":"2024-06-06T04:29:06.401940+0000","last_fullsized":"2024-06-06T04:29:06.401940+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:50:05.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:21.905209+0000","last_change":"2024-06-06T04:27:19.516235+0000","last_active":"2024-06-06T04:29:21.905209+0000","last_peered":"2024-06-06T04:29:21.905209+0000","last_clean":"2024-06-06T04:29:21.905209+0000","last_became_active":"2024-06-06T04:27:19.515849+0000","last_became_peered":"2024-06-06T04:27:19.515849+0000","last_unstale":"2024-06-06T04:29:21.905209+0000","last_undegraded":"2024-06-06T04:29:21.905209+0000","last_fullsized":"2024-06-06T04:29:21.905209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:52:42.722103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388234+0000","last_change":"2024-06-06T04:27:22.570797+0000","last_active":"2024-06-06T04:30:04.388234+0000","last_peered":"2024-06-06T04:30:04.388234+0000","last_clean":"2024-06-06T04:30:04.388234+0000","last_became_active":"2024-06-06T04:27:22.570305+0000","last_became_peered":"2024-06-06T04:27:22.570305+0000","last_unstale":"2024-06-06T04:30:04.388234+0000","last_undegraded":"2024-06-06T04:30:04.388234+0000","last_fullsized":"2024-06-06T04:30:04.388234+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:04:14.545306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401666+0000","last_change":"2024-06-06T04:27:32.623241+0000","last_active":"2024-06-06T04:29:06.401666+0000","last_peered":"2024-06-06T04:29:06.401666+0000","last_clean":"2024-06-06T04:29:06.401666+0000","last_became_active":"2024-06-06T04:27:32.622751+0000","last_became_peered":"2024-06-06T04:27:32.622751+0000","last_unstale":"2024-06-06T04:29:06.401666+0000","last_undegraded":"2024-06-06T04:29:06.401666+0000","last_fullsized":"2024-06-06T04:29:06.401666+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:24:53.446329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.909112+0000","last_change":"2024-06-06T04:28:05.733350+0000","last_active":"2024-06-06T04:29:05.909112+0000","last_peered":"2024-06-06T04:29:05.909112+0000","last_clean":"2024-06-06T04:29:05.909112+0000","last_became_active":"2024-06-06T04:28:05.726556+0000","last_became_peered":"2024-06-06T04:28:05.726556+0000","last_unstale":"2024-06-06T04:29:05.909112+0000","last_undegraded":"2024-06-06T04:29:05.909112+0000","last_fullsized":"2024-06-06T04:29:05.909112+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:26.636625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.396094+0000","last_change":"2024-06-06T04:27:22.571731+0000","last_active":"2024-06-06T04:30:04.396094+0000","last_peered":"2024-06-06T04:30:04.396094+0000","last_clean":"2024-06-06T04:30:04.396094+0000","last_became_active":"2024-06-06T04:27:22.571398+0000","last_became_peered":"2024-06-06T04:27:22.571398+0000","last_unstale":"2024-06-06T04:30:04.396094+0000","last_undegraded":"2024-06-06T04:30:04.396094+0000","last_fullsized":"2024-06-06T04:30:04.396094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:22:45.144427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401661+0000","last_change":"2024-06-06T04:27:32.618465+0000","last_active":"2024-06-06T04:29:06.401661+0000","last_peered":"2024-06-06T04:29:06.401661+0000","last_clean":"2024-06-06T04:29:06.401661+0000","last_became_active":"2024-06-06T04:27:32.618066+0000","last_became_peered":"2024-06-06T04:27:32.618066+0000","last_unstale":"2024-06-06T04:29:06.401661+0000","last_undegraded":"2024-06-06T04:29:06.401661+0000","last_fullsized":"2024-06-06T04:29:06.401661+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:54:23.988687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908783+0000","last_change":"2024-06-06T04:28:05.730381+0000","last_active":"2024-06-06T04:29:05.908783+0000","last_peered":"2024-06-06T04:29:05.908783+0000","last_clean":"2024-06-06T04:29:05.908783+0000","last_became_active":"2024-06-06T04:28:05.726552+0000","last_became_peered":"2024-06-06T04:28:05.726552+0000","last_unstale":"2024-06-06T04:29:05.908783+0000","last_undegraded":"2024-06-06T04:29:05.908783+0000","last_fullsized":"2024-06-06T04:29:05.908783+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:52.903042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394789+0000","last_change":"2024-06-06T04:27:22.573769+0000","last_active":"2024-06-06T04:30:04.394789+0000","last_peered":"2024-06-06T04:30:04.394789+0000","last_clean":"2024-06-06T04:30:04.394789+0000","last_became_active":"2024-06-06T04:27:22.572674+0000","last_became_peered":"2024-06-06T04:27:22.572674+0000","last_unstale":"2024-06-06T04:30:04.394789+0000","last_undegraded":"2024-06-06T04:30:04.394789+0000","last_fullsized":"2024-06-06T04:30:04.394789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:57:22.653052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905675+0000","last_change":"2024-06-06T04:27:32.617934+0000","last_active":"2024-06-06T04:29:05.905675+0000","last_peered":"2024-06-06T04:29:05.905675+0000","last_clean":"2024-06-06T04:29:05.905675+0000","last_became_active":"2024-06-06T04:27:32.617537+0000","last_became_peered":"2024-06-06T04:27:32.617537+0000","last_unstale":"2024-06-06T04:29:05.905675+0000","last_undegraded":"2024-06-06T04:29:05.905675+0000","last_fullsized":"2024-06-06T04:29:05.905675+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:21:48.319434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'7","reported_seq":228,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:19.968167+0000","last_change":"2024-06-06T04:27:19.306236+0000","last_active":"2024-06-06T04:31:19.968167+0000","last_peered":"2024-06-06T04:31:19.968167+0000","last_clean":"2024-06-06T04:31:19.968167+0000","last_became_active":"2024-06-06T04:27:19.305850+0000","last_became_peered":"2024-06-06T04:27:19.305850+0000","last_unstale":"2024-06-06T04:31:19.968167+0000","last_undegraded":"2024-06-06T04:31:19.968167+0000","last_fullsized":"2024-06-06T04:31:19.968167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:58:47.310040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1504,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.365431+0000","last_change":"2024-06-06T04:27:22.567440+0000","last_active":"2024-06-06T04:30:04.365431+0000","last_peered":"2024-06-06T04:30:04.365431+0000","last_clean":"2024-06-06T04:30:04.365431+0000","last_became_active":"2024-06-06T04:27:22.566732+0000","last_became_peered":"2024-06-06T04:27:22.566732+0000","last_unstale":"2024-06-06T04:30:04.365431+0000","last_undegraded":"2024-06-06T04:30:04.365431+0000","last_fullsized":"2024-06-06T04:30:04.365431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:48:46.270103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905710+0000","last_change":"2024-06-06T04:27:32.626554+0000","last_active":"2024-06-06T04:29:05.905710+0000","last_peered":"2024-06-06T04:29:05.905710+0000","last_clean":"2024-06-06T04:29:05.905710+0000","last_became_active":"2024-06-06T04:27:32.626108+0000","last_became_peered":"2024-06-06T04:27:32.626108+0000","last_unstale":"2024-06-06T04:29:05.905710+0000","last_undegraded":"2024-06-06T04:29:05.905710+0000","last_fullsized":"2024-06-06T04:29:05.905710+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:37:43.811005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.262695+0000","last_change":"2024-06-06T04:27:19.306821+0000","last_active":"2024-06-06T04:29:06.262695+0000","last_peered":"2024-06-06T04:29:06.262695+0000","last_clean":"2024-06-06T04:29:06.262695+0000","last_became_active":"2024-06-06T04:27:19.306489+0000","last_became_peered":"2024-06-06T04:27:19.306489+0000","last_unstale":"2024-06-06T04:29:06.262695+0000","last_undegraded":"2024-06-06T04:29:06.262695+0000","last_fullsized":"2024-06-06T04:29:06.262695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:31:28.973112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.378929+0000","last_change":"2024-06-06T04:27:22.570617+0000","last_active":"2024-06-06T04:30:04.378929+0000","last_peered":"2024-06-06T04:30:04.378929+0000","last_clean":"2024-06-06T04:30:04.378929+0000","last_became_active":"2024-06-06T04:27:22.570271+0000","last_became_peered":"2024-06-06T04:27:22.570271+0000","last_unstale":"2024-06-06T04:30:04.378929+0000","last_undegraded":"2024-06-06T04:30:04.378929+0000","last_fullsized":"2024-06-06T04:30:04.378929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:45:33.890164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.909010+0000","last_change":"2024-06-06T04:27:32.628132+0000","last_active":"2024-06-06T04:29:05.909010+0000","last_peered":"2024-06-06T04:29:05.909010+0000","last_clean":"2024-06-06T04:29:05.909010+0000","last_became_active":"2024-06-06T04:27:32.627658+0000","last_became_peered":"2024-06-06T04:27:32.627658+0000","last_unstale":"2024-06-06T04:29:05.909010+0000","last_undegraded":"2024-06-06T04:29:05.909010+0000","last_fullsized":"2024-06-06T04:29:05.909010+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:24.817079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904125+0000","last_change":"2024-06-06T04:27:19.300510+0000","last_active":"2024-06-06T04:29:05.904125+0000","last_peered":"2024-06-06T04:29:05.904125+0000","last_clean":"2024-06-06T04:29:05.904125+0000","last_became_active":"2024-06-06T04:27:19.300150+0000","last_became_peered":"2024-06-06T04:27:19.300150+0000","last_unstale":"2024-06-06T04:29:05.904125+0000","last_undegraded":"2024-06-06T04:29:05.904125+0000","last_fullsized":"2024-06-06T04:29:05.904125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:38:25.084690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381098+0000","last_change":"2024-06-06T04:27:22.563864+0000","last_active":"2024-06-06T04:30:04.381098+0000","last_peered":"2024-06-06T04:30:04.381098+0000","last_clean":"2024-06-06T04:30:04.381098+0000","last_became_active":"2024-06-06T04:27:22.563555+0000","last_became_peered":"2024-06-06T04:27:22.563555+0000","last_unstale":"2024-06-06T04:30:04.381098+0000","last_undegraded":"2024-06-06T04:30:04.381098+0000","last_fullsized":"2024-06-06T04:30:04.381098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:39:33.707404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.165015+0000","last_change":"2024-06-06T04:27:32.617627+0000","last_active":"2024-06-06T04:29:06.165015+0000","last_peered":"2024-06-06T04:29:06.165015+0000","last_clean":"2024-06-06T04:29:06.165015+0000","last_became_active":"2024-06-06T04:27:32.617187+0000","last_became_peered":"2024-06-06T04:27:32.617187+0000","last_unstale":"2024-06-06T04:29:06.165015+0000","last_undegraded":"2024-06-06T04:29:06.165015+0000","last_fullsized":"2024-06-06T04:29:06.165015+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:24:52.554176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790234+0000","last_change":"2024-06-06T04:27:19.304316+0000","last_active":"2024-06-06T04:29:06.790234+0000","last_peered":"2024-06-06T04:29:06.790234+0000","last_clean":"2024-06-06T04:29:06.790234+0000","last_became_active":"2024-06-06T04:27:19.303952+0000","last_became_peered":"2024-06-06T04:27:19.303952+0000","last_unstale":"2024-06-06T04:29:06.790234+0000","last_undegraded":"2024-06-06T04:29:06.790234+0000","last_fullsized":"2024-06-06T04:29:06.790234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:26.886916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.371820+0000","last_change":"2024-06-06T04:27:22.561824+0000","last_active":"2024-06-06T04:30:04.371820+0000","last_peered":"2024-06-06T04:30:04.371820+0000","last_clean":"2024-06-06T04:30:04.371820+0000","last_became_active":"2024-06-06T04:27:22.561495+0000","last_became_peered":"2024-06-06T04:27:22.561495+0000","last_unstale":"2024-06-06T04:30:04.371820+0000","last_undegraded":"2024-06-06T04:30:04.371820+0000","last_fullsized":"2024-06-06T04:30:04.371820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:40:31.314627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560380+0000","last_change":"2024-06-06T04:27:32.624540+0000","last_active":"2024-06-06T04:29:06.560380+0000","last_peered":"2024-06-06T04:29:06.560380+0000","last_clean":"2024-06-06T04:29:06.560380+0000","last_became_active":"2024-06-06T04:27:32.624036+0000","last_became_peered":"2024-06-06T04:27:32.624036+0000","last_unstale":"2024-06-06T04:29:06.560380+0000","last_undegraded":"2024-06-06T04:29:06.560380+0000","last_fullsized":"2024-06-06T04:29:06.560380+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:16.983576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'4","reported_seq":219,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:19.967188+0000","last_change":"2024-06-06T04:27:19.304614+0000","last_active":"2024-06-06T04:31:19.967188+0000","last_peered":"2024-06-06T04:31:19.967188+0000","last_clean":"2024-06-06T04:31:19.967188+0000","last_became_active":"2024-06-06T04:27:19.304126+0000","last_became_peered":"2024-06-06T04:27:19.304126+0000","last_unstale":"2024-06-06T04:31:19.967188+0000","last_undegraded":"2024-06-06T04:31:19.967188+0000","last_fullsized":"2024-06-06T04:31:19.967188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:07:39.908630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":172,"num_read_kb":172,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.396136+0000","last_change":"2024-06-06T04:27:22.572534+0000","last_active":"2024-06-06T04:30:04.396136+0000","last_peered":"2024-06-06T04:30:04.396136+0000","last_clean":"2024-06-06T04:30:04.396136+0000","last_became_active":"2024-06-06T04:27:22.572214+0000","last_became_peered":"2024-06-06T04:27:22.572214+0000","last_unstale":"2024-06-06T04:30:04.396136+0000","last_undegraded":"2024-06-06T04:30:04.396136+0000","last_fullsized":"2024-06-06T04:30:04.396136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:13.176045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.262366+0000","last_change":"2024-06-06T04:28:05.723450+0000","last_active":"2024-06-06T04:29:06.262366+0000","last_peered":"2024-06-06T04:29:06.262366+0000","last_clean":"2024-06-06T04:29:06.262366+0000","last_became_active":"2024-06-06T04:28:05.722928+0000","last_became_peered":"2024-06-06T04:28:05.722928+0000","last_unstale":"2024-06-06T04:29:06.262366+0000","last_undegraded":"2024-06-06T04:29:06.262366+0000","last_fullsized":"2024-06-06T04:29:06.262366+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:53.093276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905918+0000","last_change":"2024-06-06T04:27:19.308465+0000","last_active":"2024-06-06T04:29:05.905918+0000","last_peered":"2024-06-06T04:29:05.905918+0000","last_clean":"2024-06-06T04:29:05.905918+0000","last_became_active":"2024-06-06T04:27:19.308077+0000","last_became_peered":"2024-06-06T04:27:19.308077+0000","last_unstale":"2024-06-06T04:29:05.905918+0000","last_undegraded":"2024-06-06T04:29:05.905918+0000","last_fullsized":"2024-06-06T04:29:05.905918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:37:48.282613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385739+0000","last_change":"2024-06-06T04:27:22.574141+0000","last_active":"2024-06-06T04:30:04.385739+0000","last_peered":"2024-06-06T04:30:04.385739+0000","last_clean":"2024-06-06T04:30:04.385739+0000","last_became_active":"2024-06-06T04:27:22.573679+0000","last_became_peered":"2024-06-06T04:27:22.573679+0000","last_unstale":"2024-06-06T04:30:04.385739+0000","last_undegraded":"2024-06-06T04:30:04.385739+0000","last_fullsized":"2024-06-06T04:30:04.385739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:27:25.177990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401856+0000","last_change":"2024-06-06T04:27:32.624148+0000","last_active":"2024-06-06T04:29:06.401856+0000","last_peered":"2024-06-06T04:29:06.401856+0000","last_clean":"2024-06-06T04:29:06.401856+0000","last_became_active":"2024-06-06T04:27:32.623448+0000","last_became_peered":"2024-06-06T04:27:32.623448+0000","last_unstale":"2024-06-06T04:29:06.401856+0000","last_undegraded":"2024-06-06T04:29:06.401856+0000","last_fullsized":"2024-06-06T04:29:06.401856+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:02.037296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790253+0000","last_change":"2024-06-06T04:27:19.516875+0000","last_active":"2024-06-06T04:29:06.790253+0000","last_peered":"2024-06-06T04:29:06.790253+0000","last_clean":"2024-06-06T04:29:06.790253+0000","last_became_active":"2024-06-06T04:27:19.516393+0000","last_became_peered":"2024-06-06T04:27:19.516393+0000","last_unstale":"2024-06-06T04:29:06.790253+0000","last_undegraded":"2024-06-06T04:29:06.790253+0000","last_fullsized":"2024-06-06T04:29:06.790253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:20.079218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392596+0000","last_change":"2024-06-06T04:28:05.724043+0000","last_active":"2024-06-06T04:30:04.392596+0000","last_peered":"2024-06-06T04:30:04.392596+0000","last_clean":"2024-06-06T04:30:04.392596+0000","last_became_active":"2024-06-06T04:28:05.723622+0000","last_became_peered":"2024-06-06T04:28:05.723622+0000","last_unstale":"2024-06-06T04:30:04.392596+0000","last_undegraded":"2024-06-06T04:30:04.392596+0000","last_fullsized":"2024-06-06T04:30:04.392596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:42.162400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.907660+0000","last_change":"2024-06-06T04:27:32.626366+0000","last_active":"2024-06-06T04:29:05.907660+0000","last_peered":"2024-06-06T04:29:05.907660+0000","last_clean":"2024-06-06T04:29:05.907660+0000","last_became_active":"2024-06-06T04:27:32.625847+0000","last_became_peered":"2024-06-06T04:27:32.625847+0000","last_unstale":"2024-06-06T04:29:05.907660+0000","last_undegraded":"2024-06-06T04:29:05.907660+0000","last_fullsized":"2024-06-06T04:29:05.907660+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:47:44.167531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'42","reported_seq":441,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:20.968878+0000","last_change":"2024-06-06T04:27:19.303581+0000","last_active":"2024-06-06T04:31:20.968878+0000","last_peered":"2024-06-06T04:31:20.968878+0000","last_clean":"2024-06-06T04:31:20.968878+0000","last_became_active":"2024-06-06T04:27:19.303234+0000","last_became_peered":"2024-06-06T04:27:19.303234+0000","last_unstale":"2024-06-06T04:31:20.968878+0000","last_undegraded":"2024-06-06T04:31:20.968878+0000","last_fullsized":"2024-06-06T04:31:20.968878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:37.721588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72581,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":356,"num_read_kb":250,"num_write":43,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.391077+0000","last_change":"2024-06-06T04:27:22.567177+0000","last_active":"2024-06-06T04:30:04.391077+0000","last_peered":"2024-06-06T04:30:04.391077+0000","last_clean":"2024-06-06T04:30:04.391077+0000","last_became_active":"2024-06-06T04:27:22.566751+0000","last_became_peered":"2024-06-06T04:27:22.566751+0000","last_unstale":"2024-06-06T04:30:04.391077+0000","last_undegraded":"2024-06-06T04:30:04.391077+0000","last_fullsized":"2024-06-06T04:30:04.391077+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:37:15.914505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790613+0000","last_change":"2024-06-06T04:27:32.619319+0000","last_active":"2024-06-06T04:29:06.790613+0000","last_peered":"2024-06-06T04:29:06.790613+0000","last_clean":"2024-06-06T04:29:06.790613+0000","last_became_active":"2024-06-06T04:27:32.618843+0000","last_became_peered":"2024-06-06T04:27:32.618843+0000","last_unstale":"2024-06-06T04:29:06.790613+0000","last_undegraded":"2024-06-06T04:29:06.790613+0000","last_fullsized":"2024-06-06T04:29:06.790613+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:32:20.071512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905924+0000","last_change":"2024-06-06T04:27:19.304731+0000","last_active":"2024-06-06T04:29:05.905924+0000","last_peered":"2024-06-06T04:29:05.905924+0000","last_clean":"2024-06-06T04:29:05.905924+0000","last_became_active":"2024-06-06T04:27:19.304351+0000","last_became_peered":"2024-06-06T04:27:19.304351+0000","last_unstale":"2024-06-06T04:29:05.905924+0000","last_undegraded":"2024-06-06T04:29:05.905924+0000","last_fullsized":"2024-06-06T04:29:05.905924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:16:14.136633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.391311+0000","last_change":"2024-06-06T04:29:05.909041+0000","last_active":"2024-06-06T04:30:04.391311+0000","last_peered":"2024-06-06T04:30:04.391311+0000","last_clean":"2024-06-06T04:30:04.391311+0000","last_became_active":"2024-06-06T04:29:05.908666+0000","last_became_peered":"2024-06-06T04:29:05.908666+0000","last_unstale":"2024-06-06T04:30:04.391311+0000","last_undegraded":"2024-06-06T04:30:04.391311+0000","last_fullsized":"2024-06-06T04:30:04.391311+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:47:55.225572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908861+0000","last_change":"2024-06-06T04:27:32.628645+0000","last_active":"2024-06-06T04:29:05.908861+0000","last_peered":"2024-06-06T04:29:05.908861+0000","last_clean":"2024-06-06T04:29:05.908861+0000","last_became_active":"2024-06-06T04:27:32.627586+0000","last_became_peered":"2024-06-06T04:27:32.627586+0000","last_unstale":"2024-06-06T04:29:05.908861+0000","last_undegraded":"2024-06-06T04:29:05.908861+0000","last_fullsized":"2024-06-06T04:29:05.908861+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:46:53.094511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790029+0000","last_change":"2024-06-06T04:27:19.303805+0000","last_active":"2024-06-06T04:29:06.790029+0000","last_peered":"2024-06-06T04:29:06.790029+0000","last_clean":"2024-06-06T04:29:06.790029+0000","last_became_active":"2024-06-06T04:27:19.303391+0000","last_became_peered":"2024-06-06T04:27:19.303391+0000","last_unstale":"2024-06-06T04:29:06.790029+0000","last_undegraded":"2024-06-06T04:29:06.790029+0000","last_fullsized":"2024-06-06T04:29:06.790029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:30:27.336567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.387613+0000","last_change":"2024-06-06T04:27:22.575148+0000","last_active":"2024-06-06T04:30:04.387613+0000","last_peered":"2024-06-06T04:30:04.387613+0000","last_clean":"2024-06-06T04:30:04.387613+0000","last_became_active":"2024-06-06T04:27:22.574591+0000","last_became_peered":"2024-06-06T04:27:22.574591+0000","last_unstale":"2024-06-06T04:30:04.387613+0000","last_undegraded":"2024-06-06T04:30:04.387613+0000","last_fullsized":"2024-06-06T04:30:04.387613+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:35:23.404644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214440+0000","last_change":"2024-06-06T04:28:05.726734+0000","last_active":"2024-06-06T04:29:06.214440+0000","last_peered":"2024-06-06T04:29:06.214440+0000","last_clean":"2024-06-06T04:29:06.214440+0000","last_became_active":"2024-06-06T04:28:05.726238+0000","last_became_peered":"2024-06-06T04:28:05.726238+0000","last_unstale":"2024-06-06T04:29:06.214440+0000","last_undegraded":"2024-06-06T04:29:06.214440+0000","last_fullsized":"2024-06-06T04:29:06.214440+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:47.841475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"100'6","reported_seq":228,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:20.968408+0000","last_change":"2024-06-06T04:27:19.305571+0000","last_active":"2024-06-06T04:31:20.968408+0000","last_peered":"2024-06-06T04:31:20.968408+0000","last_clean":"2024-06-06T04:31:20.968408+0000","last_became_active":"2024-06-06T04:27:19.305183+0000","last_became_peered":"2024-06-06T04:27:19.305183+0000","last_unstale":"2024-06-06T04:31:20.968408+0000","last_undegraded":"2024-06-06T04:31:20.968408+0000","last_fullsized":"2024-06-06T04:31:20.968408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:51.205209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1611,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392588+0000","last_change":"2024-06-06T04:29:06.263872+0000","last_active":"2024-06-06T04:30:04.392588+0000","last_peered":"2024-06-06T04:30:04.392588+0000","last_clean":"2024-06-06T04:30:04.392588+0000","last_became_active":"2024-06-06T04:29:06.263487+0000","last_became_peered":"2024-06-06T04:29:06.263487+0000","last_unstale":"2024-06-06T04:30:04.392588+0000","last_undegraded":"2024-06-06T04:30:04.392588+0000","last_fullsized":"2024-06-06T04:30:04.392588+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:15.965926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904639+0000","last_change":"2024-06-06T04:28:05.724892+0000","last_active":"2024-06-06T04:29:05.904639+0000","last_peered":"2024-06-06T04:29:05.904639+0000","last_clean":"2024-06-06T04:29:05.904639+0000","last_became_active":"2024-06-06T04:28:05.724420+0000","last_became_peered":"2024-06-06T04:28:05.724420+0000","last_unstale":"2024-06-06T04:29:05.904639+0000","last_undegraded":"2024-06-06T04:29:05.904639+0000","last_fullsized":"2024-06-06T04:29:05.904639+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:39.879481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":60,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.213846+0000","last_change":"2024-06-06T04:27:19.516687+0000","last_active":"2024-06-06T04:29:06.213846+0000","last_peered":"2024-06-06T04:29:06.213846+0000","last_clean":"2024-06-06T04:29:06.213846+0000","last_became_active":"2024-06-06T04:27:19.516271+0000","last_became_peered":"2024-06-06T04:27:19.516271+0000","last_unstale":"2024-06-06T04:29:06.213846+0000","last_undegraded":"2024-06-06T04:29:06.213846+0000","last_fullsized":"2024-06-06T04:29:06.213846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:33:46.817727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394832+0000","last_change":"2024-06-06T04:27:22.572671+0000","last_active":"2024-06-06T04:30:04.394832+0000","last_peered":"2024-06-06T04:30:04.394832+0000","last_clean":"2024-06-06T04:30:04.394832+0000","last_became_active":"2024-06-06T04:27:22.572230+0000","last_became_peered":"2024-06-06T04:27:22.572230+0000","last_unstale":"2024-06-06T04:30:04.394832+0000","last_undegraded":"2024-06-06T04:30:04.394832+0000","last_fullsized":"2024-06-06T04:30:04.394832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:36:48.393092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.789710+0000","last_change":"2024-06-06T04:27:32.620131+0000","last_active":"2024-06-06T04:29:06.789710+0000","last_peered":"2024-06-06T04:29:06.789710+0000","last_clean":"2024-06-06T04:29:06.789710+0000","last_became_active":"2024-06-06T04:27:32.619686+0000","last_became_peered":"2024-06-06T04:27:32.619686+0000","last_unstale":"2024-06-06T04:29:06.789710+0000","last_undegraded":"2024-06-06T04:29:06.789710+0000","last_fullsized":"2024-06-06T04:29:06.789710+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:19:17.130410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214545+0000","last_change":"2024-06-06T04:27:19.304682+0000","last_active":"2024-06-06T04:29:06.214545+0000","last_peered":"2024-06-06T04:29:06.214545+0000","last_clean":"2024-06-06T04:29:06.214545+0000","last_became_active":"2024-06-06T04:27:19.304262+0000","last_became_peered":"2024-06-06T04:27:19.304262+0000","last_unstale":"2024-06-06T04:29:06.214545+0000","last_undegraded":"2024-06-06T04:29:06.214545+0000","last_fullsized":"2024-06-06T04:29:06.214545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:45.819520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.377096+0000","last_change":"2024-06-06T04:29:06.791555+0000","last_active":"2024-06-06T04:30:04.377096+0000","last_peered":"2024-06-06T04:30:04.377096+0000","last_clean":"2024-06-06T04:30:04.377096+0000","last_became_active":"2024-06-06T04:29:06.790925+0000","last_became_peered":"2024-06-06T04:29:06.790925+0000","last_unstale":"2024-06-06T04:30:04.377096+0000","last_undegraded":"2024-06-06T04:30:04.377096+0000","last_fullsized":"2024-06-06T04:30:04.377096+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:23:39.549238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.213893+0000","last_change":"2024-06-06T04:27:32.622092+0000","last_active":"2024-06-06T04:29:06.213893+0000","last_peered":"2024-06-06T04:29:06.213893+0000","last_clean":"2024-06-06T04:29:06.213893+0000","last_became_active":"2024-06-06T04:27:32.621618+0000","last_became_peered":"2024-06-06T04:27:32.621618+0000","last_unstale":"2024-06-06T04:29:06.213893+0000","last_undegraded":"2024-06-06T04:29:06.213893+0000","last_fullsized":"2024-06-06T04:29:06.213893+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:03.067292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.907896+0000","last_change":"2024-06-06T04:27:19.516803+0000","last_active":"2024-06-06T04:29:05.907896+0000","last_peered":"2024-06-06T04:29:05.907896+0000","last_clean":"2024-06-06T04:29:05.907896+0000","last_became_active":"2024-06-06T04:27:19.516458+0000","last_became_peered":"2024-06-06T04:27:19.516458+0000","last_unstale":"2024-06-06T04:29:05.907896+0000","last_undegraded":"2024-06-06T04:29:05.907896+0000","last_fullsized":"2024-06-06T04:29:05.907896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:34:53.593825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380334+0000","last_change":"2024-06-06T04:27:22.564306+0000","last_active":"2024-06-06T04:30:04.380334+0000","last_peered":"2024-06-06T04:30:04.380334+0000","last_clean":"2024-06-06T04:30:04.380334+0000","last_became_active":"2024-06-06T04:27:22.563922+0000","last_became_peered":"2024-06-06T04:27:22.563922+0000","last_unstale":"2024-06-06T04:30:04.380334+0000","last_undegraded":"2024-06-06T04:30:04.380334+0000","last_fullsized":"2024-06-06T04:30:04.380334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:42:23.189047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904530+0000","last_change":"2024-06-06T04:27:32.617014+0000","last_active":"2024-06-06T04:29:05.904530+0000","last_peered":"2024-06-06T04:29:05.904530+0000","last_clean":"2024-06-06T04:29:05.904530+0000","last_became_active":"2024-06-06T04:27:32.616639+0000","last_became_peered":"2024-06-06T04:27:32.616639+0000","last_unstale":"2024-06-06T04:29:05.904530+0000","last_undegraded":"2024-06-06T04:29:05.904530+0000","last_fullsized":"2024-06-06T04:29:05.904530+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:54:52.034711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905627+0000","last_change":"2024-06-06T04:27:19.305000+0000","last_active":"2024-06-06T04:29:05.905627+0000","last_peered":"2024-06-06T04:29:05.905627+0000","last_clean":"2024-06-06T04:29:05.905627+0000","last_became_active":"2024-06-06T04:27:19.304676+0000","last_became_peered":"2024-06-06T04:27:19.304676+0000","last_unstale":"2024-06-06T04:29:05.905627+0000","last_undegraded":"2024-06-06T04:29:05.905627+0000","last_fullsized":"2024-06-06T04:29:05.905627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:19:34.054455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.382734+0000","last_change":"2024-06-06T04:27:22.572680+0000","last_active":"2024-06-06T04:30:04.382734+0000","last_peered":"2024-06-06T04:30:04.382734+0000","last_clean":"2024-06-06T04:30:04.382734+0000","last_became_active":"2024-06-06T04:27:22.572348+0000","last_became_peered":"2024-06-06T04:27:22.572348+0000","last_unstale":"2024-06-06T04:30:04.382734+0000","last_undegraded":"2024-06-06T04:30:04.382734+0000","last_fullsized":"2024-06-06T04:30:04.382734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:05:26.215771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904503+0000","last_change":"2024-06-06T04:27:32.611587+0000","last_active":"2024-06-06T04:29:05.904503+0000","last_peered":"2024-06-06T04:29:05.904503+0000","last_clean":"2024-06-06T04:29:05.904503+0000","last_became_active":"2024-06-06T04:27:32.611183+0000","last_became_peered":"2024-06-06T04:27:32.611183+0000","last_unstale":"2024-06-06T04:29:05.904503+0000","last_undegraded":"2024-06-06T04:29:05.904503+0000","last_fullsized":"2024-06-06T04:29:05.904503+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:36.773254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.371730+0000","last_change":"2024-06-06T04:29:06.403321+0000","last_active":"2024-06-06T04:30:04.371730+0000","last_peered":"2024-06-06T04:30:04.371730+0000","last_clean":"2024-06-06T04:30:04.371730+0000","last_became_active":"2024-06-06T04:29:06.402909+0000","last_became_peered":"2024-06-06T04:29:06.402909+0000","last_unstale":"2024-06-06T04:30:04.371730+0000","last_undegraded":"2024-06-06T04:30:04.371730+0000","last_fullsized":"2024-06-06T04:30:04.371730+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:29:38.963372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":225,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:21.969275+0000","last_change":"2024-06-06T04:27:19.516361+0000","last_active":"2024-06-06T04:31:21.969275+0000","last_peered":"2024-06-06T04:31:21.969275+0000","last_clean":"2024-06-06T04:31:21.969275+0000","last_became_active":"2024-06-06T04:27:19.515973+0000","last_became_peered":"2024-06-06T04:27:19.515973+0000","last_unstale":"2024-06-06T04:31:21.969275+0000","last_undegraded":"2024-06-06T04:31:21.969275+0000","last_fullsized":"2024-06-06T04:31:21.969275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:37.298819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905945+0000","last_change":"2024-06-06T04:27:32.630528+0000","last_active":"2024-06-06T04:29:05.905945+0000","last_peered":"2024-06-06T04:29:05.905945+0000","last_clean":"2024-06-06T04:29:05.905945+0000","last_became_active":"2024-06-06T04:27:32.630215+0000","last_became_peered":"2024-06-06T04:27:32.630215+0000","last_unstale":"2024-06-06T04:29:05.905945+0000","last_undegraded":"2024-06-06T04:29:05.905945+0000","last_fullsized":"2024-06-06T04:29:05.905945+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:34:29.980193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392719+0000","last_change":"2024-06-06T04:27:22.567466+0000","last_active":"2024-06-06T04:30:04.392719+0000","last_peered":"2024-06-06T04:30:04.392719+0000","last_clean":"2024-06-06T04:30:04.392719+0000","last_became_active":"2024-06-06T04:27:22.567145+0000","last_became_peered":"2024-06-06T04:27:22.567145+0000","last_unstale":"2024-06-06T04:30:04.392719+0000","last_undegraded":"2024-06-06T04:30:04.392719+0000","last_fullsized":"2024-06-06T04:30:04.392719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:56:56.203027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401783+0000","last_change":"2024-06-06T04:27:19.302726+0000","last_active":"2024-06-06T04:29:06.401783+0000","last_peered":"2024-06-06T04:29:06.401783+0000","last_clean":"2024-06-06T04:29:06.401783+0000","last_became_active":"2024-06-06T04:27:19.302329+0000","last_became_peered":"2024-06-06T04:27:19.302329+0000","last_unstale":"2024-06-06T04:29:06.401783+0000","last_undegraded":"2024-06-06T04:29:06.401783+0000","last_fullsized":"2024-06-06T04:29:06.401783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:55:15.010632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":367,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905237+0000","last_change":"2024-06-06T04:25:53.854188+0000","last_active":"2024-06-06T04:29:05.905237+0000","last_peered":"2024-06-06T04:29:05.905237+0000","last_clean":"2024-06-06T04:29:05.905237+0000","last_became_active":"2024-06-06T04:25:53.853700+0000","last_became_peered":"2024-06-06T04:25:53.853700+0000","last_unstale":"2024-06-06T04:29:05.905237+0000","last_undegraded":"2024-06-06T04:29:05.905237+0000","last_fullsized":"2024-06-06T04:29:05.905237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:22:28.847333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:22:28.847333+0000","last_clean_scrub_stamp":"2024-06-06T04:22:28.847333+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:59:10.246373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790675+0000","last_change":"2024-06-06T04:27:32.623300+0000","last_active":"2024-06-06T04:29:06.790675+0000","last_peered":"2024-06-06T04:29:06.790675+0000","last_clean":"2024-06-06T04:29:06.790675+0000","last_became_active":"2024-06-06T04:27:32.622811+0000","last_became_peered":"2024-06-06T04:27:32.622811+0000","last_unstale":"2024-06-06T04:29:06.790675+0000","last_undegraded":"2024-06-06T04:29:06.790675+0000","last_fullsized":"2024-06-06T04:29:06.790675+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:28.383079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.367134+0000","last_change":"2024-06-06T04:27:22.566617+0000","last_active":"2024-06-06T04:30:04.367134+0000","last_peered":"2024-06-06T04:30:04.367134+0000","last_clean":"2024-06-06T04:30:04.367134+0000","last_became_active":"2024-06-06T04:27:22.566256+0000","last_became_peered":"2024-06-06T04:27:22.566256+0000","last_unstale":"2024-06-06T04:30:04.367134+0000","last_undegraded":"2024-06-06T04:30:04.367134+0000","last_fullsized":"2024-06-06T04:30:04.367134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:50:41.318475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:18.419034+0000","last_change":"2024-06-06T04:27:19.305941+0000","last_active":"2024-06-06T04:29:18.419034+0000","last_peered":"2024-06-06T04:29:18.419034+0000","last_clean":"2024-06-06T04:29:18.419034+0000","last_became_active":"2024-06-06T04:27:19.305037+0000","last_became_peered":"2024-06-06T04:27:19.305037+0000","last_unstale":"2024-06-06T04:29:18.419034+0000","last_undegraded":"2024-06-06T04:29:18.419034+0000","last_fullsized":"2024-06-06T04:29:18.419034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:54.084427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908935+0000","last_change":"2024-06-06T04:27:32.621182+0000","last_active":"2024-06-06T04:29:05.908935+0000","last_peered":"2024-06-06T04:29:05.908935+0000","last_clean":"2024-06-06T04:29:05.908935+0000","last_became_active":"2024-06-06T04:27:32.620710+0000","last_became_peered":"2024-06-06T04:27:32.620710+0000","last_unstale":"2024-06-06T04:29:05.908935+0000","last_undegraded":"2024-06-06T04:29:05.908935+0000","last_fullsized":"2024-06-06T04:29:05.908935+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:05.512309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214246+0000","last_change":"2024-06-06T04:27:19.517033+0000","last_active":"2024-06-06T04:29:06.214246+0000","last_peered":"2024-06-06T04:29:06.214246+0000","last_clean":"2024-06-06T04:29:06.214246+0000","last_became_active":"2024-06-06T04:27:19.516278+0000","last_became_peered":"2024-06-06T04:27:19.516278+0000","last_unstale":"2024-06-06T04:29:06.214246+0000","last_undegraded":"2024-06-06T04:29:06.214246+0000","last_fullsized":"2024-06-06T04:29:06.214246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:11:20.336818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.368634+0000","last_change":"2024-06-06T04:27:22.566261+0000","last_active":"2024-06-06T04:30:04.368634+0000","last_peered":"2024-06-06T04:30:04.368634+0000","last_clean":"2024-06-06T04:30:04.368634+0000","last_became_active":"2024-06-06T04:27:22.565875+0000","last_became_peered":"2024-06-06T04:27:22.565875+0000","last_unstale":"2024-06-06T04:30:04.368634+0000","last_undegraded":"2024-06-06T04:30:04.368634+0000","last_fullsized":"2024-06-06T04:30:04.368634+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:49:55.923950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560350+0000","last_change":"2024-06-06T04:27:32.619854+0000","last_active":"2024-06-06T04:29:06.560350+0000","last_peered":"2024-06-06T04:29:06.560350+0000","last_clean":"2024-06-06T04:29:06.560350+0000","last_became_active":"2024-06-06T04:27:32.618765+0000","last_became_peered":"2024-06-06T04:27:32.618765+0000","last_unstale":"2024-06-06T04:29:06.560350+0000","last_undegraded":"2024-06-06T04:29:06.560350+0000","last_fullsized":"2024-06-06T04:29:06.560350+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:47:05.283751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'14","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:17.603468+0000","last_change":"2024-06-06T04:27:19.303057+0000","last_active":"2024-06-06T04:31:17.603468+0000","last_peered":"2024-06-06T04:31:17.603468+0000","last_clean":"2024-06-06T04:31:17.603468+0000","last_became_active":"2024-06-06T04:27:19.302716+0000","last_became_peered":"2024-06-06T04:27:19.302716+0000","last_unstale":"2024-06-06T04:31:17.603468+0000","last_undegraded":"2024-06-06T04:31:17.603468+0000","last_fullsized":"2024-06-06T04:31:17.603468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:35.273663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381463+0000","last_change":"2024-06-06T04:27:22.575274+0000","last_active":"2024-06-06T04:30:04.381463+0000","last_peered":"2024-06-06T04:30:04.381463+0000","last_clean":"2024-06-06T04:30:04.381463+0000","last_became_active":"2024-06-06T04:27:22.574979+0000","last_became_peered":"2024-06-06T04:27:22.574979+0000","last_unstale":"2024-06-06T04:30:04.381463+0000","last_undegraded":"2024-06-06T04:30:04.381463+0000","last_fullsized":"2024-06-06T04:30:04.381463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:30.718398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905413+0000","last_change":"2024-06-06T04:27:32.619939+0000","last_active":"2024-06-06T04:29:05.905413+0000","last_peered":"2024-06-06T04:29:05.905413+0000","last_clean":"2024-06-06T04:29:05.905413+0000","last_became_active":"2024-06-06T04:27:32.619499+0000","last_became_peered":"2024-06-06T04:27:32.619499+0000","last_unstale":"2024-06-06T04:29:05.905413+0000","last_undegraded":"2024-06-06T04:29:05.905413+0000","last_fullsized":"2024-06-06T04:29:05.905413+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:41:13.933546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402340+0000","last_change":"2024-06-06T04:27:19.397552+0000","last_active":"2024-06-06T04:29:06.402340+0000","last_peered":"2024-06-06T04:29:06.402340+0000","last_clean":"2024-06-06T04:29:06.402340+0000","last_became_active":"2024-06-06T04:27:19.397186+0000","last_became_peered":"2024-06-06T04:27:19.397186+0000","last_unstale":"2024-06-06T04:29:06.402340+0000","last_undegraded":"2024-06-06T04:29:06.402340+0000","last_fullsized":"2024-06-06T04:29:06.402340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:24:13.403860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.384537+0000","last_change":"2024-06-06T04:28:05.728212+0000","last_active":"2024-06-06T04:30:04.384537+0000","last_peered":"2024-06-06T04:30:04.384537+0000","last_clean":"2024-06-06T04:30:04.384537+0000","last_became_active":"2024-06-06T04:28:05.727748+0000","last_became_peered":"2024-06-06T04:28:05.727748+0000","last_unstale":"2024-06-06T04:30:04.384537+0000","last_undegraded":"2024-06-06T04:30:04.384537+0000","last_fullsized":"2024-06-06T04:30:04.384537+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:50.583716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402083+0000","last_change":"2024-06-06T04:28:05.896480+0000","last_active":"2024-06-06T04:29:06.402083+0000","last_peered":"2024-06-06T04:29:06.402083+0000","last_clean":"2024-06-06T04:29:06.402083+0000","last_became_active":"2024-06-06T04:28:05.895967+0000","last_became_peered":"2024-06-06T04:28:05.895967+0000","last_unstale":"2024-06-06T04:29:06.402083+0000","last_undegraded":"2024-06-06T04:29:06.402083+0000","last_fullsized":"2024-06-06T04:29:06.402083+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:26:19.846884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.907843+0000","last_change":"2024-06-06T04:27:19.516515+0000","last_active":"2024-06-06T04:29:06.907843+0000","last_peered":"2024-06-06T04:29:06.907843+0000","last_clean":"2024-06-06T04:29:06.907843+0000","last_became_active":"2024-06-06T04:27:19.516087+0000","last_became_peered":"2024-06-06T04:27:19.516087+0000","last_unstale":"2024-06-06T04:29:06.907843+0000","last_undegraded":"2024-06-06T04:29:06.907843+0000","last_fullsized":"2024-06-06T04:29:06.907843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:58.440226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":40,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.342173+0000","last_change":"2024-06-06T04:27:22.562367+0000","last_active":"2024-06-06T04:30:04.342173+0000","last_peered":"2024-06-06T04:30:04.342173+0000","last_clean":"2024-06-06T04:30:04.342173+0000","last_became_active":"2024-06-06T04:27:22.561947+0000","last_became_peered":"2024-06-06T04:27:22.561947+0000","last_unstale":"2024-06-06T04:30:04.342173+0000","last_undegraded":"2024-06-06T04:30:04.342173+0000","last_fullsized":"2024-06-06T04:30:04.342173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:28:06.200748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908183+0000","last_change":"2024-06-06T04:27:32.631577+0000","last_active":"2024-06-06T04:29:05.908183+0000","last_peered":"2024-06-06T04:29:05.908183+0000","last_clean":"2024-06-06T04:29:05.908183+0000","last_became_active":"2024-06-06T04:27:32.629555+0000","last_became_peered":"2024-06-06T04:27:32.629555+0000","last_unstale":"2024-06-06T04:29:05.908183+0000","last_undegraded":"2024-06-06T04:29:05.908183+0000","last_fullsized":"2024-06-06T04:29:05.908183+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:52:56.197281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560477+0000","last_change":"2024-06-06T04:27:19.298880+0000","last_active":"2024-06-06T04:29:06.560477+0000","last_peered":"2024-06-06T04:29:06.560477+0000","last_clean":"2024-06-06T04:29:06.560477+0000","last_became_active":"2024-06-06T04:27:19.298551+0000","last_became_peered":"2024-06-06T04:27:19.298551+0000","last_unstale":"2024-06-06T04:29:06.560477+0000","last_undegraded":"2024-06-06T04:29:06.560477+0000","last_fullsized":"2024-06-06T04:29:06.560477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:11:32.145401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.383001+0000","last_change":"2024-06-06T04:27:22.575065+0000","last_active":"2024-06-06T04:30:04.383001+0000","last_peered":"2024-06-06T04:30:04.383001+0000","last_clean":"2024-06-06T04:30:04.383001+0000","last_became_active":"2024-06-06T04:27:22.574469+0000","last_became_peered":"2024-06-06T04:27:22.574469+0000","last_unstale":"2024-06-06T04:30:04.383001+0000","last_undegraded":"2024-06-06T04:30:04.383001+0000","last_fullsized":"2024-06-06T04:30:04.383001+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:29:44.943833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402041+0000","last_change":"2024-06-06T04:27:32.626678+0000","last_active":"2024-06-06T04:29:06.402041+0000","last_peered":"2024-06-06T04:29:06.402041+0000","last_clean":"2024-06-06T04:29:06.402041+0000","last_became_active":"2024-06-06T04:27:32.626167+0000","last_became_peered":"2024-06-06T04:27:32.626167+0000","last_unstale":"2024-06-06T04:29:06.402041+0000","last_undegraded":"2024-06-06T04:29:06.402041+0000","last_fullsized":"2024-06-06T04:29:06.402041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:56:26.363195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:32.586256+0000","last_change":"2024-06-06T04:27:19.306466+0000","last_active":"2024-06-06T04:29:32.586256+0000","last_peered":"2024-06-06T04:29:32.586256+0000","last_clean":"2024-06-06T04:29:32.586256+0000","last_became_active":"2024-06-06T04:27:19.303821+0000","last_became_peered":"2024-06-06T04:27:19.303821+0000","last_unstale":"2024-06-06T04:29:32.586256+0000","last_undegraded":"2024-06-06T04:29:32.586256+0000","last_fullsized":"2024-06-06T04:29:32.586256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:23:53.593998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.386917+0000","last_change":"2024-06-06T04:27:22.562662+0000","last_active":"2024-06-06T04:30:04.386917+0000","last_peered":"2024-06-06T04:30:04.386917+0000","last_clean":"2024-06-06T04:30:04.386917+0000","last_became_active":"2024-06-06T04:27:22.562318+0000","last_became_peered":"2024-06-06T04:27:22.562318+0000","last_unstale":"2024-06-06T04:30:04.386917+0000","last_undegraded":"2024-06-06T04:30:04.386917+0000","last_fullsized":"2024-06-06T04:30:04.386917+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:31:26.639275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402218+0000","last_change":"2024-06-06T04:27:32.619970+0000","last_active":"2024-06-06T04:29:06.402218+0000","last_peered":"2024-06-06T04:29:06.402218+0000","last_clean":"2024-06-06T04:29:06.402218+0000","last_became_active":"2024-06-06T04:27:32.619464+0000","last_became_peered":"2024-06-06T04:27:32.619464+0000","last_unstale":"2024-06-06T04:29:06.402218+0000","last_undegraded":"2024-06-06T04:29:06.402218+0000","last_fullsized":"2024-06-06T04:29:06.402218+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:44.844018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214056+0000","last_change":"2024-06-06T04:27:19.308172+0000","last_active":"2024-06-06T04:29:06.214056+0000","last_peered":"2024-06-06T04:29:06.214056+0000","last_clean":"2024-06-06T04:29:06.214056+0000","last_became_active":"2024-06-06T04:27:19.307780+0000","last_became_peered":"2024-06-06T04:27:19.307780+0000","last_unstale":"2024-06-06T04:29:06.214056+0000","last_undegraded":"2024-06-06T04:29:06.214056+0000","last_fullsized":"2024-06-06T04:29:06.214056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:39:11.653649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394812+0000","last_change":"2024-06-06T04:27:22.568881+0000","last_active":"2024-06-06T04:30:04.394812+0000","last_peered":"2024-06-06T04:30:04.394812+0000","last_clean":"2024-06-06T04:30:04.394812+0000","last_became_active":"2024-06-06T04:27:22.568489+0000","last_became_peered":"2024-06-06T04:27:22.568489+0000","last_unstale":"2024-06-06T04:30:04.394812+0000","last_undegraded":"2024-06-06T04:30:04.394812+0000","last_fullsized":"2024-06-06T04:30:04.394812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:00:22.339313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.907971+0000","last_change":"2024-06-06T04:27:32.622357+0000","last_active":"2024-06-06T04:29:06.907971+0000","last_peered":"2024-06-06T04:29:06.907971+0000","last_clean":"2024-06-06T04:29:06.907971+0000","last_became_active":"2024-06-06T04:27:32.621680+0000","last_became_peered":"2024-06-06T04:27:32.621680+0000","last_unstale":"2024-06-06T04:29:06.907971+0000","last_undegraded":"2024-06-06T04:29:06.907971+0000","last_fullsized":"2024-06-06T04:29:06.907971+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:00.677721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.906141+0000","last_change":"2024-06-06T04:27:19.516890+0000","last_active":"2024-06-06T04:29:05.906141+0000","last_peered":"2024-06-06T04:29:05.906141+0000","last_clean":"2024-06-06T04:29:05.906141+0000","last_became_active":"2024-06-06T04:27:19.516490+0000","last_became_peered":"2024-06-06T04:27:19.516490+0000","last_unstale":"2024-06-06T04:29:05.906141+0000","last_undegraded":"2024-06-06T04:29:05.906141+0000","last_fullsized":"2024-06-06T04:29:05.906141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:11:59.336250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.395341+0000","last_change":"2024-06-06T04:27:22.571003+0000","last_active":"2024-06-06T04:30:04.395341+0000","last_peered":"2024-06-06T04:30:04.395341+0000","last_clean":"2024-06-06T04:30:04.395341+0000","last_became_active":"2024-06-06T04:27:22.570442+0000","last_became_peered":"2024-06-06T04:27:22.570442+0000","last_unstale":"2024-06-06T04:30:04.395341+0000","last_undegraded":"2024-06-06T04:30:04.395341+0000","last_fullsized":"2024-06-06T04:30:04.395341+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:28:26.870429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.164730+0000","last_change":"2024-06-06T04:27:32.624424+0000","last_active":"2024-06-06T04:29:06.164730+0000","last_peered":"2024-06-06T04:29:06.164730+0000","last_clean":"2024-06-06T04:29:06.164730+0000","last_became_active":"2024-06-06T04:27:32.624026+0000","last_became_peered":"2024-06-06T04:27:32.624026+0000","last_unstale":"2024-06-06T04:29:06.164730+0000","last_undegraded":"2024-06-06T04:29:06.164730+0000","last_fullsized":"2024-06-06T04:29:06.164730+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:30:08.814284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790341+0000","last_change":"2024-06-06T04:27:19.301382+0000","last_active":"2024-06-06T04:29:06.790341+0000","last_peered":"2024-06-06T04:29:06.790341+0000","last_clean":"2024-06-06T04:29:06.790341+0000","last_became_active":"2024-06-06T04:27:19.300882+0000","last_became_peered":"2024-06-06T04:27:19.300882+0000","last_unstale":"2024-06-06T04:29:06.790341+0000","last_undegraded":"2024-06-06T04:29:06.790341+0000","last_fullsized":"2024-06-06T04:29:06.790341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:50.514103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392499+0000","last_change":"2024-06-06T04:27:22.570661+0000","last_active":"2024-06-06T04:30:04.392499+0000","last_peered":"2024-06-06T04:30:04.392499+0000","last_clean":"2024-06-06T04:30:04.392499+0000","last_became_active":"2024-06-06T04:27:22.570287+0000","last_became_peered":"2024-06-06T04:27:22.570287+0000","last_unstale":"2024-06-06T04:30:04.392499+0000","last_undegraded":"2024-06-06T04:30:04.392499+0000","last_fullsized":"2024-06-06T04:30:04.392499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:48.790492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.789984+0000","last_change":"2024-06-06T04:27:32.618385+0000","last_active":"2024-06-06T04:29:06.789984+0000","last_peered":"2024-06-06T04:29:06.789984+0000","last_clean":"2024-06-06T04:29:06.789984+0000","last_became_active":"2024-06-06T04:27:32.617956+0000","last_became_peered":"2024-06-06T04:27:32.617956+0000","last_unstale":"2024-06-06T04:29:06.789984+0000","last_undegraded":"2024-06-06T04:29:06.789984+0000","last_fullsized":"2024-06-06T04:29:06.789984+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:26.308657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904541+0000","last_change":"2024-06-06T04:28:05.728633+0000","last_active":"2024-06-06T04:29:05.904541+0000","last_peered":"2024-06-06T04:29:05.904541+0000","last_clean":"2024-06-06T04:29:05.904541+0000","last_became_active":"2024-06-06T04:28:05.722427+0000","last_became_peered":"2024-06-06T04:28:05.722427+0000","last_unstale":"2024-06-06T04:29:05.904541+0000","last_undegraded":"2024-06-06T04:29:05.904541+0000","last_fullsized":"2024-06-06T04:29:05.904541+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:13:22.061798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388201+0000","last_change":"2024-06-06T04:27:22.574950+0000","last_active":"2024-06-06T04:30:04.388201+0000","last_peered":"2024-06-06T04:30:04.388201+0000","last_clean":"2024-06-06T04:30:04.388201+0000","last_became_active":"2024-06-06T04:27:22.574604+0000","last_became_peered":"2024-06-06T04:27:22.574604+0000","last_unstale":"2024-06-06T04:30:04.388201+0000","last_undegraded":"2024-06-06T04:30:04.388201+0000","last_fullsized":"2024-06-06T04:30:04.388201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:17:29.395137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380448+0000","last_change":"2024-06-06T04:27:22.568554+0000","last_active":"2024-06-06T04:30:04.380448+0000","last_peered":"2024-06-06T04:30:04.380448+0000","last_clean":"2024-06-06T04:30:04.380448+0000","last_became_active":"2024-06-06T04:27:22.568196+0000","last_became_peered":"2024-06-06T04:27:22.568196+0000","last_unstale":"2024-06-06T04:30:04.380448+0000","last_undegraded":"2024-06-06T04:30:04.380448+0000","last_fullsized":"2024-06-06T04:30:04.380448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:11.564067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.372439+0000","last_change":"2024-06-06T04:27:22.565472+0000","last_active":"2024-06-06T04:30:04.372439+0000","last_peered":"2024-06-06T04:30:04.372439+0000","last_clean":"2024-06-06T04:30:04.372439+0000","last_became_active":"2024-06-06T04:27:22.565116+0000","last_became_peered":"2024-06-06T04:27:22.565116+0000","last_unstale":"2024-06-06T04:30:04.372439+0000","last_undegraded":"2024-06-06T04:30:04.372439+0000","last_fullsized":"2024-06-06T04:30:04.372439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:03:58.232707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.377900+0000","last_change":"2024-06-06T04:27:22.560210+0000","last_active":"2024-06-06T04:30:04.377900+0000","last_peered":"2024-06-06T04:30:04.377900+0000","last_clean":"2024-06-06T04:30:04.377900+0000","last_became_active":"2024-06-06T04:27:22.559853+0000","last_became_peered":"2024-06-06T04:27:22.559853+0000","last_unstale":"2024-06-06T04:30:04.377900+0000","last_undegraded":"2024-06-06T04:30:04.377900+0000","last_fullsized":"2024-06-06T04:30:04.377900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:56:18.650427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385809+0000","last_change":"2024-06-06T04:27:22.565071+0000","last_active":"2024-06-06T04:30:04.385809+0000","last_peered":"2024-06-06T04:30:04.385809+0000","last_clean":"2024-06-06T04:30:04.385809+0000","last_became_active":"2024-06-06T04:27:22.561411+0000","last_became_peered":"2024-06-06T04:27:22.561411+0000","last_unstale":"2024-06-06T04:30:04.385809+0000","last_undegraded":"2024-06-06T04:30:04.385809+0000","last_fullsized":"2024-06-06T04:30:04.385809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:41.948481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.382771+0000","last_change":"2024-06-06T04:27:22.563280+0000","last_active":"2024-06-06T04:30:04.382771+0000","last_peered":"2024-06-06T04:30:04.382771+0000","last_clean":"2024-06-06T04:30:04.382771+0000","last_became_active":"2024-06-06T04:27:22.563049+0000","last_became_peered":"2024-06-06T04:27:22.563049+0000","last_unstale":"2024-06-06T04:30:04.382771+0000","last_undegraded":"2024-06-06T04:30:04.382771+0000","last_fullsized":"2024-06-06T04:30:04.382771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:41:48.287969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.377210+0000","last_change":"2024-06-06T04:27:22.567691+0000","last_active":"2024-06-06T04:30:04.377210+0000","last_peered":"2024-06-06T04:30:04.377210+0000","last_clean":"2024-06-06T04:30:04.377210+0000","last_became_active":"2024-06-06T04:27:22.567331+0000","last_became_peered":"2024-06-06T04:27:22.567331+0000","last_unstale":"2024-06-06T04:30:04.377210+0000","last_undegraded":"2024-06-06T04:30:04.377210+0000","last_fullsized":"2024-06-06T04:30:04.377210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:03.886084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.393172+0000","last_change":"2024-06-06T04:27:22.563853+0000","last_active":"2024-06-06T04:30:04.393172+0000","last_peered":"2024-06-06T04:30:04.393172+0000","last_clean":"2024-06-06T04:30:04.393172+0000","last_became_active":"2024-06-06T04:27:22.563436+0000","last_became_peered":"2024-06-06T04:27:22.563436+0000","last_unstale":"2024-06-06T04:30:04.393172+0000","last_undegraded":"2024-06-06T04:30:04.393172+0000","last_fullsized":"2024-06-06T04:30:04.393172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:26.201308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.387060+0000","last_change":"2024-06-06T04:27:22.568189+0000","last_active":"2024-06-06T04:30:04.387060+0000","last_peered":"2024-06-06T04:30:04.387060+0000","last_clean":"2024-06-06T04:30:04.387060+0000","last_became_active":"2024-06-06T04:27:22.567815+0000","last_became_peered":"2024-06-06T04:27:22.567815+0000","last_unstale":"2024-06-06T04:30:04.387060+0000","last_undegraded":"2024-06-06T04:30:04.387060+0000","last_fullsized":"2024-06-06T04:30:04.387060+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:13:39.259012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.359906+0000","last_change":"2024-06-06T04:27:22.569698+0000","last_active":"2024-06-06T04:30:04.359906+0000","last_peered":"2024-06-06T04:30:04.359906+0000","last_clean":"2024-06-06T04:30:04.359906+0000","last_became_active":"2024-06-06T04:27:22.569347+0000","last_became_peered":"2024-06-06T04:27:22.569347+0000","last_unstale":"2024-06-06T04:30:04.359906+0000","last_undegraded":"2024-06-06T04:30:04.359906+0000","last_fullsized":"2024-06-06T04:30:04.359906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:06:34.722798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381456+0000","last_change":"2024-06-06T04:27:22.561149+0000","last_active":"2024-06-06T04:30:04.381456+0000","last_peered":"2024-06-06T04:30:04.381456+0000","last_clean":"2024-06-06T04:30:04.381456+0000","last_became_active":"2024-06-06T04:27:22.560793+0000","last_became_peered":"2024-06-06T04:27:22.560793+0000","last_unstale":"2024-06-06T04:30:04.381456+0000","last_undegraded":"2024-06-06T04:30:04.381456+0000","last_fullsized":"2024-06-06T04:30:04.381456+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:00:41.563443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394791+0000","last_change":"2024-06-06T04:27:22.569331+0000","last_active":"2024-06-06T04:30:04.394791+0000","last_peered":"2024-06-06T04:30:04.394791+0000","last_clean":"2024-06-06T04:30:04.394791+0000","last_became_active":"2024-06-06T04:27:22.569021+0000","last_became_peered":"2024-06-06T04:27:22.569021+0000","last_unstale":"2024-06-06T04:30:04.394791+0000","last_undegraded":"2024-06-06T04:30:04.394791+0000","last_fullsized":"2024-06-06T04:30:04.394791+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:52:37.064317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.391532+0000","last_change":"2024-06-06T04:27:22.574558+0000","last_active":"2024-06-06T04:30:04.391532+0000","last_peered":"2024-06-06T04:30:04.391532+0000","last_clean":"2024-06-06T04:30:04.391532+0000","last_became_active":"2024-06-06T04:27:22.574215+0000","last_became_peered":"2024-06-06T04:27:22.574215+0000","last_unstale":"2024-06-06T04:30:04.391532+0000","last_undegraded":"2024-06-06T04:30:04.391532+0000","last_fullsized":"2024-06-06T04:30:04.391532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:40:52.496322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385148+0000","last_change":"2024-06-06T04:27:22.565320+0000","last_active":"2024-06-06T04:30:04.385148+0000","last_peered":"2024-06-06T04:30:04.385148+0000","last_clean":"2024-06-06T04:30:04.385148+0000","last_became_active":"2024-06-06T04:27:22.564786+0000","last_became_peered":"2024-06-06T04:27:22.564786+0000","last_unstale":"2024-06-06T04:30:04.385148+0000","last_undegraded":"2024-06-06T04:30:04.385148+0000","last_fullsized":"2024-06-06T04:30:04.385148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:19:42.057666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249851,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78096,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1070,"num_read_kb":619,"num_write":191,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":234288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351041,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":572909,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612678,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28244,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716908,"statfs":{"total":95995035648,"available":95966113792,"internally_reserved":0,"allocated":1257472,"data_stored":1233781,"data_compressed":68587,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776203,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":577241,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037840,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573065,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299477,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":1170488,"data_compressed":68587,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463003,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":568321,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":640812,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":569709,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1200128,"data_stored":643044,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149801,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":568515,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888164,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":573994,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724639,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1175552,"data_stored":1165053,"data_compressed":68587,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34523,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34523,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":72581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":74813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T04:31:24.633 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-06-06T04:31:24.633 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-06T04:31:24.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.771+0000 7f7a3ff90640 1 -- 172.21.15.45:0/3188895301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a38101ae0 msgr2=0x7f7a38101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:24.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.771+0000 7f7a3ff90640 1 --2- 172.21.15.45:0/3188895301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a38101ae0 0x7f7a38101f60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009900 tx=0x7f7a2802a410 comp rx=0 tx=0).stop 2024-06-06T04:31:24.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.772+0000 7f7a3ff90640 1 -- 172.21.15.45:0/3188895301 shutdown_connections 2024-06-06T04:31:24.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.772+0000 7f7a3ff90640 1 --2- 172.21.15.45:0/3188895301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a381024a0 0x7f7a38108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.772+0000 7f7a3ff90640 1 --2- 172.21.15.45:0/3188895301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a38101ae0 0x7f7a38101f60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.772+0000 7f7a3ff90640 1 --2- 172.21.15.45:0/3188895301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a381008f0 0x7f7a38100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.772+0000 7f7a3ff90640 1 -- 172.21.15.45:0/3188895301 >> 172.21.15.45:0/3188895301 conn(0x7f7a380fbe80 msgr2=0x7f7a380fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:24.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.772+0000 7f7a3ff90640 1 -- 172.21.15.45:0/3188895301 shutdown_connections 2024-06-06T04:31:24.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.772+0000 7f7a3ff90640 1 -- 172.21.15.45:0/3188895301 wait complete. 2024-06-06T04:31:24.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.772+0000 7f7a3ff90640 1 Processor -- start 2024-06-06T04:31:24.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.772+0000 7f7a3ff90640 1 -- start start 2024-06-06T04:31:24.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.772+0000 7f7a3ff90640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a381008f0 0x7f7a38199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:24.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.772+0000 7f7a3ff90640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a38101ae0 0x7f7a3819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:24.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.772+0000 7f7a3ff90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a381024a0 0x7f7a381a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:24.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.772+0000 7f7a3ff90640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810c470 con 0x7f7a381008f0 2024-06-06T04:31:24.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.772+0000 7f7a3ff90640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810c2f0 con 0x7f7a381024a0 2024-06-06T04:31:24.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.772+0000 7f7a3ff90640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810c5f0 con 0x7f7a38101ae0 2024-06-06T04:31:24.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.773+0000 7f7a3dd2c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a381008f0 0x7f7a38199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:24.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.773+0000 7f7a3dd2c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a381008f0 0x7f7a38199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41206/0 (socket says 172.21.15.45:41206) 2024-06-06T04:31:24.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.773+0000 7f7a3dd2c640 1 -- 172.21.15.45:0/3870365207 learned_addr learned my addr 172.21.15.45:0/3870365207 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:24.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.773+0000 7f7a3e52d640 1 --2- 172.21.15.45:0/3870365207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a381024a0 0x7f7a381a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:24.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.773+0000 7f7a3d52b640 1 --2- 172.21.15.45:0/3870365207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a38101ae0 0x7f7a3819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:24.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.773+0000 7f7a3dd2c640 1 -- 172.21.15.45:0/3870365207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a38101ae0 msgr2=0x7f7a3819a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:24.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.773+0000 7f7a3dd2c640 1 --2- 172.21.15.45:0/3870365207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a38101ae0 0x7f7a3819a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.773+0000 7f7a3dd2c640 1 -- 172.21.15.45:0/3870365207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a381024a0 msgr2=0x7f7a381a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:24.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.773+0000 7f7a3dd2c640 1 --2- 172.21.15.45:0/3870365207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a381024a0 0x7f7a381a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.773+0000 7f7a3dd2c640 1 -- 172.21.15.45:0/3870365207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a381a1a80 con 0x7f7a381008f0 2024-06-06T04:31:24.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.773+0000 7f7a3dd2c640 1 --2- 172.21.15.45:0/3870365207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a381008f0 0x7f7a38199e60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400d770 tx=0x7f7a3400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:24.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.773+0000 7f7a2effd640 1 -- 172.21.15.45:0/3870365207 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a34017070 con 0x7f7a381008f0 2024-06-06T04:31:24.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.773+0000 7f7a2effd640 1 -- 172.21.15.45:0/3870365207 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a34009c30 con 0x7f7a381008f0 2024-06-06T04:31:24.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.773+0000 7f7a3ff90640 1 -- 172.21.15.45:0/3870365207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a381a1d70 con 0x7f7a381008f0 2024-06-06T04:31:24.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.773+0000 7f7a2effd640 1 -- 172.21.15.45:0/3870365207 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a34003d00 con 0x7f7a381008f0 2024-06-06T04:31:24.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.773+0000 7f7a3ff90640 1 -- 172.21.15.45:0/3870365207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a381a2220 con 0x7f7a381008f0 2024-06-06T04:31:24.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.774+0000 7f7a3ff90640 1 -- 172.21.15.45:0/3870365207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a381a2450 con 0x7f7a381008f0 2024-06-06T04:31:24.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.776+0000 7f7a2effd640 1 -- 172.21.15.45:0/3870365207 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f7a34003ec0 con 0x7f7a381008f0 2024-06-06T04:31:24.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.776+0000 7f7a2effd640 1 --2- 172.21.15.45:0/3870365207 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7a240a2ca0 0x7f7a240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:24.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.776+0000 7f7a2effd640 1 -- 172.21.15.45:0/3870365207 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f7a340cee90 con 0x7f7a381008f0 2024-06-06T04:31:24.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.776+0000 7f7a3d52b640 1 --2- 172.21.15.45:0/3870365207 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7a240a2ca0 0x7f7a240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:24.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.777+0000 7f7a3d52b640 1 --2- 172.21.15.45:0/3870365207 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7a240a2ca0 0x7f7a240a5160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7a28000c00 tx=0x7f7a280314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:24.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.781+0000 7f7a2effd640 1 -- 172.21.15.45:0/3870365207 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7a3409b8d0 con 0x7f7a381008f0 2024-06-06T04:31:24.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.961+0000 7f7a3ff90640 1 -- 172.21.15.45:0/3870365207 --> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a380fe070 con 0x7f7a240a2ca0 2024-06-06T04:31:24.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.971+0000 7f7a2effd640 1 -- 172.21.15.45:0/3870365207 <== mgr.14152 v2:172.21.15.45:6800/1718508550 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340942 (secure 0 0 0) 0x7f7a380fe070 con 0x7f7a240a2ca0 2024-06-06T04:31:24.975 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:31:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-06-06T04:31:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.974+0000 7f7a3ff90640 1 -- 172.21.15.45:0/3870365207 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7a240a2ca0 msgr2=0x7f7a240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.974+0000 7f7a3ff90640 1 --2- 172.21.15.45:0/3870365207 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7a240a2ca0 0x7f7a240a5160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7a28000c00 tx=0x7f7a280314a0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.974+0000 7f7a3ff90640 1 -- 172.21.15.45:0/3870365207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a381008f0 msgr2=0x7f7a38199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.974+0000 7f7a3ff90640 1 --2- 172.21.15.45:0/3870365207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a381008f0 0x7f7a38199e60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400d770 tx=0x7f7a3400dc30 comp rx=0 tx=0).stop 2024-06-06T04:31:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.974+0000 7f7a3ff90640 1 -- 172.21.15.45:0/3870365207 shutdown_connections 2024-06-06T04:31:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.974+0000 7f7a3ff90640 1 --2- 172.21.15.45:0/3870365207 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7a240a2ca0 0x7f7a240a5160 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.974+0000 7f7a3ff90640 1 --2- 172.21.15.45:0/3870365207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a381024a0 0x7f7a381a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.974+0000 7f7a3ff90640 1 --2- 172.21.15.45:0/3870365207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a38101ae0 0x7f7a3819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.974+0000 7f7a3ff90640 1 --2- 172.21.15.45:0/3870365207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a381008f0 0x7f7a38199e60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.974+0000 7f7a3ff90640 1 -- 172.21.15.45:0/3870365207 >> 172.21.15.45:0/3870365207 conn(0x7f7a380fbe80 msgr2=0x7f7a380fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.974+0000 7f7a3ff90640 1 -- 172.21.15.45:0/3870365207 shutdown_connections 2024-06-06T04:31:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:24.974+0000 7f7a3ff90640 1 -- 172.21.15.45:0/3870365207 wait complete. 2024-06-06T04:31:24.987 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-06-06T04:31:24.425531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668698,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1196,"num_read_kb":728,"num_write":374,"num_write_kb":3083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":802,"ondisk_log_size":802,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375068,"kb_used_data":13436,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566756,"statfs":{"total":1151940427776,"available":1151556358144,"internally_reserved":0,"allocated":13758464,"data_stored":8856932,"data_compressed":497595,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9090,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":47,"num_write":15,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003490"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.386822+0000","last_change":"2024-06-06T04:27:22.574900+0000","last_active":"2024-06-06T04:30:04.386822+0000","last_peered":"2024-06-06T04:30:04.386822+0000","last_clean":"2024-06-06T04:30:04.386822+0000","last_became_active":"2024-06-06T04:27:22.574389+0000","last_became_peered":"2024-06-06T04:27:22.574389+0000","last_unstale":"2024-06-06T04:30:04.386822+0000","last_undegraded":"2024-06-06T04:30:04.386822+0000","last_fullsized":"2024-06-06T04:30:04.386822+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:53:39.246463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.382712+0000","last_change":"2024-06-06T04:27:22.565543+0000","last_active":"2024-06-06T04:30:04.382712+0000","last_peered":"2024-06-06T04:30:04.382712+0000","last_clean":"2024-06-06T04:30:04.382712+0000","last_became_active":"2024-06-06T04:27:22.563833+0000","last_became_peered":"2024-06-06T04:27:22.563833+0000","last_unstale":"2024-06-06T04:30:04.382712+0000","last_undegraded":"2024-06-06T04:30:04.382712+0000","last_fullsized":"2024-06-06T04:30:04.382712+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:26:17.810782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388761+0000","last_change":"2024-06-06T04:27:22.570107+0000","last_active":"2024-06-06T04:30:04.388761+0000","last_peered":"2024-06-06T04:30:04.388761+0000","last_clean":"2024-06-06T04:30:04.388761+0000","last_became_active":"2024-06-06T04:27:22.569788+0000","last_became_peered":"2024-06-06T04:27:22.569788+0000","last_unstale":"2024-06-06T04:30:04.388761+0000","last_undegraded":"2024-06-06T04:30:04.388761+0000","last_fullsized":"2024-06-06T04:30:04.388761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:48:20.454782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385267+0000","last_change":"2024-06-06T04:27:22.559431+0000","last_active":"2024-06-06T04:30:04.385267+0000","last_peered":"2024-06-06T04:30:04.385267+0000","last_clean":"2024-06-06T04:30:04.385267+0000","last_became_active":"2024-06-06T04:27:22.558961+0000","last_became_peered":"2024-06-06T04:27:22.558961+0000","last_unstale":"2024-06-06T04:30:04.385267+0000","last_undegraded":"2024-06-06T04:30:04.385267+0000","last_fullsized":"2024-06-06T04:30:04.385267+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:59.445283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.359959+0000","last_change":"2024-06-06T04:27:22.561598+0000","last_active":"2024-06-06T04:30:04.359959+0000","last_peered":"2024-06-06T04:30:04.359959+0000","last_clean":"2024-06-06T04:30:04.359959+0000","last_became_active":"2024-06-06T04:27:22.561198+0000","last_became_peered":"2024-06-06T04:27:22.561198+0000","last_unstale":"2024-06-06T04:30:04.359959+0000","last_undegraded":"2024-06-06T04:30:04.359959+0000","last_fullsized":"2024-06-06T04:30:04.359959+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:20:28.314941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380904+0000","last_change":"2024-06-06T04:27:22.569395+0000","last_active":"2024-06-06T04:30:04.380904+0000","last_peered":"2024-06-06T04:30:04.380904+0000","last_clean":"2024-06-06T04:30:04.380904+0000","last_became_active":"2024-06-06T04:27:22.569054+0000","last_became_peered":"2024-06-06T04:27:22.569054+0000","last_unstale":"2024-06-06T04:30:04.380904+0000","last_undegraded":"2024-06-06T04:30:04.380904+0000","last_fullsized":"2024-06-06T04:30:04.380904+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:03:56.630677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392227+0000","last_change":"2024-06-06T04:27:22.577121+0000","last_active":"2024-06-06T04:30:04.392227+0000","last_peered":"2024-06-06T04:30:04.392227+0000","last_clean":"2024-06-06T04:30:04.392227+0000","last_became_active":"2024-06-06T04:27:22.575574+0000","last_became_peered":"2024-06-06T04:27:22.575574+0000","last_unstale":"2024-06-06T04:30:04.392227+0000","last_undegraded":"2024-06-06T04:30:04.392227+0000","last_fullsized":"2024-06-06T04:30:04.392227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:05:16.799335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.383226+0000","last_change":"2024-06-06T04:27:22.570051+0000","last_active":"2024-06-06T04:30:04.383226+0000","last_peered":"2024-06-06T04:30:04.383226+0000","last_clean":"2024-06-06T04:30:04.383226+0000","last_became_active":"2024-06-06T04:27:22.569640+0000","last_became_peered":"2024-06-06T04:27:22.569640+0000","last_unstale":"2024-06-06T04:30:04.383226+0000","last_undegraded":"2024-06-06T04:30:04.383226+0000","last_fullsized":"2024-06-06T04:30:04.383226+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:48:00.744167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.396353+0000","last_change":"2024-06-06T04:27:22.574900+0000","last_active":"2024-06-06T04:30:04.396353+0000","last_peered":"2024-06-06T04:30:04.396353+0000","last_clean":"2024-06-06T04:30:04.396353+0000","last_became_active":"2024-06-06T04:27:22.574570+0000","last_became_peered":"2024-06-06T04:27:22.574570+0000","last_unstale":"2024-06-06T04:30:04.396353+0000","last_undegraded":"2024-06-06T04:30:04.396353+0000","last_fullsized":"2024-06-06T04:30:04.396353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:46:54.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380728+0000","last_change":"2024-06-06T04:27:22.570581+0000","last_active":"2024-06-06T04:30:04.380728+0000","last_peered":"2024-06-06T04:30:04.380728+0000","last_clean":"2024-06-06T04:30:04.380728+0000","last_became_active":"2024-06-06T04:27:22.570243+0000","last_became_peered":"2024-06-06T04:27:22.570243+0000","last_unstale":"2024-06-06T04:30:04.380728+0000","last_undegraded":"2024-06-06T04:30:04.380728+0000","last_fullsized":"2024-06-06T04:30:04.380728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:27.087527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.395227+0000","last_change":"2024-06-06T04:27:22.571162+0000","last_active":"2024-06-06T04:30:04.395227+0000","last_peered":"2024-06-06T04:30:04.395227+0000","last_clean":"2024-06-06T04:30:04.395227+0000","last_became_active":"2024-06-06T04:27:22.570843+0000","last_became_peered":"2024-06-06T04:27:22.570843+0000","last_unstale":"2024-06-06T04:30:04.395227+0000","last_undegraded":"2024-06-06T04:30:04.395227+0000","last_fullsized":"2024-06-06T04:30:04.395227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:43:05.590240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.384953+0000","last_change":"2024-06-06T04:27:22.567420+0000","last_active":"2024-06-06T04:30:04.384953+0000","last_peered":"2024-06-06T04:30:04.384953+0000","last_clean":"2024-06-06T04:30:04.384953+0000","last_became_active":"2024-06-06T04:27:22.566750+0000","last_became_peered":"2024-06-06T04:27:22.566750+0000","last_unstale":"2024-06-06T04:30:04.384953+0000","last_undegraded":"2024-06-06T04:30:04.384953+0000","last_fullsized":"2024-06-06T04:30:04.384953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:21.669965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.364827+0000","last_change":"2024-06-06T04:27:22.566831+0000","last_active":"2024-06-06T04:30:04.364827+0000","last_peered":"2024-06-06T04:30:04.364827+0000","last_clean":"2024-06-06T04:30:04.364827+0000","last_became_active":"2024-06-06T04:27:22.566255+0000","last_became_peered":"2024-06-06T04:27:22.566255+0000","last_unstale":"2024-06-06T04:30:04.364827+0000","last_undegraded":"2024-06-06T04:30:04.364827+0000","last_fullsized":"2024-06-06T04:30:04.364827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:40:22.486674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.384669+0000","last_change":"2024-06-06T04:27:22.564596+0000","last_active":"2024-06-06T04:30:04.384669+0000","last_peered":"2024-06-06T04:30:04.384669+0000","last_clean":"2024-06-06T04:30:04.384669+0000","last_became_active":"2024-06-06T04:27:22.563694+0000","last_became_peered":"2024-06-06T04:27:22.563694+0000","last_unstale":"2024-06-06T04:30:04.384669+0000","last_undegraded":"2024-06-06T04:30:04.384669+0000","last_fullsized":"2024-06-06T04:30:04.384669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:18.568266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385298+0000","last_change":"2024-06-06T04:27:22.559395+0000","last_active":"2024-06-06T04:30:04.385298+0000","last_peered":"2024-06-06T04:30:04.385298+0000","last_clean":"2024-06-06T04:30:04.385298+0000","last_became_active":"2024-06-06T04:27:22.559030+0000","last_became_peered":"2024-06-06T04:27:22.559030+0000","last_unstale":"2024-06-06T04:30:04.385298+0000","last_undegraded":"2024-06-06T04:30:04.385298+0000","last_fullsized":"2024-06-06T04:30:04.385298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:08.562184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394517+0000","last_change":"2024-06-06T04:27:22.569657+0000","last_active":"2024-06-06T04:30:04.394517+0000","last_peered":"2024-06-06T04:30:04.394517+0000","last_clean":"2024-06-06T04:30:04.394517+0000","last_became_active":"2024-06-06T04:27:22.569359+0000","last_became_peered":"2024-06-06T04:27:22.569359+0000","last_unstale":"2024-06-06T04:30:04.394517+0000","last_undegraded":"2024-06-06T04:30:04.394517+0000","last_fullsized":"2024-06-06T04:30:04.394517+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:42:20.197715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394347+0000","last_change":"2024-06-06T04:27:22.574661+0000","last_active":"2024-06-06T04:30:04.394347+0000","last_peered":"2024-06-06T04:30:04.394347+0000","last_clean":"2024-06-06T04:30:04.394347+0000","last_became_active":"2024-06-06T04:27:22.574310+0000","last_became_peered":"2024-06-06T04:27:22.574310+0000","last_unstale":"2024-06-06T04:30:04.394347+0000","last_undegraded":"2024-06-06T04:30:04.394347+0000","last_fullsized":"2024-06-06T04:30:04.394347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:45:18.772270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388153+0000","last_change":"2024-06-06T04:27:22.565575+0000","last_active":"2024-06-06T04:30:04.388153+0000","last_peered":"2024-06-06T04:30:04.388153+0000","last_clean":"2024-06-06T04:30:04.388153+0000","last_became_active":"2024-06-06T04:27:22.565256+0000","last_became_peered":"2024-06-06T04:27:22.565256+0000","last_unstale":"2024-06-06T04:30:04.388153+0000","last_undegraded":"2024-06-06T04:30:04.388153+0000","last_fullsized":"2024-06-06T04:30:04.388153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:49.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.365082+0000","last_change":"2024-06-06T04:27:22.559606+0000","last_active":"2024-06-06T04:30:04.365082+0000","last_peered":"2024-06-06T04:30:04.365082+0000","last_clean":"2024-06-06T04:30:04.365082+0000","last_became_active":"2024-06-06T04:27:22.557881+0000","last_became_peered":"2024-06-06T04:27:22.557881+0000","last_unstale":"2024-06-06T04:30:04.365082+0000","last_undegraded":"2024-06-06T04:30:04.365082+0000","last_fullsized":"2024-06-06T04:30:04.365082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:31:06.269785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905476+0000","last_change":"2024-06-06T04:27:32.626612+0000","last_active":"2024-06-06T04:29:05.905476+0000","last_peered":"2024-06-06T04:29:05.905476+0000","last_clean":"2024-06-06T04:29:05.905476+0000","last_became_active":"2024-06-06T04:27:32.626113+0000","last_became_peered":"2024-06-06T04:27:32.626113+0000","last_unstale":"2024-06-06T04:29:05.905476+0000","last_undegraded":"2024-06-06T04:29:05.905476+0000","last_fullsized":"2024-06-06T04:29:05.905476+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:14:32.318738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908330+0000","last_change":"2024-06-06T04:27:19.302744+0000","last_active":"2024-06-06T04:29:05.908330+0000","last_peered":"2024-06-06T04:29:05.908330+0000","last_clean":"2024-06-06T04:29:05.908330+0000","last_became_active":"2024-06-06T04:27:19.302389+0000","last_became_peered":"2024-06-06T04:27:19.302389+0000","last_unstale":"2024-06-06T04:29:05.908330+0000","last_undegraded":"2024-06-06T04:29:05.908330+0000","last_fullsized":"2024-06-06T04:29:05.908330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:54:49.872618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381219+0000","last_change":"2024-06-06T04:27:22.570220+0000","last_active":"2024-06-06T04:30:04.381219+0000","last_peered":"2024-06-06T04:30:04.381219+0000","last_clean":"2024-06-06T04:30:04.381219+0000","last_became_active":"2024-06-06T04:27:22.569835+0000","last_became_peered":"2024-06-06T04:27:22.569835+0000","last_unstale":"2024-06-06T04:30:04.381219+0000","last_undegraded":"2024-06-06T04:30:04.381219+0000","last_fullsized":"2024-06-06T04:30:04.381219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:54:57.372561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401940+0000","last_change":"2024-06-06T04:27:32.622407+0000","last_active":"2024-06-06T04:29:06.401940+0000","last_peered":"2024-06-06T04:29:06.401940+0000","last_clean":"2024-06-06T04:29:06.401940+0000","last_became_active":"2024-06-06T04:27:32.621868+0000","last_became_peered":"2024-06-06T04:27:32.621868+0000","last_unstale":"2024-06-06T04:29:06.401940+0000","last_undegraded":"2024-06-06T04:29:06.401940+0000","last_fullsized":"2024-06-06T04:29:06.401940+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:50:05.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:21.905209+0000","last_change":"2024-06-06T04:27:19.516235+0000","last_active":"2024-06-06T04:29:21.905209+0000","last_peered":"2024-06-06T04:29:21.905209+0000","last_clean":"2024-06-06T04:29:21.905209+0000","last_became_active":"2024-06-06T04:27:19.515849+0000","last_became_peered":"2024-06-06T04:27:19.515849+0000","last_unstale":"2024-06-06T04:29:21.905209+0000","last_undegraded":"2024-06-06T04:29:21.905209+0000","last_fullsized":"2024-06-06T04:29:21.905209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:52:42.722103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388234+0000","last_change":"2024-06-06T04:27:22.570797+0000","last_active":"2024-06-06T04:30:04.388234+0000","last_peered":"2024-06-06T04:30:04.388234+0000","last_clean":"2024-06-06T04:30:04.388234+0000","last_became_active":"2024-06-06T04:27:22.570305+0000","last_became_peered":"2024-06-06T04:27:22.570305+0000","last_unstale":"2024-06-06T04:30:04.388234+0000","last_undegraded":"2024-06-06T04:30:04.388234+0000","last_fullsized":"2024-06-06T04:30:04.388234+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:04:14.545306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401666+0000","last_change":"2024-06-06T04:27:32.623241+0000","last_active":"2024-06-06T04:29:06.401666+0000","last_peered":"2024-06-06T04:29:06.401666+0000","last_clean":"2024-06-06T04:29:06.401666+0000","last_became_active":"2024-06-06T04:27:32.622751+0000","last_became_peered":"2024-06-06T04:27:32.622751+0000","last_unstale":"2024-06-06T04:29:06.401666+0000","last_undegraded":"2024-06-06T04:29:06.401666+0000","last_fullsized":"2024-06-06T04:29:06.401666+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:24:53.446329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.909112+0000","last_change":"2024-06-06T04:28:05.733350+0000","last_active":"2024-06-06T04:29:05.909112+0000","last_peered":"2024-06-06T04:29:05.909112+0000","last_clean":"2024-06-06T04:29:05.909112+0000","last_became_active":"2024-06-06T04:28:05.726556+0000","last_became_peered":"2024-06-06T04:28:05.726556+0000","last_unstale":"2024-06-06T04:29:05.909112+0000","last_undegraded":"2024-06-06T04:29:05.909112+0000","last_fullsized":"2024-06-06T04:29:05.909112+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:26.636625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.396094+0000","last_change":"2024-06-06T04:27:22.571731+0000","last_active":"2024-06-06T04:30:04.396094+0000","last_peered":"2024-06-06T04:30:04.396094+0000","last_clean":"2024-06-06T04:30:04.396094+0000","last_became_active":"2024-06-06T04:27:22.571398+0000","last_became_peered":"2024-06-06T04:27:22.571398+0000","last_unstale":"2024-06-06T04:30:04.396094+0000","last_undegraded":"2024-06-06T04:30:04.396094+0000","last_fullsized":"2024-06-06T04:30:04.396094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:22:45.144427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401661+0000","last_change":"2024-06-06T04:27:32.618465+0000","last_active":"2024-06-06T04:29:06.401661+0000","last_peered":"2024-06-06T04:29:06.401661+0000","last_clean":"2024-06-06T04:29:06.401661+0000","last_became_active":"2024-06-06T04:27:32.618066+0000","last_became_peered":"2024-06-06T04:27:32.618066+0000","last_unstale":"2024-06-06T04:29:06.401661+0000","last_undegraded":"2024-06-06T04:29:06.401661+0000","last_fullsized":"2024-06-06T04:29:06.401661+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:54:23.988687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908783+0000","last_change":"2024-06-06T04:28:05.730381+0000","last_active":"2024-06-06T04:29:05.908783+0000","last_peered":"2024-06-06T04:29:05.908783+0000","last_clean":"2024-06-06T04:29:05.908783+0000","last_became_active":"2024-06-06T04:28:05.726552+0000","last_became_peered":"2024-06-06T04:28:05.726552+0000","last_unstale":"2024-06-06T04:29:05.908783+0000","last_undegraded":"2024-06-06T04:29:05.908783+0000","last_fullsized":"2024-06-06T04:29:05.908783+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:08:52.903042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394789+0000","last_change":"2024-06-06T04:27:22.573769+0000","last_active":"2024-06-06T04:30:04.394789+0000","last_peered":"2024-06-06T04:30:04.394789+0000","last_clean":"2024-06-06T04:30:04.394789+0000","last_became_active":"2024-06-06T04:27:22.572674+0000","last_became_peered":"2024-06-06T04:27:22.572674+0000","last_unstale":"2024-06-06T04:30:04.394789+0000","last_undegraded":"2024-06-06T04:30:04.394789+0000","last_fullsized":"2024-06-06T04:30:04.394789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:57:22.653052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905675+0000","last_change":"2024-06-06T04:27:32.617934+0000","last_active":"2024-06-06T04:29:05.905675+0000","last_peered":"2024-06-06T04:29:05.905675+0000","last_clean":"2024-06-06T04:29:05.905675+0000","last_became_active":"2024-06-06T04:27:32.617537+0000","last_became_peered":"2024-06-06T04:27:32.617537+0000","last_unstale":"2024-06-06T04:29:05.905675+0000","last_undegraded":"2024-06-06T04:29:05.905675+0000","last_fullsized":"2024-06-06T04:29:05.905675+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:21:48.319434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'7","reported_seq":228,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:19.968167+0000","last_change":"2024-06-06T04:27:19.306236+0000","last_active":"2024-06-06T04:31:19.968167+0000","last_peered":"2024-06-06T04:31:19.968167+0000","last_clean":"2024-06-06T04:31:19.968167+0000","last_became_active":"2024-06-06T04:27:19.305850+0000","last_became_peered":"2024-06-06T04:27:19.305850+0000","last_unstale":"2024-06-06T04:31:19.968167+0000","last_undegraded":"2024-06-06T04:31:19.968167+0000","last_fullsized":"2024-06-06T04:31:19.968167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:58:47.310040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1504,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.365431+0000","last_change":"2024-06-06T04:27:22.567440+0000","last_active":"2024-06-06T04:30:04.365431+0000","last_peered":"2024-06-06T04:30:04.365431+0000","last_clean":"2024-06-06T04:30:04.365431+0000","last_became_active":"2024-06-06T04:27:22.566732+0000","last_became_peered":"2024-06-06T04:27:22.566732+0000","last_unstale":"2024-06-06T04:30:04.365431+0000","last_undegraded":"2024-06-06T04:30:04.365431+0000","last_fullsized":"2024-06-06T04:30:04.365431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:48:46.270103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905710+0000","last_change":"2024-06-06T04:27:32.626554+0000","last_active":"2024-06-06T04:29:05.905710+0000","last_peered":"2024-06-06T04:29:05.905710+0000","last_clean":"2024-06-06T04:29:05.905710+0000","last_became_active":"2024-06-06T04:27:32.626108+0000","last_became_peered":"2024-06-06T04:27:32.626108+0000","last_unstale":"2024-06-06T04:29:05.905710+0000","last_undegraded":"2024-06-06T04:29:05.905710+0000","last_fullsized":"2024-06-06T04:29:05.905710+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:37:43.811005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.262695+0000","last_change":"2024-06-06T04:27:19.306821+0000","last_active":"2024-06-06T04:29:06.262695+0000","last_peered":"2024-06-06T04:29:06.262695+0000","last_clean":"2024-06-06T04:29:06.262695+0000","last_became_active":"2024-06-06T04:27:19.306489+0000","last_became_peered":"2024-06-06T04:27:19.306489+0000","last_unstale":"2024-06-06T04:29:06.262695+0000","last_undegraded":"2024-06-06T04:29:06.262695+0000","last_fullsized":"2024-06-06T04:29:06.262695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:31:28.973112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.378929+0000","last_change":"2024-06-06T04:27:22.570617+0000","last_active":"2024-06-06T04:30:04.378929+0000","last_peered":"2024-06-06T04:30:04.378929+0000","last_clean":"2024-06-06T04:30:04.378929+0000","last_became_active":"2024-06-06T04:27:22.570271+0000","last_became_peered":"2024-06-06T04:27:22.570271+0000","last_unstale":"2024-06-06T04:30:04.378929+0000","last_undegraded":"2024-06-06T04:30:04.378929+0000","last_fullsized":"2024-06-06T04:30:04.378929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:45:33.890164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.909010+0000","last_change":"2024-06-06T04:27:32.628132+0000","last_active":"2024-06-06T04:29:05.909010+0000","last_peered":"2024-06-06T04:29:05.909010+0000","last_clean":"2024-06-06T04:29:05.909010+0000","last_became_active":"2024-06-06T04:27:32.627658+0000","last_became_peered":"2024-06-06T04:27:32.627658+0000","last_unstale":"2024-06-06T04:29:05.909010+0000","last_undegraded":"2024-06-06T04:29:05.909010+0000","last_fullsized":"2024-06-06T04:29:05.909010+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:49:24.817079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904125+0000","last_change":"2024-06-06T04:27:19.300510+0000","last_active":"2024-06-06T04:29:05.904125+0000","last_peered":"2024-06-06T04:29:05.904125+0000","last_clean":"2024-06-06T04:29:05.904125+0000","last_became_active":"2024-06-06T04:27:19.300150+0000","last_became_peered":"2024-06-06T04:27:19.300150+0000","last_unstale":"2024-06-06T04:29:05.904125+0000","last_undegraded":"2024-06-06T04:29:05.904125+0000","last_fullsized":"2024-06-06T04:29:05.904125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:38:25.084690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381098+0000","last_change":"2024-06-06T04:27:22.563864+0000","last_active":"2024-06-06T04:30:04.381098+0000","last_peered":"2024-06-06T04:30:04.381098+0000","last_clean":"2024-06-06T04:30:04.381098+0000","last_became_active":"2024-06-06T04:27:22.563555+0000","last_became_peered":"2024-06-06T04:27:22.563555+0000","last_unstale":"2024-06-06T04:30:04.381098+0000","last_undegraded":"2024-06-06T04:30:04.381098+0000","last_fullsized":"2024-06-06T04:30:04.381098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:39:33.707404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.165015+0000","last_change":"2024-06-06T04:27:32.617627+0000","last_active":"2024-06-06T04:29:06.165015+0000","last_peered":"2024-06-06T04:29:06.165015+0000","last_clean":"2024-06-06T04:29:06.165015+0000","last_became_active":"2024-06-06T04:27:32.617187+0000","last_became_peered":"2024-06-06T04:27:32.617187+0000","last_unstale":"2024-06-06T04:29:06.165015+0000","last_undegraded":"2024-06-06T04:29:06.165015+0000","last_fullsized":"2024-06-06T04:29:06.165015+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:24:52.554176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790234+0000","last_change":"2024-06-06T04:27:19.304316+0000","last_active":"2024-06-06T04:29:06.790234+0000","last_peered":"2024-06-06T04:29:06.790234+0000","last_clean":"2024-06-06T04:29:06.790234+0000","last_became_active":"2024-06-06T04:27:19.303952+0000","last_became_peered":"2024-06-06T04:27:19.303952+0000","last_unstale":"2024-06-06T04:29:06.790234+0000","last_undegraded":"2024-06-06T04:29:06.790234+0000","last_fullsized":"2024-06-06T04:29:06.790234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:22:26.886916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.371820+0000","last_change":"2024-06-06T04:27:22.561824+0000","last_active":"2024-06-06T04:30:04.371820+0000","last_peered":"2024-06-06T04:30:04.371820+0000","last_clean":"2024-06-06T04:30:04.371820+0000","last_became_active":"2024-06-06T04:27:22.561495+0000","last_became_peered":"2024-06-06T04:27:22.561495+0000","last_unstale":"2024-06-06T04:30:04.371820+0000","last_undegraded":"2024-06-06T04:30:04.371820+0000","last_fullsized":"2024-06-06T04:30:04.371820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:40:31.314627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560380+0000","last_change":"2024-06-06T04:27:32.624540+0000","last_active":"2024-06-06T04:29:06.560380+0000","last_peered":"2024-06-06T04:29:06.560380+0000","last_clean":"2024-06-06T04:29:06.560380+0000","last_became_active":"2024-06-06T04:27:32.624036+0000","last_became_peered":"2024-06-06T04:27:32.624036+0000","last_unstale":"2024-06-06T04:29:06.560380+0000","last_undegraded":"2024-06-06T04:29:06.560380+0000","last_fullsized":"2024-06-06T04:29:06.560380+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:16.983576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'4","reported_seq":219,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:19.967188+0000","last_change":"2024-06-06T04:27:19.304614+0000","last_active":"2024-06-06T04:31:19.967188+0000","last_peered":"2024-06-06T04:31:19.967188+0000","last_clean":"2024-06-06T04:31:19.967188+0000","last_became_active":"2024-06-06T04:27:19.304126+0000","last_became_peered":"2024-06-06T04:27:19.304126+0000","last_unstale":"2024-06-06T04:31:19.967188+0000","last_undegraded":"2024-06-06T04:31:19.967188+0000","last_fullsized":"2024-06-06T04:31:19.967188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:07:39.908630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":172,"num_read_kb":172,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.396136+0000","last_change":"2024-06-06T04:27:22.572534+0000","last_active":"2024-06-06T04:30:04.396136+0000","last_peered":"2024-06-06T04:30:04.396136+0000","last_clean":"2024-06-06T04:30:04.396136+0000","last_became_active":"2024-06-06T04:27:22.572214+0000","last_became_peered":"2024-06-06T04:27:22.572214+0000","last_unstale":"2024-06-06T04:30:04.396136+0000","last_undegraded":"2024-06-06T04:30:04.396136+0000","last_fullsized":"2024-06-06T04:30:04.396136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:13.176045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.262366+0000","last_change":"2024-06-06T04:28:05.723450+0000","last_active":"2024-06-06T04:29:06.262366+0000","last_peered":"2024-06-06T04:29:06.262366+0000","last_clean":"2024-06-06T04:29:06.262366+0000","last_became_active":"2024-06-06T04:28:05.722928+0000","last_became_peered":"2024-06-06T04:28:05.722928+0000","last_unstale":"2024-06-06T04:29:06.262366+0000","last_undegraded":"2024-06-06T04:29:06.262366+0000","last_fullsized":"2024-06-06T04:29:06.262366+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:49:53.093276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905918+0000","last_change":"2024-06-06T04:27:19.308465+0000","last_active":"2024-06-06T04:29:05.905918+0000","last_peered":"2024-06-06T04:29:05.905918+0000","last_clean":"2024-06-06T04:29:05.905918+0000","last_became_active":"2024-06-06T04:27:19.308077+0000","last_became_peered":"2024-06-06T04:27:19.308077+0000","last_unstale":"2024-06-06T04:29:05.905918+0000","last_undegraded":"2024-06-06T04:29:05.905918+0000","last_fullsized":"2024-06-06T04:29:05.905918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:37:48.282613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385739+0000","last_change":"2024-06-06T04:27:22.574141+0000","last_active":"2024-06-06T04:30:04.385739+0000","last_peered":"2024-06-06T04:30:04.385739+0000","last_clean":"2024-06-06T04:30:04.385739+0000","last_became_active":"2024-06-06T04:27:22.573679+0000","last_became_peered":"2024-06-06T04:27:22.573679+0000","last_unstale":"2024-06-06T04:30:04.385739+0000","last_undegraded":"2024-06-06T04:30:04.385739+0000","last_fullsized":"2024-06-06T04:30:04.385739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:27:25.177990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401856+0000","last_change":"2024-06-06T04:27:32.624148+0000","last_active":"2024-06-06T04:29:06.401856+0000","last_peered":"2024-06-06T04:29:06.401856+0000","last_clean":"2024-06-06T04:29:06.401856+0000","last_became_active":"2024-06-06T04:27:32.623448+0000","last_became_peered":"2024-06-06T04:27:32.623448+0000","last_unstale":"2024-06-06T04:29:06.401856+0000","last_undegraded":"2024-06-06T04:29:06.401856+0000","last_fullsized":"2024-06-06T04:29:06.401856+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:18:02.037296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790253+0000","last_change":"2024-06-06T04:27:19.516875+0000","last_active":"2024-06-06T04:29:06.790253+0000","last_peered":"2024-06-06T04:29:06.790253+0000","last_clean":"2024-06-06T04:29:06.790253+0000","last_became_active":"2024-06-06T04:27:19.516393+0000","last_became_peered":"2024-06-06T04:27:19.516393+0000","last_unstale":"2024-06-06T04:29:06.790253+0000","last_undegraded":"2024-06-06T04:29:06.790253+0000","last_fullsized":"2024-06-06T04:29:06.790253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:25:20.079218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392596+0000","last_change":"2024-06-06T04:28:05.724043+0000","last_active":"2024-06-06T04:30:04.392596+0000","last_peered":"2024-06-06T04:30:04.392596+0000","last_clean":"2024-06-06T04:30:04.392596+0000","last_became_active":"2024-06-06T04:28:05.723622+0000","last_became_peered":"2024-06-06T04:28:05.723622+0000","last_unstale":"2024-06-06T04:30:04.392596+0000","last_undegraded":"2024-06-06T04:30:04.392596+0000","last_fullsized":"2024-06-06T04:30:04.392596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:42.162400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.907660+0000","last_change":"2024-06-06T04:27:32.626366+0000","last_active":"2024-06-06T04:29:05.907660+0000","last_peered":"2024-06-06T04:29:05.907660+0000","last_clean":"2024-06-06T04:29:05.907660+0000","last_became_active":"2024-06-06T04:27:32.625847+0000","last_became_peered":"2024-06-06T04:27:32.625847+0000","last_unstale":"2024-06-06T04:29:05.907660+0000","last_undegraded":"2024-06-06T04:29:05.907660+0000","last_fullsized":"2024-06-06T04:29:05.907660+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:47:44.167531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'42","reported_seq":441,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:20.968878+0000","last_change":"2024-06-06T04:27:19.303581+0000","last_active":"2024-06-06T04:31:20.968878+0000","last_peered":"2024-06-06T04:31:20.968878+0000","last_clean":"2024-06-06T04:31:20.968878+0000","last_became_active":"2024-06-06T04:27:19.303234+0000","last_became_peered":"2024-06-06T04:27:19.303234+0000","last_unstale":"2024-06-06T04:31:20.968878+0000","last_undegraded":"2024-06-06T04:31:20.968878+0000","last_fullsized":"2024-06-06T04:31:20.968878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:37.721588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72581,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":356,"num_read_kb":250,"num_write":43,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.391077+0000","last_change":"2024-06-06T04:27:22.567177+0000","last_active":"2024-06-06T04:30:04.391077+0000","last_peered":"2024-06-06T04:30:04.391077+0000","last_clean":"2024-06-06T04:30:04.391077+0000","last_became_active":"2024-06-06T04:27:22.566751+0000","last_became_peered":"2024-06-06T04:27:22.566751+0000","last_unstale":"2024-06-06T04:30:04.391077+0000","last_undegraded":"2024-06-06T04:30:04.391077+0000","last_fullsized":"2024-06-06T04:30:04.391077+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:37:15.914505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790613+0000","last_change":"2024-06-06T04:27:32.619319+0000","last_active":"2024-06-06T04:29:06.790613+0000","last_peered":"2024-06-06T04:29:06.790613+0000","last_clean":"2024-06-06T04:29:06.790613+0000","last_became_active":"2024-06-06T04:27:32.618843+0000","last_became_peered":"2024-06-06T04:27:32.618843+0000","last_unstale":"2024-06-06T04:29:06.790613+0000","last_undegraded":"2024-06-06T04:29:06.790613+0000","last_fullsized":"2024-06-06T04:29:06.790613+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:32:20.071512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905924+0000","last_change":"2024-06-06T04:27:19.304731+0000","last_active":"2024-06-06T04:29:05.905924+0000","last_peered":"2024-06-06T04:29:05.905924+0000","last_clean":"2024-06-06T04:29:05.905924+0000","last_became_active":"2024-06-06T04:27:19.304351+0000","last_became_peered":"2024-06-06T04:27:19.304351+0000","last_unstale":"2024-06-06T04:29:05.905924+0000","last_undegraded":"2024-06-06T04:29:05.905924+0000","last_fullsized":"2024-06-06T04:29:05.905924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:16:14.136633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.391311+0000","last_change":"2024-06-06T04:29:05.909041+0000","last_active":"2024-06-06T04:30:04.391311+0000","last_peered":"2024-06-06T04:30:04.391311+0000","last_clean":"2024-06-06T04:30:04.391311+0000","last_became_active":"2024-06-06T04:29:05.908666+0000","last_became_peered":"2024-06-06T04:29:05.908666+0000","last_unstale":"2024-06-06T04:30:04.391311+0000","last_undegraded":"2024-06-06T04:30:04.391311+0000","last_fullsized":"2024-06-06T04:30:04.391311+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:47:55.225572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908861+0000","last_change":"2024-06-06T04:27:32.628645+0000","last_active":"2024-06-06T04:29:05.908861+0000","last_peered":"2024-06-06T04:29:05.908861+0000","last_clean":"2024-06-06T04:29:05.908861+0000","last_became_active":"2024-06-06T04:27:32.627586+0000","last_became_peered":"2024-06-06T04:27:32.627586+0000","last_unstale":"2024-06-06T04:29:05.908861+0000","last_undegraded":"2024-06-06T04:29:05.908861+0000","last_fullsized":"2024-06-06T04:29:05.908861+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:46:53.094511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790029+0000","last_change":"2024-06-06T04:27:19.303805+0000","last_active":"2024-06-06T04:29:06.790029+0000","last_peered":"2024-06-06T04:29:06.790029+0000","last_clean":"2024-06-06T04:29:06.790029+0000","last_became_active":"2024-06-06T04:27:19.303391+0000","last_became_peered":"2024-06-06T04:27:19.303391+0000","last_unstale":"2024-06-06T04:29:06.790029+0000","last_undegraded":"2024-06-06T04:29:06.790029+0000","last_fullsized":"2024-06-06T04:29:06.790029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:30:27.336567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.387613+0000","last_change":"2024-06-06T04:27:22.575148+0000","last_active":"2024-06-06T04:30:04.387613+0000","last_peered":"2024-06-06T04:30:04.387613+0000","last_clean":"2024-06-06T04:30:04.387613+0000","last_became_active":"2024-06-06T04:27:22.574591+0000","last_became_peered":"2024-06-06T04:27:22.574591+0000","last_unstale":"2024-06-06T04:30:04.387613+0000","last_undegraded":"2024-06-06T04:30:04.387613+0000","last_fullsized":"2024-06-06T04:30:04.387613+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:35:23.404644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214440+0000","last_change":"2024-06-06T04:28:05.726734+0000","last_active":"2024-06-06T04:29:06.214440+0000","last_peered":"2024-06-06T04:29:06.214440+0000","last_clean":"2024-06-06T04:29:06.214440+0000","last_became_active":"2024-06-06T04:28:05.726238+0000","last_became_peered":"2024-06-06T04:28:05.726238+0000","last_unstale":"2024-06-06T04:29:06.214440+0000","last_undegraded":"2024-06-06T04:29:06.214440+0000","last_fullsized":"2024-06-06T04:29:06.214440+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:37:47.841475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"100'6","reported_seq":228,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:20.968408+0000","last_change":"2024-06-06T04:27:19.305571+0000","last_active":"2024-06-06T04:31:20.968408+0000","last_peered":"2024-06-06T04:31:20.968408+0000","last_clean":"2024-06-06T04:31:20.968408+0000","last_became_active":"2024-06-06T04:27:19.305183+0000","last_became_peered":"2024-06-06T04:27:19.305183+0000","last_unstale":"2024-06-06T04:31:20.968408+0000","last_undegraded":"2024-06-06T04:31:20.968408+0000","last_fullsized":"2024-06-06T04:31:20.968408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:58:51.205209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1611,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392588+0000","last_change":"2024-06-06T04:29:06.263872+0000","last_active":"2024-06-06T04:30:04.392588+0000","last_peered":"2024-06-06T04:30:04.392588+0000","last_clean":"2024-06-06T04:30:04.392588+0000","last_became_active":"2024-06-06T04:29:06.263487+0000","last_became_peered":"2024-06-06T04:29:06.263487+0000","last_unstale":"2024-06-06T04:30:04.392588+0000","last_undegraded":"2024-06-06T04:30:04.392588+0000","last_fullsized":"2024-06-06T04:30:04.392588+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:15.965926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904639+0000","last_change":"2024-06-06T04:28:05.724892+0000","last_active":"2024-06-06T04:29:05.904639+0000","last_peered":"2024-06-06T04:29:05.904639+0000","last_clean":"2024-06-06T04:29:05.904639+0000","last_became_active":"2024-06-06T04:28:05.724420+0000","last_became_peered":"2024-06-06T04:28:05.724420+0000","last_unstale":"2024-06-06T04:29:05.904639+0000","last_undegraded":"2024-06-06T04:29:05.904639+0000","last_fullsized":"2024-06-06T04:29:05.904639+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:47:39.879481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":60,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.213846+0000","last_change":"2024-06-06T04:27:19.516687+0000","last_active":"2024-06-06T04:29:06.213846+0000","last_peered":"2024-06-06T04:29:06.213846+0000","last_clean":"2024-06-06T04:29:06.213846+0000","last_became_active":"2024-06-06T04:27:19.516271+0000","last_became_peered":"2024-06-06T04:27:19.516271+0000","last_unstale":"2024-06-06T04:29:06.213846+0000","last_undegraded":"2024-06-06T04:29:06.213846+0000","last_fullsized":"2024-06-06T04:29:06.213846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:33:46.817727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394832+0000","last_change":"2024-06-06T04:27:22.572671+0000","last_active":"2024-06-06T04:30:04.394832+0000","last_peered":"2024-06-06T04:30:04.394832+0000","last_clean":"2024-06-06T04:30:04.394832+0000","last_became_active":"2024-06-06T04:27:22.572230+0000","last_became_peered":"2024-06-06T04:27:22.572230+0000","last_unstale":"2024-06-06T04:30:04.394832+0000","last_undegraded":"2024-06-06T04:30:04.394832+0000","last_fullsized":"2024-06-06T04:30:04.394832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:36:48.393092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.789710+0000","last_change":"2024-06-06T04:27:32.620131+0000","last_active":"2024-06-06T04:29:06.789710+0000","last_peered":"2024-06-06T04:29:06.789710+0000","last_clean":"2024-06-06T04:29:06.789710+0000","last_became_active":"2024-06-06T04:27:32.619686+0000","last_became_peered":"2024-06-06T04:27:32.619686+0000","last_unstale":"2024-06-06T04:29:06.789710+0000","last_undegraded":"2024-06-06T04:29:06.789710+0000","last_fullsized":"2024-06-06T04:29:06.789710+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:19:17.130410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214545+0000","last_change":"2024-06-06T04:27:19.304682+0000","last_active":"2024-06-06T04:29:06.214545+0000","last_peered":"2024-06-06T04:29:06.214545+0000","last_clean":"2024-06-06T04:29:06.214545+0000","last_became_active":"2024-06-06T04:27:19.304262+0000","last_became_peered":"2024-06-06T04:27:19.304262+0000","last_unstale":"2024-06-06T04:29:06.214545+0000","last_undegraded":"2024-06-06T04:29:06.214545+0000","last_fullsized":"2024-06-06T04:29:06.214545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:32:45.819520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.377096+0000","last_change":"2024-06-06T04:29:06.791555+0000","last_active":"2024-06-06T04:30:04.377096+0000","last_peered":"2024-06-06T04:30:04.377096+0000","last_clean":"2024-06-06T04:30:04.377096+0000","last_became_active":"2024-06-06T04:29:06.790925+0000","last_became_peered":"2024-06-06T04:29:06.790925+0000","last_unstale":"2024-06-06T04:30:04.377096+0000","last_undegraded":"2024-06-06T04:30:04.377096+0000","last_fullsized":"2024-06-06T04:30:04.377096+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:23:39.549238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.213893+0000","last_change":"2024-06-06T04:27:32.622092+0000","last_active":"2024-06-06T04:29:06.213893+0000","last_peered":"2024-06-06T04:29:06.213893+0000","last_clean":"2024-06-06T04:29:06.213893+0000","last_became_active":"2024-06-06T04:27:32.621618+0000","last_became_peered":"2024-06-06T04:27:32.621618+0000","last_unstale":"2024-06-06T04:29:06.213893+0000","last_undegraded":"2024-06-06T04:29:06.213893+0000","last_fullsized":"2024-06-06T04:29:06.213893+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:58:03.067292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.907896+0000","last_change":"2024-06-06T04:27:19.516803+0000","last_active":"2024-06-06T04:29:05.907896+0000","last_peered":"2024-06-06T04:29:05.907896+0000","last_clean":"2024-06-06T04:29:05.907896+0000","last_became_active":"2024-06-06T04:27:19.516458+0000","last_became_peered":"2024-06-06T04:27:19.516458+0000","last_unstale":"2024-06-06T04:29:05.907896+0000","last_undegraded":"2024-06-06T04:29:05.907896+0000","last_fullsized":"2024-06-06T04:29:05.907896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:34:53.593825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380334+0000","last_change":"2024-06-06T04:27:22.564306+0000","last_active":"2024-06-06T04:30:04.380334+0000","last_peered":"2024-06-06T04:30:04.380334+0000","last_clean":"2024-06-06T04:30:04.380334+0000","last_became_active":"2024-06-06T04:27:22.563922+0000","last_became_peered":"2024-06-06T04:27:22.563922+0000","last_unstale":"2024-06-06T04:30:04.380334+0000","last_undegraded":"2024-06-06T04:30:04.380334+0000","last_fullsized":"2024-06-06T04:30:04.380334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:42:23.189047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904530+0000","last_change":"2024-06-06T04:27:32.617014+0000","last_active":"2024-06-06T04:29:05.904530+0000","last_peered":"2024-06-06T04:29:05.904530+0000","last_clean":"2024-06-06T04:29:05.904530+0000","last_became_active":"2024-06-06T04:27:32.616639+0000","last_became_peered":"2024-06-06T04:27:32.616639+0000","last_unstale":"2024-06-06T04:29:05.904530+0000","last_undegraded":"2024-06-06T04:29:05.904530+0000","last_fullsized":"2024-06-06T04:29:05.904530+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:54:52.034711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905627+0000","last_change":"2024-06-06T04:27:19.305000+0000","last_active":"2024-06-06T04:29:05.905627+0000","last_peered":"2024-06-06T04:29:05.905627+0000","last_clean":"2024-06-06T04:29:05.905627+0000","last_became_active":"2024-06-06T04:27:19.304676+0000","last_became_peered":"2024-06-06T04:27:19.304676+0000","last_unstale":"2024-06-06T04:29:05.905627+0000","last_undegraded":"2024-06-06T04:29:05.905627+0000","last_fullsized":"2024-06-06T04:29:05.905627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:19:34.054455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.382734+0000","last_change":"2024-06-06T04:27:22.572680+0000","last_active":"2024-06-06T04:30:04.382734+0000","last_peered":"2024-06-06T04:30:04.382734+0000","last_clean":"2024-06-06T04:30:04.382734+0000","last_became_active":"2024-06-06T04:27:22.572348+0000","last_became_peered":"2024-06-06T04:27:22.572348+0000","last_unstale":"2024-06-06T04:30:04.382734+0000","last_undegraded":"2024-06-06T04:30:04.382734+0000","last_fullsized":"2024-06-06T04:30:04.382734+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:05:26.215771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904503+0000","last_change":"2024-06-06T04:27:32.611587+0000","last_active":"2024-06-06T04:29:05.904503+0000","last_peered":"2024-06-06T04:29:05.904503+0000","last_clean":"2024-06-06T04:29:05.904503+0000","last_became_active":"2024-06-06T04:27:32.611183+0000","last_became_peered":"2024-06-06T04:27:32.611183+0000","last_unstale":"2024-06-06T04:29:05.904503+0000","last_undegraded":"2024-06-06T04:29:05.904503+0000","last_fullsized":"2024-06-06T04:29:05.904503+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:36.773254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.371730+0000","last_change":"2024-06-06T04:29:06.403321+0000","last_active":"2024-06-06T04:30:04.371730+0000","last_peered":"2024-06-06T04:30:04.371730+0000","last_clean":"2024-06-06T04:30:04.371730+0000","last_became_active":"2024-06-06T04:29:06.402909+0000","last_became_peered":"2024-06-06T04:29:06.402909+0000","last_unstale":"2024-06-06T04:30:04.371730+0000","last_undegraded":"2024-06-06T04:30:04.371730+0000","last_fullsized":"2024-06-06T04:30:04.371730+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:29:38.963372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":225,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:21.969275+0000","last_change":"2024-06-06T04:27:19.516361+0000","last_active":"2024-06-06T04:31:21.969275+0000","last_peered":"2024-06-06T04:31:21.969275+0000","last_clean":"2024-06-06T04:31:21.969275+0000","last_became_active":"2024-06-06T04:27:19.515973+0000","last_became_peered":"2024-06-06T04:27:19.515973+0000","last_unstale":"2024-06-06T04:31:21.969275+0000","last_undegraded":"2024-06-06T04:31:21.969275+0000","last_fullsized":"2024-06-06T04:31:21.969275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:48:37.298819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905945+0000","last_change":"2024-06-06T04:27:32.630528+0000","last_active":"2024-06-06T04:29:05.905945+0000","last_peered":"2024-06-06T04:29:05.905945+0000","last_clean":"2024-06-06T04:29:05.905945+0000","last_became_active":"2024-06-06T04:27:32.630215+0000","last_became_peered":"2024-06-06T04:27:32.630215+0000","last_unstale":"2024-06-06T04:29:05.905945+0000","last_undegraded":"2024-06-06T04:29:05.905945+0000","last_fullsized":"2024-06-06T04:29:05.905945+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:34:29.980193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392719+0000","last_change":"2024-06-06T04:27:22.567466+0000","last_active":"2024-06-06T04:30:04.392719+0000","last_peered":"2024-06-06T04:30:04.392719+0000","last_clean":"2024-06-06T04:30:04.392719+0000","last_became_active":"2024-06-06T04:27:22.567145+0000","last_became_peered":"2024-06-06T04:27:22.567145+0000","last_unstale":"2024-06-06T04:30:04.392719+0000","last_undegraded":"2024-06-06T04:30:04.392719+0000","last_fullsized":"2024-06-06T04:30:04.392719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:56:56.203027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.401783+0000","last_change":"2024-06-06T04:27:19.302726+0000","last_active":"2024-06-06T04:29:06.401783+0000","last_peered":"2024-06-06T04:29:06.401783+0000","last_clean":"2024-06-06T04:29:06.401783+0000","last_became_active":"2024-06-06T04:27:19.302329+0000","last_became_peered":"2024-06-06T04:27:19.302329+0000","last_unstale":"2024-06-06T04:29:06.401783+0000","last_undegraded":"2024-06-06T04:29:06.401783+0000","last_fullsized":"2024-06-06T04:29:06.401783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:55:15.010632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":367,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905237+0000","last_change":"2024-06-06T04:25:53.854188+0000","last_active":"2024-06-06T04:29:05.905237+0000","last_peered":"2024-06-06T04:29:05.905237+0000","last_clean":"2024-06-06T04:29:05.905237+0000","last_became_active":"2024-06-06T04:25:53.853700+0000","last_became_peered":"2024-06-06T04:25:53.853700+0000","last_unstale":"2024-06-06T04:29:05.905237+0000","last_undegraded":"2024-06-06T04:29:05.905237+0000","last_fullsized":"2024-06-06T04:29:05.905237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:22:28.847333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:22:28.847333+0000","last_clean_scrub_stamp":"2024-06-06T04:22:28.847333+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:59:10.246373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790675+0000","last_change":"2024-06-06T04:27:32.623300+0000","last_active":"2024-06-06T04:29:06.790675+0000","last_peered":"2024-06-06T04:29:06.790675+0000","last_clean":"2024-06-06T04:29:06.790675+0000","last_became_active":"2024-06-06T04:27:32.622811+0000","last_became_peered":"2024-06-06T04:27:32.622811+0000","last_unstale":"2024-06-06T04:29:06.790675+0000","last_undegraded":"2024-06-06T04:29:06.790675+0000","last_fullsized":"2024-06-06T04:29:06.790675+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:28.383079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.367134+0000","last_change":"2024-06-06T04:27:22.566617+0000","last_active":"2024-06-06T04:30:04.367134+0000","last_peered":"2024-06-06T04:30:04.367134+0000","last_clean":"2024-06-06T04:30:04.367134+0000","last_became_active":"2024-06-06T04:27:22.566256+0000","last_became_peered":"2024-06-06T04:27:22.566256+0000","last_unstale":"2024-06-06T04:30:04.367134+0000","last_undegraded":"2024-06-06T04:30:04.367134+0000","last_fullsized":"2024-06-06T04:30:04.367134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:50:41.318475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:18.419034+0000","last_change":"2024-06-06T04:27:19.305941+0000","last_active":"2024-06-06T04:29:18.419034+0000","last_peered":"2024-06-06T04:29:18.419034+0000","last_clean":"2024-06-06T04:29:18.419034+0000","last_became_active":"2024-06-06T04:27:19.305037+0000","last_became_peered":"2024-06-06T04:27:19.305037+0000","last_unstale":"2024-06-06T04:29:18.419034+0000","last_undegraded":"2024-06-06T04:29:18.419034+0000","last_fullsized":"2024-06-06T04:29:18.419034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:57:54.084427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908935+0000","last_change":"2024-06-06T04:27:32.621182+0000","last_active":"2024-06-06T04:29:05.908935+0000","last_peered":"2024-06-06T04:29:05.908935+0000","last_clean":"2024-06-06T04:29:05.908935+0000","last_became_active":"2024-06-06T04:27:32.620710+0000","last_became_peered":"2024-06-06T04:27:32.620710+0000","last_unstale":"2024-06-06T04:29:05.908935+0000","last_undegraded":"2024-06-06T04:29:05.908935+0000","last_fullsized":"2024-06-06T04:29:05.908935+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:13:05.512309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214246+0000","last_change":"2024-06-06T04:27:19.517033+0000","last_active":"2024-06-06T04:29:06.214246+0000","last_peered":"2024-06-06T04:29:06.214246+0000","last_clean":"2024-06-06T04:29:06.214246+0000","last_became_active":"2024-06-06T04:27:19.516278+0000","last_became_peered":"2024-06-06T04:27:19.516278+0000","last_unstale":"2024-06-06T04:29:06.214246+0000","last_undegraded":"2024-06-06T04:29:06.214246+0000","last_fullsized":"2024-06-06T04:29:06.214246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:11:20.336818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.368634+0000","last_change":"2024-06-06T04:27:22.566261+0000","last_active":"2024-06-06T04:30:04.368634+0000","last_peered":"2024-06-06T04:30:04.368634+0000","last_clean":"2024-06-06T04:30:04.368634+0000","last_became_active":"2024-06-06T04:27:22.565875+0000","last_became_peered":"2024-06-06T04:27:22.565875+0000","last_unstale":"2024-06-06T04:30:04.368634+0000","last_undegraded":"2024-06-06T04:30:04.368634+0000","last_fullsized":"2024-06-06T04:30:04.368634+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:49:55.923950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560350+0000","last_change":"2024-06-06T04:27:32.619854+0000","last_active":"2024-06-06T04:29:06.560350+0000","last_peered":"2024-06-06T04:29:06.560350+0000","last_clean":"2024-06-06T04:29:06.560350+0000","last_became_active":"2024-06-06T04:27:32.618765+0000","last_became_peered":"2024-06-06T04:27:32.618765+0000","last_unstale":"2024-06-06T04:29:06.560350+0000","last_undegraded":"2024-06-06T04:29:06.560350+0000","last_fullsized":"2024-06-06T04:29:06.560350+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:47:05.283751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'14","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:31:17.603468+0000","last_change":"2024-06-06T04:27:19.303057+0000","last_active":"2024-06-06T04:31:17.603468+0000","last_peered":"2024-06-06T04:31:17.603468+0000","last_clean":"2024-06-06T04:31:17.603468+0000","last_became_active":"2024-06-06T04:27:19.302716+0000","last_became_peered":"2024-06-06T04:27:19.302716+0000","last_unstale":"2024-06-06T04:31:17.603468+0000","last_undegraded":"2024-06-06T04:31:17.603468+0000","last_fullsized":"2024-06-06T04:31:17.603468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:11:35.273663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381463+0000","last_change":"2024-06-06T04:27:22.575274+0000","last_active":"2024-06-06T04:30:04.381463+0000","last_peered":"2024-06-06T04:30:04.381463+0000","last_clean":"2024-06-06T04:30:04.381463+0000","last_became_active":"2024-06-06T04:27:22.574979+0000","last_became_peered":"2024-06-06T04:27:22.574979+0000","last_unstale":"2024-06-06T04:30:04.381463+0000","last_undegraded":"2024-06-06T04:30:04.381463+0000","last_fullsized":"2024-06-06T04:30:04.381463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:35:30.718398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.905413+0000","last_change":"2024-06-06T04:27:32.619939+0000","last_active":"2024-06-06T04:29:05.905413+0000","last_peered":"2024-06-06T04:29:05.905413+0000","last_clean":"2024-06-06T04:29:05.905413+0000","last_became_active":"2024-06-06T04:27:32.619499+0000","last_became_peered":"2024-06-06T04:27:32.619499+0000","last_unstale":"2024-06-06T04:29:05.905413+0000","last_undegraded":"2024-06-06T04:29:05.905413+0000","last_fullsized":"2024-06-06T04:29:05.905413+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T04:41:13.933546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402340+0000","last_change":"2024-06-06T04:27:19.397552+0000","last_active":"2024-06-06T04:29:06.402340+0000","last_peered":"2024-06-06T04:29:06.402340+0000","last_clean":"2024-06-06T04:29:06.402340+0000","last_became_active":"2024-06-06T04:27:19.397186+0000","last_became_peered":"2024-06-06T04:27:19.397186+0000","last_unstale":"2024-06-06T04:29:06.402340+0000","last_undegraded":"2024-06-06T04:29:06.402340+0000","last_fullsized":"2024-06-06T04:29:06.402340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:24:13.403860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.384537+0000","last_change":"2024-06-06T04:28:05.728212+0000","last_active":"2024-06-06T04:30:04.384537+0000","last_peered":"2024-06-06T04:30:04.384537+0000","last_clean":"2024-06-06T04:30:04.384537+0000","last_became_active":"2024-06-06T04:28:05.727748+0000","last_became_peered":"2024-06-06T04:28:05.727748+0000","last_unstale":"2024-06-06T04:30:04.384537+0000","last_undegraded":"2024-06-06T04:30:04.384537+0000","last_fullsized":"2024-06-06T04:30:04.384537+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:50.583716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402083+0000","last_change":"2024-06-06T04:28:05.896480+0000","last_active":"2024-06-06T04:29:06.402083+0000","last_peered":"2024-06-06T04:29:06.402083+0000","last_clean":"2024-06-06T04:29:06.402083+0000","last_became_active":"2024-06-06T04:28:05.895967+0000","last_became_peered":"2024-06-06T04:28:05.895967+0000","last_unstale":"2024-06-06T04:29:06.402083+0000","last_undegraded":"2024-06-06T04:29:06.402083+0000","last_fullsized":"2024-06-06T04:29:06.402083+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:26:19.846884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.907843+0000","last_change":"2024-06-06T04:27:19.516515+0000","last_active":"2024-06-06T04:29:06.907843+0000","last_peered":"2024-06-06T04:29:06.907843+0000","last_clean":"2024-06-06T04:29:06.907843+0000","last_became_active":"2024-06-06T04:27:19.516087+0000","last_became_peered":"2024-06-06T04:27:19.516087+0000","last_unstale":"2024-06-06T04:29:06.907843+0000","last_undegraded":"2024-06-06T04:29:06.907843+0000","last_fullsized":"2024-06-06T04:29:06.907843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:27:58.440226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":40,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.342173+0000","last_change":"2024-06-06T04:27:22.562367+0000","last_active":"2024-06-06T04:30:04.342173+0000","last_peered":"2024-06-06T04:30:04.342173+0000","last_clean":"2024-06-06T04:30:04.342173+0000","last_became_active":"2024-06-06T04:27:22.561947+0000","last_became_peered":"2024-06-06T04:27:22.561947+0000","last_unstale":"2024-06-06T04:30:04.342173+0000","last_undegraded":"2024-06-06T04:30:04.342173+0000","last_fullsized":"2024-06-06T04:30:04.342173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:28:06.200748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.908183+0000","last_change":"2024-06-06T04:27:32.631577+0000","last_active":"2024-06-06T04:29:05.908183+0000","last_peered":"2024-06-06T04:29:05.908183+0000","last_clean":"2024-06-06T04:29:05.908183+0000","last_became_active":"2024-06-06T04:27:32.629555+0000","last_became_peered":"2024-06-06T04:27:32.629555+0000","last_unstale":"2024-06-06T04:29:05.908183+0000","last_undegraded":"2024-06-06T04:29:05.908183+0000","last_fullsized":"2024-06-06T04:29:05.908183+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:52:56.197281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.560477+0000","last_change":"2024-06-06T04:27:19.298880+0000","last_active":"2024-06-06T04:29:06.560477+0000","last_peered":"2024-06-06T04:29:06.560477+0000","last_clean":"2024-06-06T04:29:06.560477+0000","last_became_active":"2024-06-06T04:27:19.298551+0000","last_became_peered":"2024-06-06T04:27:19.298551+0000","last_unstale":"2024-06-06T04:29:06.560477+0000","last_undegraded":"2024-06-06T04:29:06.560477+0000","last_fullsized":"2024-06-06T04:29:06.560477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:11:32.145401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.383001+0000","last_change":"2024-06-06T04:27:22.575065+0000","last_active":"2024-06-06T04:30:04.383001+0000","last_peered":"2024-06-06T04:30:04.383001+0000","last_clean":"2024-06-06T04:30:04.383001+0000","last_became_active":"2024-06-06T04:27:22.574469+0000","last_became_peered":"2024-06-06T04:27:22.574469+0000","last_unstale":"2024-06-06T04:30:04.383001+0000","last_undegraded":"2024-06-06T04:30:04.383001+0000","last_fullsized":"2024-06-06T04:30:04.383001+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:29:44.943833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402041+0000","last_change":"2024-06-06T04:27:32.626678+0000","last_active":"2024-06-06T04:29:06.402041+0000","last_peered":"2024-06-06T04:29:06.402041+0000","last_clean":"2024-06-06T04:29:06.402041+0000","last_became_active":"2024-06-06T04:27:32.626167+0000","last_became_peered":"2024-06-06T04:27:32.626167+0000","last_unstale":"2024-06-06T04:29:06.402041+0000","last_undegraded":"2024-06-06T04:29:06.402041+0000","last_fullsized":"2024-06-06T04:29:06.402041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T06:56:26.363195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:32.586256+0000","last_change":"2024-06-06T04:27:19.306466+0000","last_active":"2024-06-06T04:29:32.586256+0000","last_peered":"2024-06-06T04:29:32.586256+0000","last_clean":"2024-06-06T04:29:32.586256+0000","last_became_active":"2024-06-06T04:27:19.303821+0000","last_became_peered":"2024-06-06T04:27:19.303821+0000","last_unstale":"2024-06-06T04:29:32.586256+0000","last_undegraded":"2024-06-06T04:29:32.586256+0000","last_fullsized":"2024-06-06T04:29:32.586256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:23:53.593998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.386917+0000","last_change":"2024-06-06T04:27:22.562662+0000","last_active":"2024-06-06T04:30:04.386917+0000","last_peered":"2024-06-06T04:30:04.386917+0000","last_clean":"2024-06-06T04:30:04.386917+0000","last_became_active":"2024-06-06T04:27:22.562318+0000","last_became_peered":"2024-06-06T04:27:22.562318+0000","last_unstale":"2024-06-06T04:30:04.386917+0000","last_undegraded":"2024-06-06T04:30:04.386917+0000","last_fullsized":"2024-06-06T04:30:04.386917+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:31:26.639275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.402218+0000","last_change":"2024-06-06T04:27:32.619970+0000","last_active":"2024-06-06T04:29:06.402218+0000","last_peered":"2024-06-06T04:29:06.402218+0000","last_clean":"2024-06-06T04:29:06.402218+0000","last_became_active":"2024-06-06T04:27:32.619464+0000","last_became_peered":"2024-06-06T04:27:32.619464+0000","last_unstale":"2024-06-06T04:29:06.402218+0000","last_undegraded":"2024-06-06T04:29:06.402218+0000","last_fullsized":"2024-06-06T04:29:06.402218+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:11:44.844018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.214056+0000","last_change":"2024-06-06T04:27:19.308172+0000","last_active":"2024-06-06T04:29:06.214056+0000","last_peered":"2024-06-06T04:29:06.214056+0000","last_clean":"2024-06-06T04:29:06.214056+0000","last_became_active":"2024-06-06T04:27:19.307780+0000","last_became_peered":"2024-06-06T04:27:19.307780+0000","last_unstale":"2024-06-06T04:29:06.214056+0000","last_undegraded":"2024-06-06T04:29:06.214056+0000","last_fullsized":"2024-06-06T04:29:06.214056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:39:11.653649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394812+0000","last_change":"2024-06-06T04:27:22.568881+0000","last_active":"2024-06-06T04:30:04.394812+0000","last_peered":"2024-06-06T04:30:04.394812+0000","last_clean":"2024-06-06T04:30:04.394812+0000","last_became_active":"2024-06-06T04:27:22.568489+0000","last_became_peered":"2024-06-06T04:27:22.568489+0000","last_unstale":"2024-06-06T04:30:04.394812+0000","last_undegraded":"2024-06-06T04:30:04.394812+0000","last_fullsized":"2024-06-06T04:30:04.394812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:00:22.339313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.907971+0000","last_change":"2024-06-06T04:27:32.622357+0000","last_active":"2024-06-06T04:29:06.907971+0000","last_peered":"2024-06-06T04:29:06.907971+0000","last_clean":"2024-06-06T04:29:06.907971+0000","last_became_active":"2024-06-06T04:27:32.621680+0000","last_became_peered":"2024-06-06T04:27:32.621680+0000","last_unstale":"2024-06-06T04:29:06.907971+0000","last_undegraded":"2024-06-06T04:29:06.907971+0000","last_fullsized":"2024-06-06T04:29:06.907971+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:00.677721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.906141+0000","last_change":"2024-06-06T04:27:19.516890+0000","last_active":"2024-06-06T04:29:05.906141+0000","last_peered":"2024-06-06T04:29:05.906141+0000","last_clean":"2024-06-06T04:29:05.906141+0000","last_became_active":"2024-06-06T04:27:19.516490+0000","last_became_peered":"2024-06-06T04:27:19.516490+0000","last_unstale":"2024-06-06T04:29:05.906141+0000","last_undegraded":"2024-06-06T04:29:05.906141+0000","last_fullsized":"2024-06-06T04:29:05.906141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:11:59.336250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.395341+0000","last_change":"2024-06-06T04:27:22.571003+0000","last_active":"2024-06-06T04:30:04.395341+0000","last_peered":"2024-06-06T04:30:04.395341+0000","last_clean":"2024-06-06T04:30:04.395341+0000","last_became_active":"2024-06-06T04:27:22.570442+0000","last_became_peered":"2024-06-06T04:27:22.570442+0000","last_unstale":"2024-06-06T04:30:04.395341+0000","last_undegraded":"2024-06-06T04:30:04.395341+0000","last_fullsized":"2024-06-06T04:30:04.395341+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:28:26.870429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.164730+0000","last_change":"2024-06-06T04:27:32.624424+0000","last_active":"2024-06-06T04:29:06.164730+0000","last_peered":"2024-06-06T04:29:06.164730+0000","last_clean":"2024-06-06T04:29:06.164730+0000","last_became_active":"2024-06-06T04:27:32.624026+0000","last_became_peered":"2024-06-06T04:27:32.624026+0000","last_unstale":"2024-06-06T04:29:06.164730+0000","last_undegraded":"2024-06-06T04:29:06.164730+0000","last_fullsized":"2024-06-06T04:29:06.164730+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:30:08.814284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.790341+0000","last_change":"2024-06-06T04:27:19.301382+0000","last_active":"2024-06-06T04:29:06.790341+0000","last_peered":"2024-06-06T04:29:06.790341+0000","last_clean":"2024-06-06T04:29:06.790341+0000","last_became_active":"2024-06-06T04:27:19.300882+0000","last_became_peered":"2024-06-06T04:27:19.300882+0000","last_unstale":"2024-06-06T04:29:06.790341+0000","last_undegraded":"2024-06-06T04:29:06.790341+0000","last_fullsized":"2024-06-06T04:29:06.790341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:22:50.514103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.392499+0000","last_change":"2024-06-06T04:27:22.570661+0000","last_active":"2024-06-06T04:30:04.392499+0000","last_peered":"2024-06-06T04:30:04.392499+0000","last_clean":"2024-06-06T04:30:04.392499+0000","last_became_active":"2024-06-06T04:27:22.570287+0000","last_became_peered":"2024-06-06T04:27:22.570287+0000","last_unstale":"2024-06-06T04:30:04.392499+0000","last_undegraded":"2024-06-06T04:30:04.392499+0000","last_fullsized":"2024-06-06T04:30:04.392499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:01:48.790492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:06.789984+0000","last_change":"2024-06-06T04:27:32.618385+0000","last_active":"2024-06-06T04:29:06.789984+0000","last_peered":"2024-06-06T04:29:06.789984+0000","last_clean":"2024-06-06T04:29:06.789984+0000","last_became_active":"2024-06-06T04:27:32.617956+0000","last_became_peered":"2024-06-06T04:27:32.617956+0000","last_unstale":"2024-06-06T04:29:06.789984+0000","last_undegraded":"2024-06-06T04:29:06.789984+0000","last_fullsized":"2024-06-06T04:29:06.789984+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:31.590511+0000","last_clean_scrub_stamp":"2024-06-06T04:27:31.590511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:13:26.308657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:29:05.904541+0000","last_change":"2024-06-06T04:28:05.728633+0000","last_active":"2024-06-06T04:29:05.904541+0000","last_peered":"2024-06-06T04:29:05.904541+0000","last_clean":"2024-06-06T04:29:05.904541+0000","last_became_active":"2024-06-06T04:28:05.722427+0000","last_became_peered":"2024-06-06T04:28:05.722427+0000","last_unstale":"2024-06-06T04:29:05.904541+0000","last_undegraded":"2024-06-06T04:29:05.904541+0000","last_fullsized":"2024-06-06T04:29:05.904541+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:18.288323+0000","last_clean_scrub_stamp":"2024-06-06T04:27:18.288323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:13:22.061798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.388201+0000","last_change":"2024-06-06T04:27:22.574950+0000","last_active":"2024-06-06T04:30:04.388201+0000","last_peered":"2024-06-06T04:30:04.388201+0000","last_clean":"2024-06-06T04:30:04.388201+0000","last_became_active":"2024-06-06T04:27:22.574604+0000","last_became_peered":"2024-06-06T04:27:22.574604+0000","last_unstale":"2024-06-06T04:30:04.388201+0000","last_undegraded":"2024-06-06T04:30:04.388201+0000","last_fullsized":"2024-06-06T04:30:04.388201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:17:29.395137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.380448+0000","last_change":"2024-06-06T04:27:22.568554+0000","last_active":"2024-06-06T04:30:04.380448+0000","last_peered":"2024-06-06T04:30:04.380448+0000","last_clean":"2024-06-06T04:30:04.380448+0000","last_became_active":"2024-06-06T04:27:22.568196+0000","last_became_peered":"2024-06-06T04:27:22.568196+0000","last_unstale":"2024-06-06T04:30:04.380448+0000","last_undegraded":"2024-06-06T04:30:04.380448+0000","last_fullsized":"2024-06-06T04:30:04.380448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:29:11.564067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.372439+0000","last_change":"2024-06-06T04:27:22.565472+0000","last_active":"2024-06-06T04:30:04.372439+0000","last_peered":"2024-06-06T04:30:04.372439+0000","last_clean":"2024-06-06T04:30:04.372439+0000","last_became_active":"2024-06-06T04:27:22.565116+0000","last_became_peered":"2024-06-06T04:27:22.565116+0000","last_unstale":"2024-06-06T04:30:04.372439+0000","last_undegraded":"2024-06-06T04:30:04.372439+0000","last_fullsized":"2024-06-06T04:30:04.372439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:03:58.232707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.377900+0000","last_change":"2024-06-06T04:27:22.560210+0000","last_active":"2024-06-06T04:30:04.377900+0000","last_peered":"2024-06-06T04:30:04.377900+0000","last_clean":"2024-06-06T04:30:04.377900+0000","last_became_active":"2024-06-06T04:27:22.559853+0000","last_became_peered":"2024-06-06T04:27:22.559853+0000","last_unstale":"2024-06-06T04:30:04.377900+0000","last_undegraded":"2024-06-06T04:30:04.377900+0000","last_fullsized":"2024-06-06T04:30:04.377900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:56:18.650427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385809+0000","last_change":"2024-06-06T04:27:22.565071+0000","last_active":"2024-06-06T04:30:04.385809+0000","last_peered":"2024-06-06T04:30:04.385809+0000","last_clean":"2024-06-06T04:30:04.385809+0000","last_became_active":"2024-06-06T04:27:22.561411+0000","last_became_peered":"2024-06-06T04:27:22.561411+0000","last_unstale":"2024-06-06T04:30:04.385809+0000","last_undegraded":"2024-06-06T04:30:04.385809+0000","last_fullsized":"2024-06-06T04:30:04.385809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:08:41.948481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.382771+0000","last_change":"2024-06-06T04:27:22.563280+0000","last_active":"2024-06-06T04:30:04.382771+0000","last_peered":"2024-06-06T04:30:04.382771+0000","last_clean":"2024-06-06T04:30:04.382771+0000","last_became_active":"2024-06-06T04:27:22.563049+0000","last_became_peered":"2024-06-06T04:27:22.563049+0000","last_unstale":"2024-06-06T04:30:04.382771+0000","last_undegraded":"2024-06-06T04:30:04.382771+0000","last_fullsized":"2024-06-06T04:30:04.382771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:41:48.287969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.377210+0000","last_change":"2024-06-06T04:27:22.567691+0000","last_active":"2024-06-06T04:30:04.377210+0000","last_peered":"2024-06-06T04:30:04.377210+0000","last_clean":"2024-06-06T04:30:04.377210+0000","last_became_active":"2024-06-06T04:27:22.567331+0000","last_became_peered":"2024-06-06T04:27:22.567331+0000","last_unstale":"2024-06-06T04:30:04.377210+0000","last_undegraded":"2024-06-06T04:30:04.377210+0000","last_fullsized":"2024-06-06T04:30:04.377210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:03.886084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.393172+0000","last_change":"2024-06-06T04:27:22.563853+0000","last_active":"2024-06-06T04:30:04.393172+0000","last_peered":"2024-06-06T04:30:04.393172+0000","last_clean":"2024-06-06T04:30:04.393172+0000","last_became_active":"2024-06-06T04:27:22.563436+0000","last_became_peered":"2024-06-06T04:27:22.563436+0000","last_unstale":"2024-06-06T04:30:04.393172+0000","last_undegraded":"2024-06-06T04:30:04.393172+0000","last_fullsized":"2024-06-06T04:30:04.393172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:20:26.201308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.387060+0000","last_change":"2024-06-06T04:27:22.568189+0000","last_active":"2024-06-06T04:30:04.387060+0000","last_peered":"2024-06-06T04:30:04.387060+0000","last_clean":"2024-06-06T04:30:04.387060+0000","last_became_active":"2024-06-06T04:27:22.567815+0000","last_became_peered":"2024-06-06T04:27:22.567815+0000","last_unstale":"2024-06-06T04:30:04.387060+0000","last_undegraded":"2024-06-06T04:30:04.387060+0000","last_fullsized":"2024-06-06T04:30:04.387060+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:13:39.259012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.359906+0000","last_change":"2024-06-06T04:27:22.569698+0000","last_active":"2024-06-06T04:30:04.359906+0000","last_peered":"2024-06-06T04:30:04.359906+0000","last_clean":"2024-06-06T04:30:04.359906+0000","last_became_active":"2024-06-06T04:27:22.569347+0000","last_became_peered":"2024-06-06T04:27:22.569347+0000","last_unstale":"2024-06-06T04:30:04.359906+0000","last_undegraded":"2024-06-06T04:30:04.359906+0000","last_fullsized":"2024-06-06T04:30:04.359906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:06:34.722798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.381456+0000","last_change":"2024-06-06T04:27:22.561149+0000","last_active":"2024-06-06T04:30:04.381456+0000","last_peered":"2024-06-06T04:30:04.381456+0000","last_clean":"2024-06-06T04:30:04.381456+0000","last_became_active":"2024-06-06T04:27:22.560793+0000","last_became_peered":"2024-06-06T04:27:22.560793+0000","last_unstale":"2024-06-06T04:30:04.381456+0000","last_undegraded":"2024-06-06T04:30:04.381456+0000","last_fullsized":"2024-06-06T04:30:04.381456+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:00:41.563443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.394791+0000","last_change":"2024-06-06T04:27:22.569331+0000","last_active":"2024-06-06T04:30:04.394791+0000","last_peered":"2024-06-06T04:30:04.394791+0000","last_clean":"2024-06-06T04:30:04.394791+0000","last_became_active":"2024-06-06T04:27:22.569021+0000","last_became_peered":"2024-06-06T04:27:22.569021+0000","last_unstale":"2024-06-06T04:30:04.394791+0000","last_undegraded":"2024-06-06T04:30:04.394791+0000","last_fullsized":"2024-06-06T04:30:04.394791+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:52:37.064317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.391532+0000","last_change":"2024-06-06T04:27:22.574558+0000","last_active":"2024-06-06T04:30:04.391532+0000","last_peered":"2024-06-06T04:30:04.391532+0000","last_clean":"2024-06-06T04:30:04.391532+0000","last_became_active":"2024-06-06T04:27:22.574215+0000","last_became_peered":"2024-06-06T04:27:22.574215+0000","last_unstale":"2024-06-06T04:30:04.391532+0000","last_undegraded":"2024-06-06T04:30:04.391532+0000","last_fullsized":"2024-06-06T04:30:04.391532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T05:40:52.496322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T04:30:04.385148+0000","last_change":"2024-06-06T04:27:22.565320+0000","last_active":"2024-06-06T04:30:04.385148+0000","last_peered":"2024-06-06T04:30:04.385148+0000","last_clean":"2024-06-06T04:30:04.385148+0000","last_became_active":"2024-06-06T04:27:22.564786+0000","last_became_peered":"2024-06-06T04:27:22.564786+0000","last_unstale":"2024-06-06T04:30:04.385148+0000","last_undegraded":"2024-06-06T04:30:04.385148+0000","last_fullsized":"2024-06-06T04:30:04.385148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T04:27:21.537062+0000","last_clean_scrub_stamp":"2024-06-06T04:27:21.537062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:19:42.057666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249851,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78096,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1070,"num_read_kb":619,"num_write":191,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":234288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351041,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":572909,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612678,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28244,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716908,"statfs":{"total":95995035648,"available":95966113792,"internally_reserved":0,"allocated":1257472,"data_stored":1233781,"data_compressed":68587,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776203,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":577241,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037840,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573065,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299477,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":1170488,"data_compressed":68587,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463003,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":568321,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":640812,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":569709,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1200128,"data_stored":643044,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149801,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":568515,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888164,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":573994,"data_compressed":32426,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724639,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1175552,"data_stored":1165053,"data_compressed":68587,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34523,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34523,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36161,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":72581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":74813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T04:31:24.991 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:31:25.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:24 smithi045 ceph-mon[32299]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:31:25.278 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:31:25.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:24 smithi086 ceph-mon[36053]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:31:25.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:24 smithi154 ceph-mon[36428]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:31:26.219 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:25 smithi045 ceph-mon[32299]: pgmap v430: 129 pgs: 129 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-06-06T04:31:26.219 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:25 smithi045 ceph-mon[32299]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:31:26.219 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:25 smithi045 ceph-mon[32299]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:31:26.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:25 smithi086 ceph-mon[36053]: pgmap v430: 129 pgs: 129 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-06-06T04:31:26.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:25 smithi086 ceph-mon[36053]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:31:26.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:25 smithi086 ceph-mon[36053]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:31:26.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:25 smithi154 ceph-mon[36428]: pgmap v430: 129 pgs: 129 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-06-06T04:31:26.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:25 smithi154 ceph-mon[36428]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:31:26.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:25 smithi154 ceph-mon[36428]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T04:31:27.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.071+0000 7f9ce751c700 1 -- 172.21.15.45:0/988970616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce0107840 msgr2=0x7f9ce010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:27.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.071+0000 7f9ce751c700 1 --2- 172.21.15.45:0/988970616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce0107840 0x7f9ce010f3c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9cd80099c0 tx=0x7f9cd8009cd0 comp rx=0 tx=0).stop 2024-06-06T04:31:27.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.071+0000 7f9ce751c700 1 -- 172.21.15.45:0/988970616 shutdown_connections 2024-06-06T04:31:27.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.071+0000 7f9ce751c700 1 --2- 172.21.15.45:0/988970616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce0107840 0x7f9ce010f3c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:27.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.071+0000 7f9ce751c700 1 --2- 172.21.15.45:0/988970616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce0102740 0x7f9ce0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:27.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.071+0000 7f9ce751c700 1 --2- 172.21.15.45:0/988970616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ce01030f0 0x7f9ce0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:27.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.071+0000 7f9ce751c700 1 -- 172.21.15.45:0/988970616 >> 172.21.15.45:0/988970616 conn(0x7f9ce00795f0 msgr2=0x7f9ce0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:27.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.071+0000 7f9ce751c700 1 -- 172.21.15.45:0/988970616 shutdown_connections 2024-06-06T04:31:27.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.071+0000 7f9ce751c700 1 -- 172.21.15.45:0/988970616 wait complete. 2024-06-06T04:31:27.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.072+0000 7f9ce751c700 1 Processor -- start 2024-06-06T04:31:27.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.072+0000 7f9ce751c700 1 -- start start 2024-06-06T04:31:27.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.072+0000 7f9ce751c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ce0102740 0x7f9ce019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:27.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.073+0000 7f9ce751c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce01030f0 0x7f9ce019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:27.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.073+0000 7f9ce751c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce0107840 0x7f9ce0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:27.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.073+0000 7f9ce751c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce0112680 con 0x7f9ce0107840 2024-06-06T04:31:27.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.073+0000 7f9ce751c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce0112500 con 0x7f9ce01030f0 2024-06-06T04:31:27.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.073+0000 7f9ce751c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce0112800 con 0x7f9ce0102740 2024-06-06T04:31:27.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.073+0000 7f9ce5ab9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce0107840 0x7f9ce0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:27.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.073+0000 7f9ce5ab9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce0107840 0x7f9ce0198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41220/0 (socket says 172.21.15.45:41220) 2024-06-06T04:31:27.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.073+0000 7f9ce5ab9700 1 -- 172.21.15.45:0/3601941072 learned_addr learned my addr 172.21.15.45:0/3601941072 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:27.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.073+0000 7f9ce52b8700 1 --2- 172.21.15.45:0/3601941072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ce0102740 0x7f9ce019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:27.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.073+0000 7f9ce4ab7700 1 --2- 172.21.15.45:0/3601941072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce01030f0 0x7f9ce019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:27.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.073+0000 7f9ce5ab9700 1 -- 172.21.15.45:0/3601941072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ce0102740 msgr2=0x7f9ce019e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:27.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.073+0000 7f9ce5ab9700 1 --2- 172.21.15.45:0/3601941072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ce0102740 0x7f9ce019e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:27.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.073+0000 7f9ce5ab9700 1 -- 172.21.15.45:0/3601941072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce01030f0 msgr2=0x7f9ce019ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:27.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.073+0000 7f9ce5ab9700 1 --2- 172.21.15.45:0/3601941072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce01030f0 0x7f9ce019ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:27.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.073+0000 7f9ce5ab9700 1 -- 172.21.15.45:0/3601941072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce01990a0 con 0x7f9ce0107840 2024-06-06T04:31:27.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.073+0000 7f9ce5ab9700 1 --2- 172.21.15.45:0/3601941072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce0107840 0x7f9ce0198940 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9cd80059d0 tx=0x7f9cd80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:27.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.074+0000 7f9cde7fc700 1 -- 172.21.15.45:0/3601941072 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8027070 con 0x7f9ce0107840 2024-06-06T04:31:27.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.074+0000 7f9cde7fc700 1 -- 172.21.15.45:0/3601941072 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cd8004090 con 0x7f9ce0107840 2024-06-06T04:31:27.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.074+0000 7f9ce751c700 1 -- 172.21.15.45:0/3601941072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce0199330 con 0x7f9ce0107840 2024-06-06T04:31:27.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.074+0000 7f9cde7fc700 1 -- 172.21.15.45:0/3601941072 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8022d40 con 0x7f9ce0107840 2024-06-06T04:31:27.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.074+0000 7f9ce751c700 1 -- 172.21.15.45:0/3601941072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce01a5600 con 0x7f9ce0107840 2024-06-06T04:31:27.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.075+0000 7f9ce751c700 1 -- 172.21.15.45:0/3601941072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce0069180 con 0x7f9ce0107840 2024-06-06T04:31:27.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.076+0000 7f9cde7fc700 1 -- 172.21.15.45:0/3601941072 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f9cd802a070 con 0x7f9ce0107840 2024-06-06T04:31:27.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.077+0000 7f9cde7fc700 1 --2- 172.21.15.45:0/3601941072 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9cc80a2cb0 0x7f9cc80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:27.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.077+0000 7f9ce52b8700 1 --2- 172.21.15.45:0/3601941072 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9cc80a2cb0 0x7f9cc80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:27.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.077+0000 7f9cde7fc700 1 -- 172.21.15.45:0/3601941072 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f9cd80d0880 con 0x7f9ce0107840 2024-06-06T04:31:27.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.078+0000 7f9ce52b8700 1 --2- 172.21.15.45:0/3601941072 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9cc80a2cb0 0x7f9cc80a5170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00060b0 tx=0x7f9cd000c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:27.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.080+0000 7f9cde7fc700 1 -- 172.21.15.45:0/3601941072 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9cd8031070 con 0x7f9ce0107840 2024-06-06T04:31:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.335+0000 7f9ce751c700 1 -- 172.21.15.45:0/3601941072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ce002dca0 con 0x7f9ce0107840 2024-06-06T04:31:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.336+0000 7f9cde7fc700 1 -- 172.21.15.45:0/3601941072 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11858 (secure 0 0 0) 0x7f9cd8013070 con 0x7f9ce0107840 2024-06-06T04:31:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:31:27.337 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T04:27:24.462897+0000","modified":"2024-06-06T04:31:17.956589+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24451,24463,24473]},"id":1}]} 2024-06-06T04:31:27.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.338+0000 7f9ce751c700 1 -- 172.21.15.45:0/3601941072 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9cc80a2cb0 msgr2=0x7f9cc80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.338+0000 7f9ce751c700 1 --2- 172.21.15.45:0/3601941072 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9cc80a2cb0 0x7f9cc80a5170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00060b0 tx=0x7f9cd000c040 comp rx=0 tx=0).stop 2024-06-06T04:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.338+0000 7f9ce751c700 1 -- 172.21.15.45:0/3601941072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce0107840 msgr2=0x7f9ce0198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.338+0000 7f9ce751c700 1 --2- 172.21.15.45:0/3601941072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce0107840 0x7f9ce0198940 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9cd80059d0 tx=0x7f9cd80047b0 comp rx=0 tx=0).stop 2024-06-06T04:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.338+0000 7f9ce751c700 1 -- 172.21.15.45:0/3601941072 shutdown_connections 2024-06-06T04:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.338+0000 7f9ce751c700 1 --2- 172.21.15.45:0/3601941072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce0107840 0x7f9ce0198940 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.338+0000 7f9ce751c700 1 --2- 172.21.15.45:0/3601941072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ce0102740 0x7f9ce019e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.338+0000 7f9ce751c700 1 --2- 172.21.15.45:0/3601941072 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9cc80a2cb0 0x7f9cc80a5170 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.338+0000 7f9ce751c700 1 --2- 172.21.15.45:0/3601941072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce01030f0 0x7f9ce019ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.338+0000 7f9ce751c700 1 -- 172.21.15.45:0/3601941072 >> 172.21.15.45:0/3601941072 conn(0x7f9ce00795f0 msgr2=0x7f9ce0100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.338+0000 7f9ce751c700 1 -- 172.21.15.45:0/3601941072 shutdown_connections 2024-06-06T04:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:27.338+0000 7f9ce751c700 1 -- 172.21.15.45:0/3601941072 wait complete. 2024-06-06T04:31:27.340 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-06-06T04:31:28.046 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T04:31:28.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:27 smithi045 ceph-mon[32299]: pgmap v431: 129 pgs: 129 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-06-06T04:31:28.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:27 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3601941072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:31:28.304 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:31:28.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:27 smithi086 ceph-mon[36053]: pgmap v431: 129 pgs: 129 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-06-06T04:31:28.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:27 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3601941072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:31:28.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:27 smithi154 ceph-mon[36428]: pgmap v431: 129 pgs: 129 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-06-06T04:31:28.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:27 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3601941072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:31:29.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.936+0000 7f2e6bb1f700 1 -- 172.21.15.45:0/1856318605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e64102030 msgr2=0x7f2e64105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:29.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.936+0000 7f2e6bb1f700 1 --2- 172.21.15.45:0/1856318605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e64102030 0x7f2e64105ff0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2e580099c0 tx=0x7f2e58009cd0 comp rx=0 tx=0).stop 2024-06-06T04:31:29.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.936+0000 7f2e6bb1f700 1 -- 172.21.15.45:0/1856318605 shutdown_connections 2024-06-06T04:31:29.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.936+0000 7f2e6bb1f700 1 --2- 172.21.15.45:0/1856318605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e641066d0 0x7f2e6410e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:29.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.936+0000 7f2e6bb1f700 1 --2- 172.21.15.45:0/1856318605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e64101680 0x7f2e64101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:29.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.936+0000 7f2e6bb1f700 1 --2- 172.21.15.45:0/1856318605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e64102030 0x7f2e64105ff0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:29.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.936+0000 7f2e6bb1f700 1 -- 172.21.15.45:0/1856318605 >> 172.21.15.45:0/1856318605 conn(0x7f2e640773e0 msgr2=0x7f2e640777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:29.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.936+0000 7f2e6bb1f700 1 -- 172.21.15.45:0/1856318605 shutdown_connections 2024-06-06T04:31:29.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.936+0000 7f2e6bb1f700 1 -- 172.21.15.45:0/1856318605 wait complete. 2024-06-06T04:31:29.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.937+0000 7f2e6bb1f700 1 Processor -- start 2024-06-06T04:31:29.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.937+0000 7f2e6bb1f700 1 -- start start 2024-06-06T04:31:29.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.937+0000 7f2e6bb1f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e64101680 0x7f2e64103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:29.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.937+0000 7f2e6bb1f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e64102030 0x7f2e641035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:29.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.937+0000 7f2e6bb1f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e641066d0 0x7f2e64103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:29.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.937+0000 7f2e6bb1f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e641115c0 con 0x7f2e641066d0 2024-06-06T04:31:29.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.937+0000 7f2e6bb1f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e64111440 con 0x7f2e64101680 2024-06-06T04:31:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.937+0000 7f2e6bb1f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2e64111740 con 0x7f2e64102030 2024-06-06T04:31:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.937+0000 7f2e6a0bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e641066d0 0x7f2e64103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.937+0000 7f2e698bb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e64101680 0x7f2e64103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.937+0000 7f2e6a0bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e641066d0 0x7f2e64103b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33106/0 (socket says 172.21.15.45:33106) 2024-06-06T04:31:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.938+0000 7f2e6a0bc700 1 -- 172.21.15.45:0/2301257223 learned_addr learned my addr 172.21.15.45:0/2301257223 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.938+0000 7f2e690ba700 1 --2- 172.21.15.45:0/2301257223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e64102030 0x7f2e641035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.938+0000 7f2e6a0bc700 1 -- 172.21.15.45:0/2301257223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e64102030 msgr2=0x7f2e641035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.938+0000 7f2e6a0bc700 1 --2- 172.21.15.45:0/2301257223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e64102030 0x7f2e641035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.938+0000 7f2e6a0bc700 1 -- 172.21.15.45:0/2301257223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e64101680 msgr2=0x7f2e64103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.938+0000 7f2e6a0bc700 1 --2- 172.21.15.45:0/2301257223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e64101680 0x7f2e64103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.938+0000 7f2e6a0bc700 1 -- 172.21.15.45:0/2301257223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e641a7150 con 0x7f2e641066d0 2024-06-06T04:31:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.938+0000 7f2e6a0bc700 1 --2- 172.21.15.45:0/2301257223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e641066d0 0x7f2e64103b10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c00c900 tx=0x7f2e5c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.938+0000 7f2e62ffd700 1 -- 172.21.15.45:0/2301257223 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c0078d0 con 0x7f2e641066d0 2024-06-06T04:31:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.938+0000 7f2e6bb1f700 1 -- 172.21.15.45:0/2301257223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e641a7440 con 0x7f2e641066d0 2024-06-06T04:31:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.938+0000 7f2e62ffd700 1 -- 172.21.15.45:0/2301257223 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e5c00f450 con 0x7f2e641066d0 2024-06-06T04:31:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.938+0000 7f2e62ffd700 1 -- 172.21.15.45:0/2301257223 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c018620 con 0x7f2e641066d0 2024-06-06T04:31:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.938+0000 7f2e6bb1f700 1 -- 172.21.15.45:0/2301257223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e641a7900 con 0x7f2e641066d0 2024-06-06T04:31:29.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.939+0000 7f2e6bb1f700 1 -- 172.21.15.45:0/2301257223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e64069180 con 0x7f2e641066d0 2024-06-06T04:31:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.940+0000 7f2e62ffd700 1 -- 172.21.15.45:0/2301257223 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f2e5c00f600 con 0x7f2e641066d0 2024-06-06T04:31:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.941+0000 7f2e62ffd700 1 --2- 172.21.15.45:0/2301257223 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2e4c0a2cb0 0x7f2e4c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.941+0000 7f2e698bb700 1 --2- 172.21.15.45:0/2301257223 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2e4c0a2cb0 0x7f2e4c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.941+0000 7f2e62ffd700 1 -- 172.21.15.45:0/2301257223 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f2e5c0cf4b0 con 0x7f2e641066d0 2024-06-06T04:31:29.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.942+0000 7f2e698bb700 1 --2- 172.21.15.45:0/2301257223 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2e4c0a2cb0 0x7f2e4c0a5170 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400a400 tx=0x7f2e5400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:29.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:29.946+0000 7f2e62ffd700 1 -- 172.21.15.45:0/2301257223 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2e5c014030 con 0x7f2e641066d0 2024-06-06T04:31:30.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:30.151+0000 7f2e6bb1f700 1 -- 172.21.15.45:0/2301257223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e641045c0 con 0x7f2e641066d0 2024-06-06T04:31:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:30.153+0000 7f2e62ffd700 1 -- 172.21.15.45:0/2301257223 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24789 (secure 0 0 0) 0x7f2e5c09be90 con 0x7f2e641066d0 2024-06-06T04:31:30.154 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:31:30.154 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":100,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","created":"2024-06-06T04:19:02.626916+0000","modified":"2024-06-06T04:29:05.901370+0000","last_up_change":"2024-06-06T04:26:17.390429+0000","last_in_change":"2024-06-06T04:25:56.938388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T04:22:28.797137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T04:27:17.750625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T04:27:21.486917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T04:27:31.029818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d3cee4a-5025-45b3-b217-082af37e79db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6803","nonce":2111759250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6805","nonce":2111759250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6809","nonce":2111759250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6807","nonce":2111759250}]},"public_addr":"172.21.15.45:6803/2111759250","cluster_addr":"172.21.15.45:6805/2111759250","heartbeat_back_addr":"172.21.15.45:6809/2111759250","heartbeat_front_addr":"172.21.15.45:6807/2111759250","state":["exists","up"]},{"osd":1,"uuid":"75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6801","nonce":3732225287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6803","nonce":3732225287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6807","nonce":3732225287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6805","nonce":3732225287}]},"public_addr":"172.21.15.86:6801/3732225287","cluster_addr":"172.21.15.86:6803/3732225287","heartbeat_back_addr":"172.21.15.86:6807/3732225287","heartbeat_front_addr":"172.21.15.86:6805/3732225287","state":["exists","up"]},{"osd":2,"uuid":"bb20565f-f9ae-47e3-9604-c69f737cff4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6801","nonce":2898783039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6803","nonce":2898783039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6807","nonce":2898783039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6805","nonce":2898783039}]},"public_addr":"172.21.15.154:6801/2898783039","cluster_addr":"172.21.15.154:6803/2898783039","heartbeat_back_addr":"172.21.15.154:6807/2898783039","heartbeat_front_addr":"172.21.15.154:6805/2898783039","state":["exists","up"]},{"osd":3,"uuid":"e1160799-7811-4326-9792-7e6c7d189c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6811","nonce":3436753135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6813","nonce":3436753135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6817","nonce":3436753135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6815","nonce":3436753135}]},"public_addr":"172.21.15.45:6811/3436753135","cluster_addr":"172.21.15.45:6813/3436753135","heartbeat_back_addr":"172.21.15.45:6817/3436753135","heartbeat_front_addr":"172.21.15.45:6815/3436753135","state":["exists","up"]},{"osd":4,"uuid":"6b389bc8-e19a-47f3-83ff-7b782d146d0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6809","nonce":3030235796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6811","nonce":3030235796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6815","nonce":3030235796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6813","nonce":3030235796}]},"public_addr":"172.21.15.86:6809/3030235796","cluster_addr":"172.21.15.86:6811/3030235796","heartbeat_back_addr":"172.21.15.86:6815/3030235796","heartbeat_front_addr":"172.21.15.86:6813/3030235796","state":["exists","up"]},{"osd":5,"uuid":"069de7b2-89bf-4150-bed7-7b8832573dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6809","nonce":2222268646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6811","nonce":2222268646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6815","nonce":2222268646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6813","nonce":2222268646}]},"public_addr":"172.21.15.154:6809/2222268646","cluster_addr":"172.21.15.154:6811/2222268646","heartbeat_back_addr":"172.21.15.154:6815/2222268646","heartbeat_front_addr":"172.21.15.154:6813/2222268646","state":["exists","up"]},{"osd":6,"uuid":"4c70b627-eafa-4e27-ba2a-4d8e1e335962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6819","nonce":384734939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6821","nonce":384734939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6825","nonce":384734939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6823","nonce":384734939}]},"public_addr":"172.21.15.45:6819/384734939","cluster_addr":"172.21.15.45:6821/384734939","heartbeat_back_addr":"172.21.15.45:6825/384734939","heartbeat_front_addr":"172.21.15.45:6823/384734939","state":["exists","up"]},{"osd":7,"uuid":"3b09b875-be52-4be0-b116-7fd9b0c26705","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6817","nonce":1119791005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6819","nonce":1119791005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6823","nonce":1119791005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6821","nonce":1119791005}]},"public_addr":"172.21.15.86:6817/1119791005","cluster_addr":"172.21.15.86:6819/1119791005","heartbeat_back_addr":"172.21.15.86:6823/1119791005","heartbeat_front_addr":"172.21.15.86:6821/1119791005","state":["exists","up"]},{"osd":8,"uuid":"cdf1dd1b-5392-4836-8e65-4c7c5b688d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6817","nonce":3054812496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6819","nonce":3054812496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6823","nonce":3054812496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6821","nonce":3054812496}]},"public_addr":"172.21.15.154:6817/3054812496","cluster_addr":"172.21.15.154:6819/3054812496","heartbeat_back_addr":"172.21.15.154:6823/3054812496","heartbeat_front_addr":"172.21.15.154:6821/3054812496","state":["exists","up"]},{"osd":9,"uuid":"acede92d-2cc1-4967-a357-7e5e201e99d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6827","nonce":1360655510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6829","nonce":1360655510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6833","nonce":1360655510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6831","nonce":1360655510}]},"public_addr":"172.21.15.45:6827/1360655510","cluster_addr":"172.21.15.45:6829/1360655510","heartbeat_back_addr":"172.21.15.45:6833/1360655510","heartbeat_front_addr":"172.21.15.45:6831/1360655510","state":["exists","up"]},{"osd":10,"uuid":"a2d0c488-1892-4da4-adbf-952c22df0ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6825","nonce":1701691877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6827","nonce":1701691877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6831","nonce":1701691877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6829","nonce":1701691877}]},"public_addr":"172.21.15.86:6825/1701691877","cluster_addr":"172.21.15.86:6827/1701691877","heartbeat_back_addr":"172.21.15.86:6831/1701691877","heartbeat_front_addr":"172.21.15.86:6829/1701691877","state":["exists","up"]},{"osd":11,"uuid":"6b349c4c-b672-480b-ac6a-8594dab0eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6825","nonce":2065648007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6827","nonce":2065648007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6831","nonce":2065648007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6829","nonce":2065648007}]},"public_addr":"172.21.15.154:6825/2065648007","cluster_addr":"172.21.15.154:6827/2065648007","heartbeat_back_addr":"172.21.15.154:6831/2065648007","heartbeat_front_addr":"172.21.15.154:6829/2065648007","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:21:37.147394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:00.776617+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:24.786849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:50.487455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:14.869272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:39.815158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:05.972619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:31.410063+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:56.795119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:23.114884+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:49.256504+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:26:15.251967+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/36732849":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/1500318775":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6801/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6800/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/2041380391":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/2504003341":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/3798552599":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/498952822":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6800/1562511056":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6801/1562511056":"2024-06-07T04:19:37.026199+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T04:31:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:30.155+0000 7f2e6bb1f700 1 -- 172.21.15.45:0/2301257223 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2e4c0a2cb0 msgr2=0x7f2e4c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:30.155+0000 7f2e6bb1f700 1 --2- 172.21.15.45:0/2301257223 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2e4c0a2cb0 0x7f2e4c0a5170 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400a400 tx=0x7f2e5400d040 comp rx=0 tx=0).stop 2024-06-06T04:31:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:30.155+0000 7f2e6bb1f700 1 -- 172.21.15.45:0/2301257223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e641066d0 msgr2=0x7f2e64103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:30.155+0000 7f2e6bb1f700 1 --2- 172.21.15.45:0/2301257223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e641066d0 0x7f2e64103b10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c00c900 tx=0x7f2e5c00cc10 comp rx=0 tx=0).stop 2024-06-06T04:31:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:30.155+0000 7f2e6bb1f700 1 -- 172.21.15.45:0/2301257223 shutdown_connections 2024-06-06T04:31:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:30.155+0000 7f2e6bb1f700 1 --2- 172.21.15.45:0/2301257223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e641066d0 0x7f2e64103b10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:30.155+0000 7f2e6bb1f700 1 --2- 172.21.15.45:0/2301257223 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2e4c0a2cb0 0x7f2e4c0a5170 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:30.155+0000 7f2e6bb1f700 1 --2- 172.21.15.45:0/2301257223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e64101680 0x7f2e64103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:30.155+0000 7f2e6bb1f700 1 --2- 172.21.15.45:0/2301257223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e64102030 0x7f2e641035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:30.155+0000 7f2e6bb1f700 1 -- 172.21.15.45:0/2301257223 >> 172.21.15.45:0/2301257223 conn(0x7f2e640773e0 msgr2=0x7f2e6406af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:30.155+0000 7f2e6bb1f700 1 -- 172.21.15.45:0/2301257223 shutdown_connections 2024-06-06T04:31:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:30.155+0000 7f2e6bb1f700 1 -- 172.21.15.45:0/2301257223 wait complete. 2024-06-06T04:31:30.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:29 smithi045 ceph-mon[32299]: pgmap v432: 129 pgs: 129 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-06-06T04:31:30.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:29 smithi086 ceph-mon[36053]: pgmap v432: 129 pgs: 129 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-06-06T04:31:30.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:29 smithi154 ceph-mon[36428]: pgmap v432: 129 pgs: 129 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-06-06T04:31:30.863 DEBUG:tasks.fwd_scrub:Yielding 2024-06-06T04:31:30.863 INFO:teuthology.run_tasks:Running task background_exec... 2024-06-06T04:31:30.874 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-06-06T04:31:30.874 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-06-06T04:31:30.874 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-06-06T04:31:30.875 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-06T04:31:30.875 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-06T04:31:30.878 INFO:teuthology.run_tasks:Running task workunit... 2024-06-06T04:31:30.891 INFO:tasks.workunit:Pulling workunits from ref 3d746ca5346f34eef7e826a1eb71e59e02b682ae 2024-06-06T04:31:30.892 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-06T04:31:30.892 INFO:tasks.workunit:timeout=3h 2024-06-06T04:31:30.892 INFO:tasks.workunit:cleanup=True 2024-06-06T04:31:30.892 DEBUG:teuthology.orchestra.run.smithi045:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-06T04:31:30.950 INFO:teuthology.orchestra.run.smithi045.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-06-06T04:31:30.950 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-06-06T04:31:30.951 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6bh/107d Inode: 1 Links: 2 2024-06-06T04:31:30.951 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T04:31:30.951 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:fusefs_t:s0 2024-06-06T04:31:30.951 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-06-06 04:28:38.349761973 +0000 2024-06-06T04:31:30.951 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-06-06 04:28:38.349761973 +0000 2024-06-06T04:31:30.951 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-06-06 04:31:17.572712901 +0000 2024-06-06T04:31:30.951 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2024-06-06T04:31:30.951 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-06-06T04:31:30.951 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-06-06T04:31:31.025 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3d746ca5346f34eef7e826a1eb71e59e02b682ae 2024-06-06T04:31:31.082 INFO:tasks.workunit.client.0.smithi045.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-06T04:31:31.135 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:31:31.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:30 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2301257223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:31:31.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:30 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2301257223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:31:31.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:30 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2301257223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:31:32.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:31 smithi045 ceph-mon[32299]: pgmap v433: 129 pgs: 129 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-06-06T04:31:32.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:31 smithi086 ceph-mon[36053]: pgmap v433: 129 pgs: 129 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-06-06T04:31:32.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:31 smithi154 ceph-mon[36428]: pgmap v433: 129 pgs: 129 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-06-06T04:31:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.005+0000 7fce93ac3700 1 -- 172.21.15.45:0/869885035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8c10ef70 msgr2=0x7fce8c111360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.005+0000 7fce93ac3700 1 --2- 172.21.15.45:0/869885035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8c10ef70 0x7fce8c111360 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fce8400b0a0 tx=0x7fce8400b3b0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.005+0000 7fce93ac3700 1 -- 172.21.15.45:0/869885035 shutdown_connections 2024-06-06T04:31:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.005+0000 7fce93ac3700 1 --2- 172.21.15.45:0/869885035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8c10ef70 0x7fce8c111360 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.005+0000 7fce93ac3700 1 --2- 172.21.15.45:0/869885035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8c101ed0 0x7fce8c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.005+0000 7fce93ac3700 1 --2- 172.21.15.45:0/869885035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce8c1027f0 0x7fce8c10ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.005+0000 7fce93ac3700 1 -- 172.21.15.45:0/869885035 >> 172.21.15.45:0/869885035 conn(0x7fce8c077400 msgr2=0x7fce8c077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.005+0000 7fce93ac3700 1 -- 172.21.15.45:0/869885035 shutdown_connections 2024-06-06T04:31:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.005+0000 7fce93ac3700 1 -- 172.21.15.45:0/869885035 wait complete. 2024-06-06T04:31:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce93ac3700 1 Processor -- start 2024-06-06T04:31:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce93ac3700 1 -- start start 2024-06-06T04:31:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce93ac3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8c101ed0 0x7fce8c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce93ac3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8c1027f0 0x7fce8c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce93ac3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce8c10ef70 0x7fce8c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce93ac3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c1148f0 con 0x7fce8c1027f0 2024-06-06T04:31:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce93ac3700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c114770 con 0x7fce8c101ed0 2024-06-06T04:31:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce93ac3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c114a70 con 0x7fce8c10ef70 2024-06-06T04:31:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce9105e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8c1027f0 0x7fce8c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce9105e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8c1027f0 0x7fce8c19b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33112/0 (socket says 172.21.15.45:33112) 2024-06-06T04:31:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce9105e700 1 -- 172.21.15.45:0/3505986344 learned_addr learned my addr 172.21.15.45:0/3505986344 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce92060700 1 --2- 172.21.15.45:0/3505986344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce8c10ef70 0x7fce8c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce9185f700 1 --2- 172.21.15.45:0/3505986344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8c101ed0 0x7fce8c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce9105e700 1 -- 172.21.15.45:0/3505986344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce8c10ef70 msgr2=0x7fce8c19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce9105e700 1 --2- 172.21.15.45:0/3505986344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce8c10ef70 0x7fce8c19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce9105e700 1 -- 172.21.15.45:0/3505986344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8c101ed0 msgr2=0x7fce8c19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce9105e700 1 --2- 172.21.15.45:0/3505986344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8c101ed0 0x7fce8c19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.006+0000 7fce9105e700 1 -- 172.21.15.45:0/3505986344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8c19fdb0 con 0x7fce8c1027f0 2024-06-06T04:31:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.007+0000 7fce9105e700 1 --2- 172.21.15.45:0/3505986344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8c1027f0 0x7fce8c19b2e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00c970 tx=0x7fce7c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.007+0000 7fce82ffd700 1 -- 172.21.15.45:0/3505986344 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c007aa0 con 0x7fce8c1027f0 2024-06-06T04:31:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.007+0000 7fce82ffd700 1 -- 172.21.15.45:0/3505986344 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce7c004ce0 con 0x7fce8c1027f0 2024-06-06T04:31:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.007+0000 7fce93ac3700 1 -- 172.21.15.45:0/3505986344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8c1a00a0 con 0x7fce8c1027f0 2024-06-06T04:31:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.007+0000 7fce82ffd700 1 -- 172.21.15.45:0/3505986344 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c005520 con 0x7fce8c1027f0 2024-06-06T04:31:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.007+0000 7fce93ac3700 1 -- 172.21.15.45:0/3505986344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce8c1a7920 con 0x7fce8c1027f0 2024-06-06T04:31:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.009+0000 7fce82ffd700 1 -- 172.21.15.45:0/3505986344 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fce7c004750 con 0x7fce8c1027f0 2024-06-06T04:31:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.010+0000 7fce82ffd700 1 --2- 172.21.15.45:0/3505986344 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fce780a2d30 0x7fce780a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.010+0000 7fce9185f700 1 --2- 172.21.15.45:0/3505986344 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fce780a2d30 0x7fce780a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.010+0000 7fce82ffd700 1 -- 172.21.15.45:0/3505986344 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fce7c0cf370 con 0x7fce8c1027f0 2024-06-06T04:31:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.010+0000 7fce9e346b80 1 Processor -- start 2024-06-06T04:31:33.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.010+0000 7fce9185f700 1 --2- 172.21.15.45:0/3505986344 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fce780a2d30 0x7fce780a51f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fce88006fd0 tx=0x7fce88008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce9e346b80 1 -- start start 2024-06-06T04:31:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce9e346b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2a6c0 0x55876ed2aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce9e346b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed2afe0 0x55876ed24a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce9e346b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed24fa0 0x55876ed253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce9e346b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55876ec76890 con 0x55876ed24fa0 2024-06-06T04:31:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce9e346b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55876ec1e7a0 con 0x55876ed2a6c0 2024-06-06T04:31:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce9e346b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55876ec72850 con 0x55876ed2afe0 2024-06-06T04:31:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce92060700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed24fa0 0x55876ed253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce92060700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed24fa0 0x55876ed253c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33122/0 (socket says 172.21.15.45:33122) 2024-06-06T04:31:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce92060700 1 -- 172.21.15.45:0/638030876 learned_addr learned my addr 172.21.15.45:0/638030876 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce9185f700 1 --2- 172.21.15.45:0/638030876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2a6c0 0x55876ed2aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce9105e700 1 --2- 172.21.15.45:0/638030876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed2afe0 0x55876ed24a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce92060700 1 -- 172.21.15.45:0/638030876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed2afe0 msgr2=0x55876ed24a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce92060700 1 --2- 172.21.15.45:0/638030876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed2afe0 0x55876ed24a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce92060700 1 -- 172.21.15.45:0/638030876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2a6c0 msgr2=0x55876ed2aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce92060700 1 --2- 172.21.15.45:0/638030876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2a6c0 0x55876ed2aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.011+0000 7fce92060700 1 -- 172.21.15.45:0/638030876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55876ec14fd0 con 0x55876ed24fa0 2024-06-06T04:31:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.012+0000 7fce92060700 1 --2- 172.21.15.45:0/638030876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed24fa0 0x55876ed253c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fce84009390 tx=0x7fce840093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30640256aa1ba570 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.012+0000 7fce77fff700 1 -- 172.21.15.45:0/638030876 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84020070 con 0x55876ed24fa0 2024-06-06T04:31:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.012+0000 7fce77fff700 1 -- 172.21.15.45:0/638030876 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce840074e0 con 0x55876ed24fa0 2024-06-06T04:31:33.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.012+0000 7fce77fff700 1 -- 172.21.15.45:0/638030876 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce840052a0 con 0x55876ed24fa0 2024-06-06T04:31:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.012+0000 7fce9e346b80 1 -- 172.21.15.45:0/638030876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed24fa0 msgr2=0x55876ed253c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.012+0000 7fce9e346b80 1 --2- 172.21.15.45:0/638030876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed24fa0 0x55876ed253c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fce84009390 tx=0x7fce840093c0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.013+0000 7fce9e346b80 1 -- 172.21.15.45:0/638030876 shutdown_connections 2024-06-06T04:31:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.013+0000 7fce9e346b80 1 --2- 172.21.15.45:0/638030876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed24fa0 0x55876ed253c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.013+0000 7fce9e346b80 1 --2- 172.21.15.45:0/638030876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2a6c0 0x55876ed2aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.013+0000 7fce9e346b80 1 --2- 172.21.15.45:0/638030876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed2afe0 0x55876ed24a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.013+0000 7fce9e346b80 1 -- 172.21.15.45:0/638030876 >> 172.21.15.45:0/638030876 conn(0x55876ec4fd40 msgr2=0x55876ed1f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.013+0000 7fce9e346b80 1 -- 172.21.15.45:0/638030876 shutdown_connections 2024-06-06T04:31:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.013+0000 7fce9e346b80 1 -- 172.21.15.45:0/638030876 wait complete. 2024-06-06T04:31:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.013+0000 7fce9e346b80 1 -- start start 2024-06-06T04:31:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.013+0000 7fce9e346b80 1 Processor -- start 2024-06-06T04:31:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.013+0000 7fce9e346b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed24fa0 0x55876ed30d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.013+0000 7fce9e346b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2a6c0 0x55876ed2e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.013+0000 7fce9e346b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2afe0 0x55876ed2e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.013+0000 7fce9e346b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55876ed319f0 con 0x55876ed2a6c0 2024-06-06T04:31:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce9e346b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55876ec16c70 con 0x55876ed2afe0 2024-06-06T04:31:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce9e346b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55876ed31b70 con 0x55876ed24fa0 2024-06-06T04:31:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce9e346b80 10 client.0 _ll_register_callbacks cb 0x55876ec109a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:31:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce9105e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2a6c0 0x55876ed2e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce9105e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2a6c0 0x55876ed2e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33138/0 (socket says 172.21.15.45:33138) 2024-06-06T04:31:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce9105e700 1 -- 172.21.15.45:0/780703359 learned_addr learned my addr 172.21.15.45:0/780703359 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce9185f700 1 --2- 172.21.15.45:0/780703359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed24fa0 0x55876ed30d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce92060700 1 --2- 172.21.15.45:0/780703359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2afe0 0x55876ed2e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce9105e700 1 -- 172.21.15.45:0/780703359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed24fa0 msgr2=0x55876ed30d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:33.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce9105e700 1 --2- 172.21.15.45:0/780703359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed24fa0 0x55876ed30d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce9105e700 1 -- 172.21.15.45:0/780703359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2afe0 msgr2=0x55876ed2e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce9105e700 1 --2- 172.21.15.45:0/780703359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2afe0 0x55876ed2e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce9105e700 1 -- 172.21.15.45:0/780703359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55876ebc5950 con 0x55876ed2a6c0 2024-06-06T04:31:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce92060700 1 --2- 172.21.15.45:0/780703359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2afe0 0x55876ed2e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:31:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce9105e700 1 --2- 172.21.15.45:0/780703359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2a6c0 0x55876ed2e080 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0cc990 tx=0x7fce7c09b150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410a98bf393b2d86 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce757fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:31:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce757fa700 1 -- 172.21.15.45:0/780703359 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c0cf1b0 con 0x55876ed2a6c0 2024-06-06T04:31:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce757fa700 1 -- 172.21.15.45:0/780703359 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fce7c0ccbb0 con 0x55876ed2a6c0 2024-06-06T04:31:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.014+0000 7fce9e346b80 1 -- 172.21.15.45:0/780703359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce840074e0 con 0x55876ed2a6c0 2024-06-06T04:31:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.015+0000 7fce757fa700 1 -- 172.21.15.45:0/780703359 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fce7c09b740 con 0x55876ed2a6c0 2024-06-06T04:31:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.015+0000 7fce9e346b80 10 client.15312 fetch_fsmap learned FSMap version 27 2024-06-06T04:31:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.015+0000 7fce9e346b80 1 -- 172.21.15.45:0/780703359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55876ed3c380 con 0x55876ed2a6c0 2024-06-06T04:31:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.015+0000 7fce757fa700 1 -- 172.21.15.45:0/780703359 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce840074e0 con 0x55876ed2a6c0 2024-06-06T04:31:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.015+0000 7fce757fa700 1 -- 172.21.15.45:0/780703359 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fce7c0bb040 con 0x55876ed2a6c0 2024-06-06T04:31:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.015+0000 7fce9e346b80 10 client.15312 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:31:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.015+0000 7fce9e346b80 10 client.15312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:31:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.015+0000 7fce9e346b80 20 client.15312 populate_metadata read hostname 'smithi045' 2024-06-06T04:31:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.015+0000 7fce9e346b80 1 --2- 172.21.15.45:0/780703359 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55876ed3e9c0 0x55876ed40db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.015+0000 7fce9e346b80 4 client.15312 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:31:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.015+0000 7fce9e346b80 1 -- 172.21.15.45:0/780703359 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55876ec83d90 con 0x55876ed3e9c0 2024-06-06T04:31:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.015+0000 7fce9185f700 1 --2- 172.21.15.45:0/780703359 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55876ed3e9c0 0x55876ed40db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.016+0000 7fce757fa700 10 client.15312 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.016+0000 7fce9185f700 1 --2- 172.21.15.45:0/780703359 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55876ed3e9c0 0x55876ed40db0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75204c30e254d98d server_cookie=99a228f063d55027 in_seq=0 out_seq=0 2024-06-06T04:31:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.016+0000 7fce757fa700 1 -- 172.21.15.45:0/780703359 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55876ec83d90 con 0x55876ed3e9c0 2024-06-06T04:31:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.017+0000 7fce757fa700 10 client.15312 handle_command_reply: tid=0 2024-06-06T04:31:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.017+0000 7fce9e346b80 1 client.15312 shutdown 2024-06-06T04:31:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.017+0000 7fce9e346b80 1 -- 172.21.15.45:0/780703359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2a6c0 msgr2=0x55876ed2e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.017+0000 7fce9e346b80 1 --2- 172.21.15.45:0/780703359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2a6c0 0x55876ed2e080 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0cc990 tx=0x7fce7c09b150 comp rx=0 tx=0).stop 2024-06-06T04:31:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.017+0000 7fce9e346b80 1 -- 172.21.15.45:0/780703359 shutdown_connections 2024-06-06T04:31:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.017+0000 7fce9e346b80 1 --2- 172.21.15.45:0/780703359 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55876ed3e9c0 0x55876ed40db0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.017+0000 7fce9e346b80 1 --2- 172.21.15.45:0/780703359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2afe0 0x55876ed2e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.017+0000 7fce9e346b80 1 --2- 172.21.15.45:0/780703359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed24fa0 0x55876ed30d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.017+0000 7fce757fa700 0 client.15312 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.017+0000 7fce9e346b80 1 --2- 172.21.15.45:0/780703359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2a6c0 0x55876ed2e080 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.017+0000 7fce9e346b80 1 -- 172.21.15.45:0/780703359 >> 172.21.15.45:0/780703359 conn(0x55876ec4fd40 msgr2=0x55876ed23a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.017+0000 7fce9e346b80 1 -- 172.21.15.45:0/780703359 shutdown_connections 2024-06-06T04:31:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.017+0000 7fce9e346b80 1 -- 172.21.15.45:0/780703359 wait complete. 2024-06-06T04:31:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.017+0000 7fce9e346b80 20 client.15312 trim_cache size 0 max 16384 2024-06-06T04:31:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.038+0000 7fce9e346b80 1 Processor -- start 2024-06-06T04:31:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.038+0000 7fce9e346b80 1 -- start start 2024-06-06T04:31:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce9e346b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed24fa0 0x55876ed24a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce9e346b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2a6c0 0x55876ed2e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce9e346b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2afe0 0x55876ed29930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce9e346b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c1b2cd0 con 0x55876ed2afe0 2024-06-06T04:31:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce9e346b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c1b0560 con 0x55876ed2a6c0 2024-06-06T04:31:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce9e346b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c1ae3b0 con 0x55876ed24fa0 2024-06-06T04:31:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce92060700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2afe0 0x55876ed29930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce92060700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2afe0 0x55876ed29930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33142/0 (socket says 172.21.15.45:33142) 2024-06-06T04:31:33.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce92060700 1 -- 172.21.15.45:0/1727180629 learned_addr learned my addr 172.21.15.45:0/1727180629 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce9185f700 1 --2- 172.21.15.45:0/1727180629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed24fa0 0x55876ed24a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce9105e700 1 --2- 172.21.15.45:0/1727180629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2a6c0 0x55876ed2e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce92060700 1 -- 172.21.15.45:0/1727180629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed24fa0 msgr2=0x55876ed24a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce92060700 1 --2- 172.21.15.45:0/1727180629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed24fa0 0x55876ed24a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce92060700 1 -- 172.21.15.45:0/1727180629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2a6c0 msgr2=0x55876ed2e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce92060700 1 --2- 172.21.15.45:0/1727180629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2a6c0 0x55876ed2e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce92060700 1 -- 172.21.15.45:0/1727180629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55876ed28bd0 con 0x55876ed2afe0 2024-06-06T04:31:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce9185f700 1 --2- 172.21.15.45:0/1727180629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed24fa0 0x55876ed24a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:31:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce9105e700 1 --2- 172.21.15.45:0/1727180629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2a6c0 0x55876ed2e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:31:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce92060700 1 --2- 172.21.15.45:0/1727180629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2afe0 0x55876ed29930 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fce84023000 tx=0x7fce84009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c5cf40463e5ca7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce777fe700 1 -- 172.21.15.45:0/1727180629 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84020070 con 0x55876ed2afe0 2024-06-06T04:31:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce777fe700 1 -- 172.21.15.45:0/1727180629 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce840074e0 con 0x55876ed2afe0 2024-06-06T04:31:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.039+0000 7fce777fe700 1 -- 172.21.15.45:0/1727180629 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce840052b0 con 0x55876ed2afe0 2024-06-06T04:31:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.040+0000 7fce9e346b80 1 -- 172.21.15.45:0/1727180629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2afe0 msgr2=0x55876ed29930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.040+0000 7fce9e346b80 1 --2- 172.21.15.45:0/1727180629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2afe0 0x55876ed29930 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fce84023000 tx=0x7fce84009e10 comp rx=0 tx=0).stop 2024-06-06T04:31:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.040+0000 7fce9e346b80 1 -- 172.21.15.45:0/1727180629 shutdown_connections 2024-06-06T04:31:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.040+0000 7fce9e346b80 1 --2- 172.21.15.45:0/1727180629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2afe0 0x55876ed29930 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.040+0000 7fce9e346b80 1 --2- 172.21.15.45:0/1727180629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed24fa0 0x55876ed24a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.040+0000 7fce9e346b80 1 --2- 172.21.15.45:0/1727180629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2a6c0 0x55876ed2e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.040+0000 7fce9e346b80 1 -- 172.21.15.45:0/1727180629 >> 172.21.15.45:0/1727180629 conn(0x55876ec4fd40 msgr2=0x55876ed29220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.040+0000 7fce9e346b80 1 -- 172.21.15.45:0/1727180629 shutdown_connections 2024-06-06T04:31:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.040+0000 7fce9e346b80 1 -- 172.21.15.45:0/1727180629 wait complete. 2024-06-06T04:31:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.040+0000 7fce9e346b80 1 -- start start 2024-06-06T04:31:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.041+0000 7fce9e346b80 1 Processor -- start 2024-06-06T04:31:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.041+0000 7fce9e346b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed24fa0 0x55876ed33f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.041+0000 7fce9e346b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2a6c0 0x55876ed34450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.041+0000 7fce9e346b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2afe0 0x55876ed349d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.041+0000 7fce9e346b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c1aea20 con 0x55876ed2afe0 2024-06-06T04:31:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.041+0000 7fce9e346b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c1afe90 con 0x55876ed2a6c0 2024-06-06T04:31:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.041+0000 7fce9e346b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c1af0f0 con 0x55876ed24fa0 2024-06-06T04:31:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.041+0000 7fce9e346b80 10 client.0 _ll_register_callbacks cb 0x55876ec109a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:31:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.041+0000 7fce92060700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2afe0 0x55876ed349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.041+0000 7fce92060700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2afe0 0x55876ed349d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33158/0 (socket says 172.21.15.45:33158) 2024-06-06T04:31:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.041+0000 7fce92060700 1 -- 172.21.15.45:0/1807528319 learned_addr learned my addr 172.21.15.45:0/1807528319 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.041+0000 7fce9105e700 1 --2- 172.21.15.45:0/1807528319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2a6c0 0x55876ed34450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.041+0000 7fce9185f700 1 --2- 172.21.15.45:0/1807528319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed24fa0 0x55876ed33f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.041+0000 7fce92060700 1 -- 172.21.15.45:0/1807528319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed24fa0 msgr2=0x55876ed33f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.041+0000 7fce92060700 1 --2- 172.21.15.45:0/1807528319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed24fa0 0x55876ed33f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.041+0000 7fce92060700 1 -- 172.21.15.45:0/1807528319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2a6c0 msgr2=0x55876ed34450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.042+0000 7fce92060700 1 --2- 172.21.15.45:0/1807528319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2a6c0 0x55876ed34450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.042+0000 7fce92060700 1 -- 172.21.15.45:0/1807528319 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55876eda9970 con 0x55876ed2afe0 2024-06-06T04:31:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.042+0000 7fce92060700 1 --2- 172.21.15.45:0/1807528319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2afe0 0x55876ed349d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fce840078c0 tx=0x7fce840099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c45d1efc69e9928 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.042+0000 7fce757fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:31:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.042+0000 7fce757fa700 1 -- 172.21.15.45:0/1807528319 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8402b050 con 0x55876ed2afe0 2024-06-06T04:31:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.042+0000 7fce757fa700 1 -- 172.21.15.45:0/1807528319 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fce840050b0 con 0x55876ed2afe0 2024-06-06T04:31:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.042+0000 7fce9e346b80 1 -- 172.21.15.45:0/1807528319 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce840074e0 con 0x55876ed2afe0 2024-06-06T04:31:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.042+0000 7fce757fa700 1 -- 172.21.15.45:0/1807528319 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fce8403a050 con 0x55876ed2afe0 2024-06-06T04:31:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.042+0000 7fce9e346b80 10 client.15318 fetch_fsmap learned FSMap version 27 2024-06-06T04:31:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.042+0000 7fce9e346b80 1 -- 172.21.15.45:0/1807528319 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55876edaabc0 con 0x55876ed2afe0 2024-06-06T04:31:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.042+0000 7fce757fa700 1 -- 172.21.15.45:0/1807528319 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce840074e0 con 0x55876ed2afe0 2024-06-06T04:31:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.043+0000 7fce757fa700 1 -- 172.21.15.45:0/1807528319 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fce84011070 con 0x55876ed2afe0 2024-06-06T04:31:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.043+0000 7fce9e346b80 10 client.15318 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:31:33.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.043+0000 7fce9e346b80 10 client.15318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:31:33.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.043+0000 7fce9e346b80 20 client.15318 populate_metadata read hostname 'smithi045' 2024-06-06T04:31:33.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.043+0000 7fce9e346b80 1 --2- 172.21.15.45:0/1807528319 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55876edaad90 0x55876edad180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:33.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.043+0000 7fce9e346b80 4 client.15318 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-06T04:31:33.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.043+0000 7fce9e346b80 1 -- 172.21.15.45:0/1807528319 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55876ec7e310 con 0x55876edaad90 2024-06-06T04:31:33.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.043+0000 7fce9185f700 1 --2- 172.21.15.45:0/1807528319 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55876edaad90 0x55876edad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:33.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.043+0000 7fce9185f700 1 --2- 172.21.15.45:0/1807528319 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55876edaad90 0x55876edad180 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46f9b99ace2b058f server_cookie=e87abad18116f5a9 in_seq=0 out_seq=0 2024-06-06T04:31:33.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.043+0000 7fce757fa700 10 client.15318 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:33.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.045+0000 7fce757fa700 1 -- 172.21.15.45:0/1807528319 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55876ec7e310 con 0x55876edaad90 2024-06-06T04:31:33.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.045+0000 7fce757fa700 10 client.15318 handle_command_reply: tid=0 2024-06-06T04:31:33.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.045+0000 7fce9e346b80 1 client.15318 shutdown 2024-06-06T04:31:33.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.045+0000 7fce9e346b80 1 -- 172.21.15.45:0/1807528319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2afe0 msgr2=0x55876ed349d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:33.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.045+0000 7fce9e346b80 1 --2- 172.21.15.45:0/1807528319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2afe0 0x55876ed349d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fce840078c0 tx=0x7fce840099e0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.045+0000 7fce9e346b80 1 -- 172.21.15.45:0/1807528319 shutdown_connections 2024-06-06T04:31:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.045+0000 7fce9e346b80 1 --2- 172.21.15.45:0/1807528319 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55876edaad90 0x55876edad180 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.045+0000 7fce9e346b80 1 --2- 172.21.15.45:0/1807528319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55876ed2afe0 0x55876ed349d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.045+0000 7fce9e346b80 1 --2- 172.21.15.45:0/1807528319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55876ed24fa0 0x55876ed33f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.045+0000 7fce9e346b80 1 --2- 172.21.15.45:0/1807528319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55876ed2a6c0 0x55876ed34450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.045+0000 7fce757fa700 0 client.15318 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.045+0000 7fce9e346b80 1 -- 172.21.15.45:0/1807528319 >> 172.21.15.45:0/1807528319 conn(0x55876ec4fd40 msgr2=0x55876ed2f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.045+0000 7fce9e346b80 1 -- 172.21.15.45:0/1807528319 shutdown_connections 2024-06-06T04:31:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.045+0000 7fce9e346b80 1 -- 172.21.15.45:0/1807528319 wait complete. 2024-06-06T04:31:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.045+0000 7fce9e346b80 20 client.15318 trim_cache size 0 max 16384 2024-06-06T04:31:33.050 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:31:33.051 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-06-06T04:31:33.051 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "ccb2508b-ba39-4879-8225-19d94bd85a42", 2024-06-06T04:31:33.051 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-06-06T04:31:33.051 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:31:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.046+0000 7fce93ac3700 1 -- 172.21.15.45:0/3505986344 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fce780a2d30 msgr2=0x7fce780a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.046+0000 7fce93ac3700 1 --2- 172.21.15.45:0/3505986344 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fce780a2d30 0x7fce780a51f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fce88006fd0 tx=0x7fce88008040 comp rx=0 tx=0).stop 2024-06-06T04:31:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.046+0000 7fce93ac3700 1 -- 172.21.15.45:0/3505986344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8c1027f0 msgr2=0x7fce8c19b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.046+0000 7fce93ac3700 1 --2- 172.21.15.45:0/3505986344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8c1027f0 0x7fce8c19b2e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00c970 tx=0x7fce7c00cc80 comp rx=0 tx=0).stop 2024-06-06T04:31:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.046+0000 7fce93ac3700 1 -- 172.21.15.45:0/3505986344 shutdown_connections 2024-06-06T04:31:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.046+0000 7fce93ac3700 1 --2- 172.21.15.45:0/3505986344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce8c10ef70 0x7fce8c19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.046+0000 7fce93ac3700 1 --2- 172.21.15.45:0/3505986344 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fce780a2d30 0x7fce780a51f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.046+0000 7fce93ac3700 1 --2- 172.21.15.45:0/3505986344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8c101ed0 0x7fce8c19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.046+0000 7fce93ac3700 1 --2- 172.21.15.45:0/3505986344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8c1027f0 0x7fce8c19b2e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.046+0000 7fce93ac3700 1 -- 172.21.15.45:0/3505986344 >> 172.21.15.45:0/3505986344 conn(0x7fce8c077400 msgr2=0x7fce8c1103c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.046+0000 7fce93ac3700 1 -- 172.21.15.45:0/3505986344 shutdown_connections 2024-06-06T04:31:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:33.046+0000 7fce93ac3700 1 -- 172.21.15.45:0/3505986344 wait complete. 2024-06-06T04:31:33.906 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T04:31:34.197 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:31:34.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:33 smithi045 ceph-mon[32299]: pgmap v434: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-06-06T04:31:34.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:33 smithi045 ceph-mon[32299]: scrub queued for path: / 2024-06-06T04:31:34.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:33 smithi045 ceph-mon[32299]: scrub summary: idle+waiting paths [/] 2024-06-06T04:31:34.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:33 smithi045 ceph-mon[32299]: scrub summary: active paths [/] 2024-06-06T04:31:34.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:33 smithi045 ceph-mon[32299]: scrub summary: idle+waiting paths [/] 2024-06-06T04:31:34.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:33 smithi086 ceph-mon[36053]: pgmap v434: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-06-06T04:31:34.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:33 smithi086 ceph-mon[36053]: scrub queued for path: / 2024-06-06T04:31:34.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:33 smithi086 ceph-mon[36053]: scrub summary: idle+waiting paths [/] 2024-06-06T04:31:34.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:33 smithi086 ceph-mon[36053]: scrub summary: active paths [/] 2024-06-06T04:31:34.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:33 smithi086 ceph-mon[36053]: scrub summary: idle+waiting paths [/] 2024-06-06T04:31:34.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:33 smithi154 ceph-mon[36428]: pgmap v434: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-06-06T04:31:34.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:33 smithi154 ceph-mon[36428]: scrub queued for path: / 2024-06-06T04:31:34.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:33 smithi154 ceph-mon[36428]: scrub summary: idle+waiting paths [/] 2024-06-06T04:31:34.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:33 smithi154 ceph-mon[36428]: scrub summary: active paths [/] 2024-06-06T04:31:34.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:33 smithi154 ceph-mon[36428]: scrub summary: idle+waiting paths [/] 2024-06-06T04:31:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.871+0000 7f9626a0c700 1 -- 172.21.15.45:0/359507591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9620101d40 msgr2=0x7f962010e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.871+0000 7f9626a0c700 1 --2- 172.21.15.45:0/359507591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9620101d40 0x7f962010e230 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f961800b0a0 tx=0x7f961800b3b0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.871+0000 7f9626a0c700 1 -- 172.21.15.45:0/359507591 shutdown_connections 2024-06-06T04:31:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.871+0000 7f9626a0c700 1 --2- 172.21.15.45:0/359507591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9620101d40 0x7f962010e230 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.871+0000 7f9626a0c700 1 --2- 172.21.15.45:0/359507591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96201073f0 0x7f96201077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.871+0000 7f9626a0c700 1 --2- 172.21.15.45:0/359507591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9620101380 0x7f9620101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.871+0000 7f9626a0c700 1 -- 172.21.15.45:0/359507591 >> 172.21.15.45:0/359507591 conn(0x7f9620078300 msgr2=0x7f9620078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.871+0000 7f9626a0c700 1 -- 172.21.15.45:0/359507591 shutdown_connections 2024-06-06T04:31:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.871+0000 7f9626a0c700 1 -- 172.21.15.45:0/359507591 wait complete. 2024-06-06T04:31:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.872+0000 7f9626a0c700 1 Processor -- start 2024-06-06T04:31:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.872+0000 7f9626a0c700 1 -- start start 2024-06-06T04:31:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.872+0000 7f9626a0c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9620101380 0x7f962019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.872+0000 7f9626a0c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9620101d40 0x7f962019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.872+0000 7f9626a0c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96201073f0 0x7f9620198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.872+0000 7f9626a0c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9620111530 con 0x7f96201073f0 2024-06-06T04:31:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.872+0000 7f9626a0c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96201113b0 con 0x7f9620101d40 2024-06-06T04:31:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.872+0000 7f9626a0c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f96201116b0 con 0x7f9620101380 2024-06-06T04:31:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.872+0000 7f9624fa9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96201073f0 0x7f9620198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.872+0000 7f9624fa9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96201073f0 0x7f9620198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33180/0 (socket says 172.21.15.45:33180) 2024-06-06T04:31:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.872+0000 7f9624fa9700 1 -- 172.21.15.45:0/2190947769 learned_addr learned my addr 172.21.15.45:0/2190947769 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.872+0000 7f961ffff700 1 --2- 172.21.15.45:0/2190947769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9620101380 0x7f962019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.872+0000 7f9617fff700 1 --2- 172.21.15.45:0/2190947769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9620101d40 0x7f962019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.873+0000 7f9624fa9700 1 -- 172.21.15.45:0/2190947769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9620101380 msgr2=0x7f962019e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.873+0000 7f9624fa9700 1 --2- 172.21.15.45:0/2190947769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9620101380 0x7f962019e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.873+0000 7f9624fa9700 1 -- 172.21.15.45:0/2190947769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9620101d40 msgr2=0x7f962019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.873+0000 7f9624fa9700 1 --2- 172.21.15.45:0/2190947769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9620101d40 0x7f962019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.873+0000 7f9624fa9700 1 -- 172.21.15.45:0/2190947769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96201990d0 con 0x7f96201073f0 2024-06-06T04:31:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.873+0000 7f9624fa9700 1 --2- 172.21.15.45:0/2190947769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96201073f0 0x7f9620198970 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f961800b830 tx=0x7f9618004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.873+0000 7f961dffb700 1 -- 172.21.15.45:0/2190947769 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618021070 con 0x7f96201073f0 2024-06-06T04:31:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.873+0000 7f961dffb700 1 -- 172.21.15.45:0/2190947769 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9618007e30 con 0x7f96201073f0 2024-06-06T04:31:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.873+0000 7f9626a0c700 1 -- 172.21.15.45:0/2190947769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9620199360 con 0x7f96201073f0 2024-06-06T04:31:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.873+0000 7f961dffb700 1 -- 172.21.15.45:0/2190947769 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618004700 con 0x7f96201073f0 2024-06-06T04:31:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.873+0000 7f9626a0c700 1 -- 172.21.15.45:0/2190947769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96201a5680 con 0x7f96201073f0 2024-06-06T04:31:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.876+0000 7f961dffb700 1 -- 172.21.15.45:0/2190947769 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f9618007980 con 0x7f96201073f0 2024-06-06T04:31:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.877+0000 7f961dffb700 1 --2- 172.21.15.45:0/2190947769 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f960c0a2cb0 0x7f960c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.877+0000 7f961ffff700 1 --2- 172.21.15.45:0/2190947769 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f960c0a2cb0 0x7f960c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.877+0000 7f961dffb700 1 -- 172.21.15.45:0/2190947769 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f961800f020 con 0x7f96201073f0 2024-06-06T04:31:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.877+0000 7f961ffff700 1 --2- 172.21.15.45:0/2190947769 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f960c0a2cb0 0x7f960c0a5170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9620199ab0 tx=0x7f961000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.877+0000 7f963128fb80 1 Processor -- start 2024-06-06T04:31:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.877+0000 7f963128fb80 1 -- start start 2024-06-06T04:31:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f963128fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa07a70 0x55722aa07e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f963128fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa08390 0x55722aa01df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f963128fb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa02330 0x55722aa02750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f963128fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55722a94f810 con 0x55722aa07a70 2024-06-06T04:31:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f963128fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55722a8f7720 con 0x55722aa08390 2024-06-06T04:31:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f963128fb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55722a94b7d0 con 0x55722aa02330 2024-06-06T04:31:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f961ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa07a70 0x55722aa07e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f961ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa07a70 0x55722aa07e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33186/0 (socket says 172.21.15.45:33186) 2024-06-06T04:31:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f961ffff700 1 -- 172.21.15.45:0/2392213466 learned_addr learned my addr 172.21.15.45:0/2392213466 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f9617fff700 1 --2- 172.21.15.45:0/2392213466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa08390 0x55722aa01df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f9624fa9700 1 --2- 172.21.15.45:0/2392213466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa02330 0x55722aa02750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f961ffff700 1 -- 172.21.15.45:0/2392213466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa02330 msgr2=0x55722aa02750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f961ffff700 1 --2- 172.21.15.45:0/2392213466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa02330 0x55722aa02750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f961ffff700 1 -- 172.21.15.45:0/2392213466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa08390 msgr2=0x55722aa01df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f961ffff700 1 --2- 172.21.15.45:0/2392213466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa08390 0x55722aa01df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f961ffff700 1 -- 172.21.15.45:0/2392213466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55722a8edf50 con 0x55722aa07a70 2024-06-06T04:31:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.878+0000 7f961ffff700 1 --2- 172.21.15.45:0/2392213466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa07a70 0x55722aa07e50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9610005e10 tx=0x7f9610013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ab01386312150a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.879+0000 7f96167fc700 1 -- 172.21.15.45:0/2392213466 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96100118d0 con 0x55722aa07a70 2024-06-06T04:31:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.879+0000 7f96167fc700 1 -- 172.21.15.45:0/2392213466 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f961001e420 con 0x55722aa07a70 2024-06-06T04:31:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.879+0000 7f96167fc700 1 -- 172.21.15.45:0/2392213466 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961001c610 con 0x55722aa07a70 2024-06-06T04:31:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.879+0000 7f963128fb80 1 -- 172.21.15.45:0/2392213466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa07a70 msgr2=0x55722aa07e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.879+0000 7f963128fb80 1 --2- 172.21.15.45:0/2392213466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa07a70 0x55722aa07e50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9610005e10 tx=0x7f9610013000 comp rx=0 tx=0).stop 2024-06-06T04:31:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.879+0000 7f963128fb80 1 -- 172.21.15.45:0/2392213466 shutdown_connections 2024-06-06T04:31:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.879+0000 7f963128fb80 1 --2- 172.21.15.45:0/2392213466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa02330 0x55722aa02750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.879+0000 7f963128fb80 1 --2- 172.21.15.45:0/2392213466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa07a70 0x55722aa07e50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.879+0000 7f963128fb80 1 --2- 172.21.15.45:0/2392213466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa08390 0x55722aa01df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.879+0000 7f963128fb80 1 -- 172.21.15.45:0/2392213466 >> 172.21.15.45:0/2392213466 conn(0x55722a928cc0 msgr2=0x55722a9f8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.880+0000 7f963128fb80 1 -- 172.21.15.45:0/2392213466 shutdown_connections 2024-06-06T04:31:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.880+0000 7f963128fb80 1 -- 172.21.15.45:0/2392213466 wait complete. 2024-06-06T04:31:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.880+0000 7f963128fb80 1 -- start start 2024-06-06T04:31:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.880+0000 7f963128fb80 1 Processor -- start 2024-06-06T04:31:35.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f963128fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722aa0e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f963128fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa07a70 0x55722aa0b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f963128fb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa08390 0x55722aa0b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f963128fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55722aa0edd0 con 0x55722aa02330 2024-06-06T04:31:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f963128fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55722a8efbf0 con 0x55722aa07a70 2024-06-06T04:31:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f963128fb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55722aa0ef50 con 0x55722aa08390 2024-06-06T04:31:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f963128fb80 10 client.0 _ll_register_callbacks cb 0x55722a8e9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:31:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f961ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722aa0e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f961ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722aa0e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33198/0 (socket says 172.21.15.45:33198) 2024-06-06T04:31:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f961ffff700 1 -- 172.21.15.45:0/3011088475 learned_addr learned my addr 172.21.15.45:0/3011088475 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f9617fff700 1 --2- 172.21.15.45:0/3011088475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa07a70 0x55722aa0b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f961ffff700 1 -- 172.21.15.45:0/3011088475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa08390 msgr2=0x55722aa0b960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T04:31:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f961ffff700 1 --2- 172.21.15.45:0/3011088475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa08390 0x55722aa0b960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f961ffff700 1 -- 172.21.15.45:0/3011088475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa07a70 msgr2=0x55722aa0b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f961ffff700 1 --2- 172.21.15.45:0/3011088475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa07a70 0x55722aa0b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f961ffff700 1 -- 172.21.15.45:0/3011088475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55722a89e8d0 con 0x55722aa02330 2024-06-06T04:31:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f9617fff700 1 --2- 172.21.15.45:0/3011088475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa07a70 0x55722aa0b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:31:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f961ffff700 1 --2- 172.21.15.45:0/3011088475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722aa0e120 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f961000b4a0 tx=0x7f9610005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fc43adf1910648 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.881+0000 7f95ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:31:35.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.882+0000 7f95ff7fe700 1 -- 172.21.15.45:0/3011088475 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610035060 con 0x55722aa02330 2024-06-06T04:31:35.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.882+0000 7f95ff7fe700 1 -- 172.21.15.45:0/3011088475 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f961001ee00 con 0x55722aa02330 2024-06-06T04:31:35.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.882+0000 7f963128fb80 1 -- 172.21.15.45:0/3011088475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f961001e420 con 0x55722aa02330 2024-06-06T04:31:35.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.882+0000 7f95ff7fe700 1 -- 172.21.15.45:0/3011088475 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f961003c050 con 0x55722aa02330 2024-06-06T04:31:35.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.882+0000 7f963128fb80 10 client.15330 fetch_fsmap learned FSMap version 27 2024-06-06T04:31:35.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.882+0000 7f963128fb80 1 -- 172.21.15.45:0/3011088475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55722aa19700 con 0x55722aa02330 2024-06-06T04:31:35.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.882+0000 7f95ff7fe700 1 -- 172.21.15.45:0/3011088475 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f961001e420 con 0x55722aa02330 2024-06-06T04:31:35.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.882+0000 7f95ff7fe700 1 -- 172.21.15.45:0/3011088475 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9610026a00 con 0x55722aa02330 2024-06-06T04:31:35.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.882+0000 7f963128fb80 10 client.15330 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:31:35.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.882+0000 7f963128fb80 10 client.15330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:31:35.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.882+0000 7f963128fb80 20 client.15330 populate_metadata read hostname 'smithi045' 2024-06-06T04:31:35.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.882+0000 7f963128fb80 1 --2- 172.21.15.45:0/3011088475 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55722aa1bc80 0x55722aa1e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.882+0000 7f963128fb80 4 client.15330 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:31:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.882+0000 7f963128fb80 1 -- 172.21.15.45:0/3011088475 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55722a95cd10 con 0x55722aa1bc80 2024-06-06T04:31:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.883+0000 7f9617fff700 1 --2- 172.21.15.45:0/3011088475 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55722aa1bc80 0x55722aa1e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.883+0000 7f9617fff700 1 --2- 172.21.15.45:0/3011088475 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55722aa1bc80 0x55722aa1e070 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=310bb3d3aa67eb3e server_cookie=e4eb79c0ab627018 in_seq=0 out_seq=0 2024-06-06T04:31:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.883+0000 7f95ff7fe700 10 client.15330 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.884+0000 7f95ff7fe700 1 -- 172.21.15.45:0/3011088475 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55722a95cd10 con 0x55722aa1bc80 2024-06-06T04:31:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.884+0000 7f95ff7fe700 10 client.15330 handle_command_reply: tid=0 2024-06-06T04:31:35.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.884+0000 7f963128fb80 1 client.15330 shutdown 2024-06-06T04:31:35.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.884+0000 7f963128fb80 1 -- 172.21.15.45:0/3011088475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 msgr2=0x55722aa0e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:35.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.884+0000 7f963128fb80 1 --2- 172.21.15.45:0/3011088475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722aa0e120 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f961000b4a0 tx=0x7f9610005720 comp rx=0 tx=0).stop 2024-06-06T04:31:35.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.884+0000 7f963128fb80 1 -- 172.21.15.45:0/3011088475 shutdown_connections 2024-06-06T04:31:35.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.884+0000 7f963128fb80 1 --2- 172.21.15.45:0/3011088475 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55722aa1bc80 0x55722aa1e070 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.884+0000 7f963128fb80 1 --2- 172.21.15.45:0/3011088475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa08390 0x55722aa0b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.884+0000 7f963128fb80 1 --2- 172.21.15.45:0/3011088475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722aa0e120 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.884+0000 7f963128fb80 1 --2- 172.21.15.45:0/3011088475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa07a70 0x55722aa0b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.884+0000 7f963128fb80 1 -- 172.21.15.45:0/3011088475 >> 172.21.15.45:0/3011088475 conn(0x55722a928cc0 msgr2=0x55722a9fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:35.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.884+0000 7f95ff7fe700 0 client.15330 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:35.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.884+0000 7f963128fb80 1 -- 172.21.15.45:0/3011088475 shutdown_connections 2024-06-06T04:31:35.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.884+0000 7f963128fb80 1 -- 172.21.15.45:0/3011088475 wait complete. 2024-06-06T04:31:35.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.884+0000 7f963128fb80 20 client.15330 trim_cache size 0 max 16384 2024-06-06T04:31:35.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.905+0000 7f963128fb80 1 Processor -- start 2024-06-06T04:31:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.905+0000 7f963128fb80 1 -- start start 2024-06-06T04:31:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.905+0000 7f963128fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722a9f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.905+0000 7f963128fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa07a70 0x55722aa09000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.905+0000 7f963128fb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa08390 0x55722a9fd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.905+0000 7f963128fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f96201ae960 con 0x55722aa02330 2024-06-06T04:31:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.905+0000 7f963128fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96201ac1f0 con 0x55722aa07a70 2024-06-06T04:31:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.905+0000 7f963128fb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f96201aa040 con 0x55722aa08390 2024-06-06T04:31:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.905+0000 7f961ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722a9f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.905+0000 7f961ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722a9f89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33208/0 (socket says 172.21.15.45:33208) 2024-06-06T04:31:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.905+0000 7f961ffff700 1 -- 172.21.15.45:0/1619359505 learned_addr learned my addr 172.21.15.45:0/1619359505 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.906+0000 7f9617fff700 1 --2- 172.21.15.45:0/1619359505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa07a70 0x55722aa09000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.906+0000 7f9624fa9700 1 --2- 172.21.15.45:0/1619359505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa08390 0x55722a9fd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.906+0000 7f961ffff700 1 -- 172.21.15.45:0/1619359505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa08390 msgr2=0x55722a9fd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.906+0000 7f961ffff700 1 --2- 172.21.15.45:0/1619359505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa08390 0x55722a9fd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.906+0000 7f961ffff700 1 -- 172.21.15.45:0/1619359505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa07a70 msgr2=0x55722aa09000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.906+0000 7f961ffff700 1 --2- 172.21.15.45:0/1619359505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa07a70 0x55722aa09000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.906+0000 7f961ffff700 1 -- 172.21.15.45:0/1619359505 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55722aa099b0 con 0x55722aa02330 2024-06-06T04:31:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.906+0000 7f961ffff700 1 --2- 172.21.15.45:0/1619359505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722a9f89f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f961001d5c0 tx=0x7f961001d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299de22f52621c92 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.906+0000 7f9615ffb700 1 -- 172.21.15.45:0/1619359505 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610035070 con 0x55722aa02330 2024-06-06T04:31:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.906+0000 7f9615ffb700 1 -- 172.21.15.45:0/1619359505 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f961001e420 con 0x55722aa02330 2024-06-06T04:31:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.906+0000 7f9615ffb700 1 -- 172.21.15.45:0/1619359505 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961001db90 con 0x55722aa02330 2024-06-06T04:31:35.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.907+0000 7f963128fb80 1 -- 172.21.15.45:0/1619359505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 msgr2=0x55722a9f89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:35.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.907+0000 7f963128fb80 1 --2- 172.21.15.45:0/1619359505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722a9f89f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f961001d5c0 tx=0x7f961001d5f0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.907+0000 7f963128fb80 1 -- 172.21.15.45:0/1619359505 shutdown_connections 2024-06-06T04:31:35.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.907+0000 7f963128fb80 1 --2- 172.21.15.45:0/1619359505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa08390 0x55722a9fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.907+0000 7f963128fb80 1 --2- 172.21.15.45:0/1619359505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722a9f89f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.907+0000 7f963128fb80 1 --2- 172.21.15.45:0/1619359505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa07a70 0x55722aa09000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.907+0000 7f963128fb80 1 -- 172.21.15.45:0/1619359505 >> 172.21.15.45:0/1619359505 conn(0x55722a928cc0 msgr2=0x55722a9fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:35.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.907+0000 7f963128fb80 1 -- 172.21.15.45:0/1619359505 shutdown_connections 2024-06-06T04:31:35.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.907+0000 7f963128fb80 1 -- 172.21.15.45:0/1619359505 wait complete. 2024-06-06T04:31:35.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.907+0000 7f963128fb80 1 -- start start 2024-06-06T04:31:35.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.907+0000 7f963128fb80 1 Processor -- start 2024-06-06T04:31:35.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f963128fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722aa145a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f963128fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa07a70 0x55722aa14ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f963128fb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa08390 0x55722aa22e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f963128fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f96201aa6b0 con 0x55722aa02330 2024-06-06T04:31:35.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f963128fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96201abb20 con 0x55722aa07a70 2024-06-06T04:31:35.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f963128fb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f96201aad80 con 0x55722aa08390 2024-06-06T04:31:35.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f963128fb80 10 client.0 _ll_register_callbacks cb 0x55722a8e9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:31:35.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f961ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722aa145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f961ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722aa145a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33216/0 (socket says 172.21.15.45:33216) 2024-06-06T04:31:35.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f961ffff700 1 -- 172.21.15.45:0/2702175574 learned_addr learned my addr 172.21.15.45:0/2702175574 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:35.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f961ffff700 1 -- 172.21.15.45:0/2702175574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa08390 msgr2=0x55722aa22e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:35.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f9624fa9700 1 --2- 172.21.15.45:0/2702175574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa08390 0x55722aa22e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f9617fff700 1 --2- 172.21.15.45:0/2702175574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa07a70 0x55722aa14ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f961ffff700 1 --2- 172.21.15.45:0/2702175574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa08390 0x55722aa22e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f961ffff700 1 -- 172.21.15.45:0/2702175574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa07a70 msgr2=0x55722aa14ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:35.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f961ffff700 1 --2- 172.21.15.45:0/2702175574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa07a70 0x55722aa14ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f961ffff700 1 -- 172.21.15.45:0/2702175574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55722aa23610 con 0x55722aa02330 2024-06-06T04:31:35.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f9624fa9700 1 --2- 172.21.15.45:0/2702175574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa08390 0x55722aa22e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:31:35.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f961ffff700 1 --2- 172.21.15.45:0/2702175574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722aa145a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f96100119b0 tx=0x7f9610011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c32597eca5bf56a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:35.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f95ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:31:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f95ff7fe700 1 -- 172.21.15.45:0/2702175574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610044070 con 0x55722aa02330 2024-06-06T04:31:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f95ff7fe700 1 -- 172.21.15.45:0/2702175574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f9610005250 con 0x55722aa02330 2024-06-06T04:31:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.908+0000 7f963128fb80 1 -- 172.21.15.45:0/2702175574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f961001e420 con 0x55722aa02330 2024-06-06T04:31:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.909+0000 7f95ff7fe700 1 -- 172.21.15.45:0/2702175574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9610050050 con 0x55722aa02330 2024-06-06T04:31:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.909+0000 7f963128fb80 10 client.15336 fetch_fsmap learned FSMap version 27 2024-06-06T04:31:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.909+0000 7f963128fb80 1 -- 172.21.15.45:0/2702175574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55722aa862e0 con 0x55722aa02330 2024-06-06T04:31:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.909+0000 7f95ff7fe700 1 -- 172.21.15.45:0/2702175574 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f961001e420 con 0x55722aa02330 2024-06-06T04:31:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.909+0000 7f95ff7fe700 1 -- 172.21.15.45:0/2702175574 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9610004710 con 0x55722aa02330 2024-06-06T04:31:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.909+0000 7f963128fb80 10 client.15336 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:31:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.909+0000 7f963128fb80 10 client.15336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:31:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.909+0000 7f963128fb80 20 client.15336 populate_metadata read hostname 'smithi045' 2024-06-06T04:31:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.909+0000 7f963128fb80 1 --2- 172.21.15.45:0/2702175574 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55722aa86850 0x55722aa88c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.909+0000 7f963128fb80 4 client.15336 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T04:31:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.909+0000 7f963128fb80 1 -- 172.21.15.45:0/2702175574 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55722a957290 con 0x55722aa86850 2024-06-06T04:31:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.909+0000 7f9617fff700 1 --2- 172.21.15.45:0/2702175574 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55722aa86850 0x55722aa88c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f9617fff700 1 --2- 172.21.15.45:0/2702175574 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55722aa86850 0x55722aa88c70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2014c7e9e84eea33 server_cookie=baee59ef00ee1905 in_seq=0 out_seq=0 2024-06-06T04:31:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f95ff7fe700 10 client.15336 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:35.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f95ff7fe700 1 -- 172.21.15.45:0/2702175574 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55722a957290 con 0x55722aa86850 2024-06-06T04:31:35.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f95ff7fe700 10 client.15336 handle_command_reply: tid=0 2024-06-06T04:31:35.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f963128fb80 1 client.15336 shutdown 2024-06-06T04:31:35.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f963128fb80 1 -- 172.21.15.45:0/2702175574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 msgr2=0x55722aa145a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:35.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f963128fb80 1 --2- 172.21.15.45:0/2702175574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722aa145a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f96100119b0 tx=0x7f9610011a90 comp rx=0 tx=0).stop 2024-06-06T04:31:35.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f963128fb80 1 -- 172.21.15.45:0/2702175574 shutdown_connections 2024-06-06T04:31:35.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f963128fb80 1 --2- 172.21.15.45:0/2702175574 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55722aa86850 0x55722aa88c70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f963128fb80 1 --2- 172.21.15.45:0/2702175574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55722aa08390 0x55722aa22e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f963128fb80 1 --2- 172.21.15.45:0/2702175574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722aa02330 0x55722aa145a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f963128fb80 1 --2- 172.21.15.45:0/2702175574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55722aa07a70 0x55722aa14ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f963128fb80 1 -- 172.21.15.45:0/2702175574 >> 172.21.15.45:0/2702175574 conn(0x55722a928cc0 msgr2=0x55722a9fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:35.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f95ff7fe700 0 client.15336 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:35.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f963128fb80 1 -- 172.21.15.45:0/2702175574 shutdown_connections 2024-06-06T04:31:35.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f963128fb80 1 -- 172.21.15.45:0/2702175574 wait complete. 2024-06-06T04:31:35.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.910+0000 7f963128fb80 20 client.15336 trim_cache size 0 max 16384 2024-06-06T04:31:35.922 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:31:35.922 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-06T04:31:35.922 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-06-06T04:31:35.922 INFO:teuthology.orchestra.run.smithi045.stdout: "ccb2508b-ba39-4879-8225-19d94bd85a42": { 2024-06-06T04:31:35.922 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-06-06T04:31:35.922 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "ccb2508b-ba39-4879-8225-19d94bd85a42", 2024-06-06T04:31:35.922 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-06-06T04:31:35.923 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:31:35.923 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:31:35.923 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:31:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.911+0000 7f9626a0c700 1 -- 172.21.15.45:0/2190947769 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f960c0a2cb0 msgr2=0x7f960c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.911+0000 7f9626a0c700 1 --2- 172.21.15.45:0/2190947769 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f960c0a2cb0 0x7f960c0a5170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9620199ab0 tx=0x7f961000b3c0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.911+0000 7f9626a0c700 1 -- 172.21.15.45:0/2190947769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96201073f0 msgr2=0x7f9620198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.911+0000 7f9626a0c700 1 --2- 172.21.15.45:0/2190947769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96201073f0 0x7f9620198970 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f961800b830 tx=0x7f9618004ce0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.911+0000 7f9626a0c700 1 -- 172.21.15.45:0/2190947769 shutdown_connections 2024-06-06T04:31:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.911+0000 7f9626a0c700 1 --2- 172.21.15.45:0/2190947769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96201073f0 0x7f9620198970 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.911+0000 7f9626a0c700 1 --2- 172.21.15.45:0/2190947769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9620101380 0x7f962019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.911+0000 7f9626a0c700 1 --2- 172.21.15.45:0/2190947769 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f960c0a2cb0 0x7f960c0a5170 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.911+0000 7f9626a0c700 1 --2- 172.21.15.45:0/2190947769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9620101d40 0x7f962019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.911+0000 7f9626a0c700 1 -- 172.21.15.45:0/2190947769 >> 172.21.15.45:0/2190947769 conn(0x7f9620078300 msgr2=0x7f9620100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.911+0000 7f9626a0c700 1 -- 172.21.15.45:0/2190947769 shutdown_connections 2024-06-06T04:31:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:31:35.911+0000 7f9626a0c700 1 -- 172.21.15.45:0/2190947769 wait complete. 2024-06-06T04:31:36.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:35 smithi045 ceph-mon[32299]: pgmap v435: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-06T04:31:36.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:35 smithi086 ceph-mon[36053]: pgmap v435: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-06T04:31:36.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:35 smithi154 ceph-mon[36428]: pgmap v435: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-06T04:31:36.710 INFO:tasks.cephfs.filesystem:scrub status for tag:ccb2508b-ba39-4879-8225-19d94bd85a42 - {'path': '/', 'tag': 'ccb2508b-ba39-4879-8225-19d94bd85a42', 'options': 'recursive,force'} 2024-06-06T04:31:38.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:37 smithi086 ceph-mon[36053]: pgmap v436: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-06-06T04:31:38.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:37 smithi154 ceph-mon[36428]: pgmap v436: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-06-06T04:31:38.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:37 smithi045 ceph-mon[32299]: pgmap v436: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-06-06T04:31:40.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:39 smithi086 ceph-mon[36053]: pgmap v437: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-06T04:31:40.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:39 smithi154 ceph-mon[36428]: pgmap v437: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-06T04:31:40.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:39 smithi045 ceph-mon[32299]: pgmap v437: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-06T04:31:42.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:41 smithi086 ceph-mon[36053]: pgmap v438: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-06-06T04:31:42.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:41 smithi154 ceph-mon[36428]: pgmap v438: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-06-06T04:31:42.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:41 smithi045 ceph-mon[32299]: pgmap v438: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-06-06T04:31:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.030+0000 7f2a5fce7640 1 -- 172.21.15.45:0/2015910933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a58104e60 msgr2=0x7f2a58107250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.030+0000 7f2a5fce7640 1 --2- 172.21.15.45:0/2015910933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a58104e60 0x7f2a58107250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2a48009900 tx=0x7f2a4802a410 comp rx=0 tx=0).stop 2024-06-06T04:31:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.030+0000 7f2a5fce7640 1 -- 172.21.15.45:0/2015910933 shutdown_connections 2024-06-06T04:31:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.030+0000 7f2a5fce7640 1 --2- 172.21.15.45:0/2015910933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a58104e60 0x7f2a58107250 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.030+0000 7f2a5fce7640 1 --2- 172.21.15.45:0/2015910933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a58107790 0x7f2a58109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.030+0000 7f2a5fce7640 1 --2- 172.21.15.45:0/2015910933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a58102530 0x7f2a58104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.030+0000 7f2a5fce7640 1 -- 172.21.15.45:0/2015910933 >> 172.21.15.45:0/2015910933 conn(0x7f2a580fbf20 msgr2=0x7f2a580fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.030+0000 7f2a5fce7640 1 -- 172.21.15.45:0/2015910933 shutdown_connections 2024-06-06T04:31:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.030+0000 7f2a5fce7640 1 -- 172.21.15.45:0/2015910933 wait complete. 2024-06-06T04:31:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5fce7640 1 Processor -- start 2024-06-06T04:31:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5fce7640 1 -- start start 2024-06-06T04:31:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5fce7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a58102530 0x7f2a58197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5fce7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a58104e60 0x7f2a581980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5fce7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a58107790 0x7f2a5819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5fce7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5810d560 con 0x7f2a58102530 2024-06-06T04:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5fce7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5810d3e0 con 0x7f2a58104e60 2024-06-06T04:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5fce7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5810d6e0 con 0x7f2a58107790 2024-06-06T04:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5da83640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a58102530 0x7f2a58197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5da83640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a58102530 0x7f2a58197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37564/0 (socket says 172.21.15.45:37564) 2024-06-06T04:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5da83640 1 -- 172.21.15.45:0/3151390456 learned_addr learned my addr 172.21.15.45:0/3151390456 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5d282640 1 --2- 172.21.15.45:0/3151390456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a58104e60 0x7f2a581980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5e284640 1 --2- 172.21.15.45:0/3151390456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a58107790 0x7f2a5819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5da83640 1 -- 172.21.15.45:0/3151390456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a58107790 msgr2=0x7f2a5819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5da83640 1 --2- 172.21.15.45:0/3151390456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a58107790 0x7f2a5819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5da83640 1 -- 172.21.15.45:0/3151390456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a58104e60 msgr2=0x7f2a581980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5da83640 1 --2- 172.21.15.45:0/3151390456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a58104e60 0x7f2a581980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5da83640 1 -- 172.21.15.45:0/3151390456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5819f790 con 0x7f2a58102530 2024-06-06T04:31:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.031+0000 7f2a5e284640 1 --2- 172.21.15.45:0/3151390456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a58107790 0x7f2a5819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:31:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.032+0000 7f2a5da83640 1 --2- 172.21.15.45:0/3151390456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a58102530 0x7f2a58197b70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400ea60 tx=0x7f2a5400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.032+0000 7f2a4effd640 1 -- 172.21.15.45:0/3151390456 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400cde0 con 0x7f2a58102530 2024-06-06T04:31:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.032+0000 7f2a5fce7640 1 -- 172.21.15.45:0/3151390456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5819fa80 con 0x7f2a58102530 2024-06-06T04:31:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.032+0000 7f2a4effd640 1 -- 172.21.15.45:0/3151390456 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a540035f0 con 0x7f2a58102530 2024-06-06T04:31:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.032+0000 7f2a4effd640 1 -- 172.21.15.45:0/3151390456 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54010690 con 0x7f2a58102530 2024-06-06T04:31:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.032+0000 7f2a5fce7640 1 -- 172.21.15.45:0/3151390456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5819fff0 con 0x7f2a58102530 2024-06-06T04:31:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.034+0000 7f2a4effd640 1 -- 172.21.15.45:0/3151390456 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f2a540037a0 con 0x7f2a58102530 2024-06-06T04:31:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.035+0000 7f2a629f0740 1 Processor -- start 2024-06-06T04:31:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.035+0000 7f2a629f0740 1 -- start start 2024-06-06T04:31:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.035+0000 7f2a629f0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b5522f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.035+0000 7f2a629f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 0x562b55234910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.035+0000 7f2a629f0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b55237410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.035+0000 7f2a629f0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562b5518e4f0 con 0x562b55234f80 2024-06-06T04:31:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.035+0000 7f2a629f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b5512c870 con 0x562b5522fe10 2024-06-06T04:31:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.035+0000 7f2a629f0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562b5518a4b0 con 0x562b5522f420 2024-06-06T04:31:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.035+0000 7f2a5d282640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b55237410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.035+0000 7f2a5d282640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b55237410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37580/0 (socket says 172.21.15.45:37580) 2024-06-06T04:31:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.035+0000 7f2a5d282640 1 -- 172.21.15.45:0/248244073 learned_addr learned my addr 172.21.15.45:0/248244073 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.035+0000 7f2a4effd640 1 --2- 172.21.15.45:0/3151390456 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2a440a2cd0 0x7f2a440a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.036+0000 7f2a5e284640 1 --2- 172.21.15.45:0/248244073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b5522f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.036+0000 7f2a5da83640 1 --2- 172.21.15.45:0/248244073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 0x562b55234910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.036+0000 7f2a5e284640 1 --2- 172.21.15.45:0/3151390456 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2a440a2cd0 0x7f2a440a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.036+0000 7f2a5d282640 1 -- 172.21.15.45:0/248244073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 msgr2=0x562b5522f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.036+0000 7f2a5d282640 1 --2- 172.21.15.45:0/248244073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b5522f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.036+0000 7f2a5d282640 1 -- 172.21.15.45:0/248244073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 msgr2=0x562b55234910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.036+0000 7f2a5d282640 1 --2- 172.21.15.45:0/248244073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 0x562b55234910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.036+0000 7f2a5d282640 1 -- 172.21.15.45:0/248244073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b551489b0 con 0x562b55234f80 2024-06-06T04:31:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.036+0000 7f2a5da83640 1 --2- 172.21.15.45:0/248244073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 0x562b55234910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:31:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.036+0000 7f2a4effd640 1 -- 172.21.15.45:0/3151390456 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f2a540cf8d0 con 0x7f2a58102530 2024-06-06T04:31:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.036+0000 7f2a5d282640 1 --2- 172.21.15.45:0/248244073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b55237410 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2a5806ad30 tx=0x7f2a48005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dda50dc587ea2b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.036+0000 7f2a5e284640 1 --2- 172.21.15.45:0/3151390456 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2a440a2cd0 0x7f2a440a5190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x562b55230670 tx=0x7f2a50014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.036+0000 7f2a43fff640 1 -- 172.21.15.45:0/248244073 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a48035030 con 0x562b55234f80 2024-06-06T04:31:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.036+0000 7f2a43fff640 1 -- 172.21.15.45:0/248244073 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a48040070 con 0x562b55234f80 2024-06-06T04:31:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.036+0000 7f2a43fff640 1 -- 172.21.15.45:0/248244073 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a48003a50 con 0x562b55234f80 2024-06-06T04:31:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.037+0000 7f2a629f0740 1 -- 172.21.15.45:0/248244073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 msgr2=0x562b55237410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.037+0000 7f2a629f0740 1 --2- 172.21.15.45:0/248244073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b55237410 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2a5806ad30 tx=0x7f2a48005c00 comp rx=0 tx=0).stop 2024-06-06T04:31:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.037+0000 7f2a629f0740 1 -- 172.21.15.45:0/248244073 shutdown_connections 2024-06-06T04:31:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.037+0000 7f2a629f0740 1 --2- 172.21.15.45:0/248244073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 0x562b55234910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.037+0000 7f2a629f0740 1 --2- 172.21.15.45:0/248244073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b55237410 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.037+0000 7f2a629f0740 1 --2- 172.21.15.45:0/248244073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b5522f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.037+0000 7f2a629f0740 1 -- 172.21.15.45:0/248244073 >> 172.21.15.45:0/248244073 conn(0x562b55226680 msgr2=0x562b55226a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.037+0000 7f2a629f0740 1 -- 172.21.15.45:0/248244073 shutdown_connections 2024-06-06T04:31:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.037+0000 7f2a629f0740 1 -- 172.21.15.45:0/248244073 wait complete. 2024-06-06T04:31:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.038+0000 7f2a629f0740 1 -- start start 2024-06-06T04:31:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.038+0000 7f2a629f0740 1 Processor -- start 2024-06-06T04:31:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.038+0000 7f2a629f0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b55239600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.038+0000 7f2a629f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 0x562b55239b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.038+0000 7f2a629f0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b55240b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.038+0000 7f2a629f0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562b55237ed0 con 0x562b55234f80 2024-06-06T04:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.038+0000 7f2a629f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b55117ae0 con 0x562b5522fe10 2024-06-06T04:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.038+0000 7f2a629f0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562b55238050 con 0x562b5522f420 2024-06-06T04:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.038+0000 7f2a629f0740 10 client.0 _ll_register_callbacks cb 0x562b551509b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.038+0000 7f2a5e284640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b55240b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.038+0000 7f2a5e284640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b55240b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37596/0 (socket says 172.21.15.45:37596) 2024-06-06T04:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.038+0000 7f2a5e284640 1 -- 172.21.15.45:0/490786807 learned_addr learned my addr 172.21.15.45:0/490786807 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.038+0000 7f2a5d282640 1 --2- 172.21.15.45:0/490786807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 0x562b55239b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.038+0000 7f2a5e284640 1 -- 172.21.15.45:0/490786807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 msgr2=0x562b55239600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.038+0000 7f2a5da83640 1 --2- 172.21.15.45:0/490786807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b55239600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.039+0000 7f2a5e284640 1 --2- 172.21.15.45:0/490786807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b55239600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.039+0000 7f2a5e284640 1 -- 172.21.15.45:0/490786807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 msgr2=0x562b55239b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.039+0000 7f2a5e284640 1 --2- 172.21.15.45:0/490786807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 0x562b55239b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.039+0000 7f2a5e284640 1 -- 172.21.15.45:0/490786807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b54ea2210 con 0x562b55234f80 2024-06-06T04:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.039+0000 7f2a5d282640 1 --2- 172.21.15.45:0/490786807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 0x562b55239b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.039+0000 7f2a5da83640 1 --2- 172.21.15.45:0/490786807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b55239600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.039+0000 7f2a5e284640 1 --2- 172.21.15.45:0/490786807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b55240b80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009fa0 tx=0x7f2a50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c81d40d2d433fa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.039+0000 7f2a417fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.039+0000 7f2a417fa640 1 -- 172.21.15.45:0/490786807 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a50024070 con 0x562b55234f80 2024-06-06T04:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.039+0000 7f2a417fa640 1 -- 172.21.15.45:0/490786807 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f2a50011440 con 0x562b55234f80 2024-06-06T04:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.039+0000 7f2a629f0740 1 -- 172.21.15.45:0/490786807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a48040070 con 0x562b55234f80 2024-06-06T04:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.039+0000 7f2a417fa640 1 -- 172.21.15.45:0/490786807 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a50030050 con 0x562b55234f80 2024-06-06T04:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.039+0000 7f2a629f0740 10 client.15348 fetch_fsmap learned FSMap version 27 2024-06-06T04:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.039+0000 7f2a629f0740 1 -- 172.21.15.45:0/490786807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562b54f6e620 con 0x562b55234f80 2024-06-06T04:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.039+0000 7f2a417fa640 1 -- 172.21.15.45:0/490786807 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a48040070 con 0x562b55234f80 2024-06-06T04:31:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.040+0000 7f2a417fa640 1 -- 172.21.15.45:0/490786807 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2a50020750 con 0x562b55234f80 2024-06-06T04:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.040+0000 7f2a629f0740 10 client.15348 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.040+0000 7f2a629f0740 10 client.15348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.040+0000 7f2a629f0740 20 client.15348 populate_metadata read hostname 'smithi045' 2024-06-06T04:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.040+0000 7f2a629f0740 1 --2- 172.21.15.45:0/490786807 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x562b55245550 0x562b55247a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.040+0000 7f2a629f0740 4 client.15348 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.040+0000 7f2a629f0740 1 -- 172.21.15.45:0/490786807 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b551a1f10 con 0x562b55245550 2024-06-06T04:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.040+0000 7f2a5da83640 1 --2- 172.21.15.45:0/490786807 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x562b55245550 0x562b55247a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.040+0000 7f2a5da83640 1 --2- 172.21.15.45:0/490786807 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x562b55245550 0x562b55247a10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38c329fb46e692b9 server_cookie=76709d695d1c3ebb in_seq=0 out_seq=0 2024-06-06T04:31:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.040+0000 7f2a417fa640 10 client.15348 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.041+0000 7f2a417fa640 1 -- 172.21.15.45:0/490786807 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562b551a1f10 con 0x562b55245550 2024-06-06T04:31:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.041+0000 7f2a417fa640 10 client.15348 handle_command_reply: tid=0 2024-06-06T04:31:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.041+0000 7f2a629f0740 1 client.15348 shutdown 2024-06-06T04:31:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.042+0000 7f2a629f0740 1 -- 172.21.15.45:0/490786807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 msgr2=0x562b55240b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.042+0000 7f2a629f0740 1 --2- 172.21.15.45:0/490786807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b55240b80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009fa0 tx=0x7f2a50004a40 comp rx=0 tx=0).stop 2024-06-06T04:31:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.042+0000 7f2a629f0740 1 -- 172.21.15.45:0/490786807 shutdown_connections 2024-06-06T04:31:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.042+0000 7f2a629f0740 1 --2- 172.21.15.45:0/490786807 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x562b55245550 0x562b55247a10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.042+0000 7f2a629f0740 1 --2- 172.21.15.45:0/490786807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 0x562b55239b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.042+0000 7f2a629f0740 1 --2- 172.21.15.45:0/490786807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b55240b80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.042+0000 7f2a417fa640 0 client.15348 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.042+0000 7f2a629f0740 1 --2- 172.21.15.45:0/490786807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b55239600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.042+0000 7f2a629f0740 1 -- 172.21.15.45:0/490786807 >> 172.21.15.45:0/490786807 conn(0x562b55226680 msgr2=0x562b552257e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.042+0000 7f2a629f0740 1 -- 172.21.15.45:0/490786807 shutdown_connections 2024-06-06T04:31:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.042+0000 7f2a629f0740 1 -- 172.21.15.45:0/490786807 wait complete. 2024-06-06T04:31:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.042+0000 7f2a629f0740 20 client.15348 trim_cache size 0 max 16384 2024-06-06T04:31:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.057+0000 7f2a629f0740 1 Processor -- start 2024-06-06T04:31:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.057+0000 7f2a629f0740 1 -- start start 2024-06-06T04:31:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a629f0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b55230ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a629f0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b5522fe10 0x562b55240b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a629f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55234f80 0x562b55231530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a629f0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a581aabc0 con 0x562b5522fe10 2024-06-06T04:31:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a629f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a581a8490 con 0x562b55234f80 2024-06-06T04:31:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a629f0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2a581a62e0 con 0x562b5522f420 2024-06-06T04:31:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a5d282640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b5522fe10 0x562b55240b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a5d282640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b5522fe10 0x562b55240b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37602/0 (socket says 172.21.15.45:37602) 2024-06-06T04:31:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a5d282640 1 -- 172.21.15.45:0/1978038484 learned_addr learned my addr 172.21.15.45:0/1978038484 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a5e284640 1 --2- 172.21.15.45:0/1978038484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55234f80 0x562b55231530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a5d282640 1 -- 172.21.15.45:0/1978038484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 msgr2=0x562b55230ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a5da83640 1 --2- 172.21.15.45:0/1978038484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b55230ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a5d282640 1 --2- 172.21.15.45:0/1978038484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b55230ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a5d282640 1 -- 172.21.15.45:0/1978038484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55234f80 msgr2=0x562b55231530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a5d282640 1 --2- 172.21.15.45:0/1978038484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55234f80 0x562b55231530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a5d282640 1 -- 172.21.15.45:0/1978038484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b552264b0 con 0x562b5522fe10 2024-06-06T04:31:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a5e284640 1 --2- 172.21.15.45:0/1978038484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55234f80 0x562b55231530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:31:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a5da83640 1 --2- 172.21.15.45:0/1978038484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b55230ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:31:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a5d282640 1 --2- 172.21.15.45:0/1978038484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b5522fe10 0x562b55240b40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2a480314e0 tx=0x7f2a48005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e24bb3cc29408 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a437fe640 1 -- 172.21.15.45:0/1978038484 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4804d070 con 0x562b5522fe10 2024-06-06T04:31:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a437fe640 1 -- 172.21.15.45:0/1978038484 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a48040070 con 0x562b5522fe10 2024-06-06T04:31:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.058+0000 7f2a437fe640 1 -- 172.21.15.45:0/1978038484 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a48035030 con 0x562b5522fe10 2024-06-06T04:31:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.059+0000 7f2a629f0740 1 -- 172.21.15.45:0/1978038484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b5522fe10 msgr2=0x562b55240b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.059+0000 7f2a629f0740 1 --2- 172.21.15.45:0/1978038484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b5522fe10 0x562b55240b40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2a480314e0 tx=0x7f2a48005850 comp rx=0 tx=0).stop 2024-06-06T04:31:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.059+0000 7f2a629f0740 1 -- 172.21.15.45:0/1978038484 shutdown_connections 2024-06-06T04:31:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.059+0000 7f2a629f0740 1 --2- 172.21.15.45:0/1978038484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55234f80 0x562b55231530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.059+0000 7f2a629f0740 1 --2- 172.21.15.45:0/1978038484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b5522fe10 0x562b55240b40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.059+0000 7f2a629f0740 1 --2- 172.21.15.45:0/1978038484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b55230ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.059+0000 7f2a629f0740 1 -- 172.21.15.45:0/1978038484 >> 172.21.15.45:0/1978038484 conn(0x562b55226680 msgr2=0x562b552383f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.059+0000 7f2a629f0740 1 -- 172.21.15.45:0/1978038484 shutdown_connections 2024-06-06T04:31:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.059+0000 7f2a629f0740 1 -- 172.21.15.45:0/1978038484 wait complete. 2024-06-06T04:31:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.059+0000 7f2a629f0740 1 -- start start 2024-06-06T04:31:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.059+0000 7f2a629f0740 1 Processor -- start 2024-06-06T04:31:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a629f0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b5523dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a629f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 0x562b5523e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a629f0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b5523f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a629f0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a581a6950 con 0x562b55234f80 2024-06-06T04:31:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a629f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a581a7dc0 con 0x562b5522fe10 2024-06-06T04:31:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a629f0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2a581a7020 con 0x562b5522f420 2024-06-06T04:31:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a629f0740 10 client.0 _ll_register_callbacks cb 0x7f2a5000d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:31:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a5e284640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b5523f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a5e284640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b5523f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37614/0 (socket says 172.21.15.45:37614) 2024-06-06T04:31:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a5e284640 1 -- 172.21.15.45:0/1038014452 learned_addr learned my addr 172.21.15.45:0/1038014452 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a5d282640 1 --2- 172.21.15.45:0/1038014452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 0x562b5523e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a5da83640 1 --2- 172.21.15.45:0/1038014452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b5523dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a5e284640 1 -- 172.21.15.45:0/1038014452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 msgr2=0x562b5523dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a5e284640 1 --2- 172.21.15.45:0/1038014452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b5523dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a5e284640 1 -- 172.21.15.45:0/1038014452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 msgr2=0x562b5523e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a5e284640 1 --2- 172.21.15.45:0/1038014452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 0x562b5523e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a5e284640 1 -- 172.21.15.45:0/1038014452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b5523fa20 con 0x562b55234f80 2024-06-06T04:31:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a5e284640 1 --2- 172.21.15.45:0/1038014452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b5523f430 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009d00 tx=0x7f2a50012470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7388f3167a38888 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.060+0000 7f2a417fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:31:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.061+0000 7f2a417fa640 1 -- 172.21.15.45:0/1038014452 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5003a070 con 0x562b55234f80 2024-06-06T04:31:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.061+0000 7f2a417fa640 1 -- 172.21.15.45:0/1038014452 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f2a50020820 con 0x562b55234f80 2024-06-06T04:31:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.061+0000 7f2a629f0740 1 -- 172.21.15.45:0/1038014452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a48040070 con 0x562b55234f80 2024-06-06T04:31:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.061+0000 7f2a417fa640 1 -- 172.21.15.45:0/1038014452 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a50045050 con 0x562b55234f80 2024-06-06T04:31:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.061+0000 7f2a629f0740 10 client.15354 fetch_fsmap learned FSMap version 27 2024-06-06T04:31:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.061+0000 7f2a629f0740 1 -- 172.21.15.45:0/1038014452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562b55240ca0 con 0x562b55234f80 2024-06-06T04:31:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.061+0000 7f2a417fa640 1 -- 172.21.15.45:0/1038014452 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a48040070 con 0x562b55234f80 2024-06-06T04:31:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.061+0000 7f2a417fa640 1 -- 172.21.15.45:0/1038014452 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2a500344c0 con 0x562b55234f80 2024-06-06T04:31:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.061+0000 7f2a629f0740 10 client.15354 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:31:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.061+0000 7f2a629f0740 10 client.15354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:31:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.061+0000 7f2a629f0740 20 client.15354 populate_metadata read hostname 'smithi045' 2024-06-06T04:31:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.061+0000 7f2a629f0740 1 --2- 172.21.15.45:0/1038014452 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x562b552b0850 0x562b552b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.061+0000 7f2a629f0740 4 client.15354 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T04:31:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.061+0000 7f2a629f0740 1 -- 172.21.15.45:0/1038014452 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b5519bc20 con 0x562b552b0850 2024-06-06T04:31:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.062+0000 7f2a5da83640 1 --2- 172.21.15.45:0/1038014452 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x562b552b0850 0x562b552b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.062+0000 7f2a5da83640 1 --2- 172.21.15.45:0/1038014452 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x562b552b0850 0x562b552b2c70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c07902ff8f9a2df server_cookie=f0ec6b879f6134ee in_seq=0 out_seq=0 2024-06-06T04:31:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.062+0000 7f2a417fa640 10 client.15354 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.075+0000 7f2a417fa640 1 -- 172.21.15.45:0/1038014452 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562b5519bc20 con 0x562b552b0850 2024-06-06T04:31:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.075+0000 7f2a417fa640 10 client.15354 handle_command_reply: tid=0 2024-06-06T04:31:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.075+0000 7f2a629f0740 1 client.15354 shutdown 2024-06-06T04:31:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.075+0000 7f2a629f0740 1 -- 172.21.15.45:0/1038014452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 msgr2=0x562b5523f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.075+0000 7f2a629f0740 1 --2- 172.21.15.45:0/1038014452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b5523f430 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009d00 tx=0x7f2a50012470 comp rx=0 tx=0).stop 2024-06-06T04:31:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.075+0000 7f2a629f0740 1 -- 172.21.15.45:0/1038014452 shutdown_connections 2024-06-06T04:31:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.075+0000 7f2a629f0740 1 --2- 172.21.15.45:0/1038014452 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x562b552b0850 0x562b552b2c70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.075+0000 7f2a629f0740 1 --2- 172.21.15.45:0/1038014452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5522fe10 0x562b5523e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.075+0000 7f2a629f0740 1 --2- 172.21.15.45:0/1038014452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b55234f80 0x562b5523f430 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.075+0000 7f2a417fa640 0 client.15354 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.075+0000 7f2a629f0740 1 --2- 172.21.15.45:0/1038014452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562b5522f420 0x562b5523dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.075+0000 7f2a629f0740 1 -- 172.21.15.45:0/1038014452 >> 172.21.15.45:0/1038014452 conn(0x562b55226680 msgr2=0x562b55239520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.075+0000 7f2a629f0740 1 -- 172.21.15.45:0/1038014452 shutdown_connections 2024-06-06T04:31:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.075+0000 7f2a629f0740 1 -- 172.21.15.45:0/1038014452 wait complete. 2024-06-06T04:31:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.075+0000 7f2a629f0740 20 client.15354 trim_cache size 0 max 16384 2024-06-06T04:31:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-06-06T04:31:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-06-06T04:31:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-06-06T04:31:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.076+0000 7f2a5fce7640 1 -- 172.21.15.45:0/3151390456 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2a440a2cd0 msgr2=0x7f2a440a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.076+0000 7f2a5fce7640 1 --2- 172.21.15.45:0/3151390456 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2a440a2cd0 0x7f2a440a5190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x562b55230670 tx=0x7f2a50014040 comp rx=0 tx=0).stop 2024-06-06T04:31:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.076+0000 7f2a5fce7640 1 -- 172.21.15.45:0/3151390456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a58102530 msgr2=0x7f2a58197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.076+0000 7f2a5fce7640 1 --2- 172.21.15.45:0/3151390456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a58102530 0x7f2a58197b70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400ea60 tx=0x7f2a5400ef20 comp rx=0 tx=0).stop 2024-06-06T04:31:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.076+0000 7f2a5fce7640 1 -- 172.21.15.45:0/3151390456 shutdown_connections 2024-06-06T04:31:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.076+0000 7f2a5fce7640 1 --2- 172.21.15.45:0/3151390456 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2a440a2cd0 0x7f2a440a5190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.076+0000 7f2a5fce7640 1 --2- 172.21.15.45:0/3151390456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a58104e60 0x7f2a581980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.076+0000 7f2a5fce7640 1 --2- 172.21.15.45:0/3151390456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a58107790 0x7f2a5819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.076+0000 7f2a5fce7640 1 --2- 172.21.15.45:0/3151390456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a58102530 0x7f2a58197b70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.076+0000 7f2a5fce7640 1 -- 172.21.15.45:0/3151390456 >> 172.21.15.45:0/3151390456 conn(0x7f2a580fbf20 msgr2=0x7f2a58102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.076+0000 7f2a5fce7640 1 -- 172.21.15.45:0/3151390456 shutdown_connections 2024-06-06T04:31:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:44.076+0000 7f2a5fce7640 1 -- 172.21.15.45:0/3151390456 wait complete. 2024-06-06T04:31:44.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:43 smithi086 ceph-mon[36053]: pgmap v439: 129 pgs: 129 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T04:31:44.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:43 smithi154 ceph-mon[36428]: pgmap v439: 129 pgs: 129 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T04:31:44.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:43 smithi045 ceph-mon[32299]: pgmap v439: 129 pgs: 129 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T04:31:45.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:45 smithi086 ceph-mon[36053]: scrub summary: idle 2024-06-06T04:31:45.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:44 smithi154 ceph-mon[36428]: scrub summary: idle 2024-06-06T04:31:45.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:44 smithi045 ceph-mon[32299]: scrub summary: idle 2024-06-06T04:31:46.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:46 smithi086 ceph-mon[36053]: pgmap v440: 129 pgs: 129 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-06T04:31:46.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:46 smithi154 ceph-mon[36428]: pgmap v440: 129 pgs: 129 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-06T04:31:46.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:46 smithi045 ceph-mon[32299]: pgmap v440: 129 pgs: 129 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-06T04:31:48.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:48 smithi086 ceph-mon[36053]: pgmap v441: 129 pgs: 129 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-06-06T04:31:48.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:48 smithi154 ceph-mon[36428]: pgmap v441: 129 pgs: 129 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-06-06T04:31:48.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:48 smithi045 ceph-mon[32299]: pgmap v441: 129 pgs: 129 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-06-06T04:31:50.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:50 smithi086 ceph-mon[36053]: pgmap v442: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-06-06T04:31:50.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:50 smithi154 ceph-mon[36428]: pgmap v442: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-06-06T04:31:50.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:50 smithi045 ceph-mon[32299]: pgmap v442: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-06-06T04:31:52.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:52 smithi086 ceph-mon[36053]: pgmap v443: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-06-06T04:31:52.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:52 smithi154 ceph-mon[36428]: pgmap v443: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-06-06T04:31:52.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:52 smithi045 ceph-mon[32299]: pgmap v443: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-06-06T04:31:54.279 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:54 smithi045 ceph-mon[32299]: pgmap v444: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-06-06T04:31:54.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:54 smithi086 ceph-mon[36053]: pgmap v444: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-06-06T04:31:54.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:54 smithi154 ceph-mon[36428]: pgmap v444: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-06-06T04:31:56.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:56 smithi086 ceph-mon[36053]: pgmap v445: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-06T04:31:56.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:56 smithi154 ceph-mon[36428]: pgmap v445: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-06T04:31:56.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:56 smithi045 ceph-mon[32299]: pgmap v445: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-06T04:31:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.179+0000 7f356142a640 1 -- 172.21.15.45:0/585892914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c102560 msgr2=0x7f355c108df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.179+0000 7f356142a640 1 --2- 172.21.15.45:0/585892914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c102560 0x7f355c108df0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3554009ca0 tx=0x7f355402a8c0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.179+0000 7f356142a640 1 -- 172.21.15.45:0/585892914 shutdown_connections 2024-06-06T04:31:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.179+0000 7f356142a640 1 --2- 172.21.15.45:0/585892914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f355c101ba0 0x7f355c102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.179+0000 7f356142a640 1 --2- 172.21.15.45:0/585892914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c102560 0x7f355c108df0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.179+0000 7f356142a640 1 --2- 172.21.15.45:0/585892914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f355c1009b0 0x7f355c100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.179+0000 7f356142a640 1 -- 172.21.15.45:0/585892914 >> 172.21.15.45:0/585892914 conn(0x7f355c0fbf20 msgr2=0x7f355c0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.179+0000 7f356142a640 1 -- 172.21.15.45:0/585892914 shutdown_connections 2024-06-06T04:31:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.179+0000 7f356142a640 1 -- 172.21.15.45:0/585892914 wait complete. 2024-06-06T04:31:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.179+0000 7f356142a640 1 Processor -- start 2024-06-06T04:31:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f356142a640 1 -- start start 2024-06-06T04:31:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f356142a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f355c1009b0 0x7f355c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f356142a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f355c101ba0 0x7f355c19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f356142a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c102560 0x7f355c1a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f356142a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f355c10c450 con 0x7f355c102560 2024-06-06T04:31:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f356142a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f355c10c2d0 con 0x7f355c1009b0 2024-06-06T04:31:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f356142a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f355c10c5d0 con 0x7f355c101ba0 2024-06-06T04:31:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f355b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c102560 0x7f355c1a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f355b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c102560 0x7f355c1a1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56102/0 (socket says 172.21.15.45:56102) 2024-06-06T04:31:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f355b7fe640 1 -- 172.21.15.45:0/3266590981 learned_addr learned my addr 172.21.15.45:0/3266590981 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f355a7fc640 1 --2- 172.21.15.45:0/3266590981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f355c101ba0 0x7f355c19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f355affd640 1 --2- 172.21.15.45:0/3266590981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f355c1009b0 0x7f355c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f355b7fe640 1 -- 172.21.15.45:0/3266590981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f355c101ba0 msgr2=0x7f355c19a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f355b7fe640 1 --2- 172.21.15.45:0/3266590981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f355c101ba0 0x7f355c19a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f355b7fe640 1 -- 172.21.15.45:0/3266590981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f355c1009b0 msgr2=0x7f355c199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f355b7fe640 1 --2- 172.21.15.45:0/3266590981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f355c1009b0 0x7f355c199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f355b7fe640 1 -- 172.21.15.45:0/3266590981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c1a18b0 con 0x7f355c102560 2024-06-06T04:31:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f355affd640 1 --2- 172.21.15.45:0/3266590981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f355c1009b0 0x7f355c199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:31:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.180+0000 7f355b7fe640 1 --2- 172.21.15.45:0/3266590981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c102560 0x7f355c1a1210 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3554009dc0 tx=0x7f3554003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.181+0000 7f3543fff640 1 -- 172.21.15.45:0/3266590981 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554032ec0 con 0x7f355c102560 2024-06-06T04:31:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.181+0000 7f3543fff640 1 -- 172.21.15.45:0/3266590981 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35540044f0 con 0x7f355c102560 2024-06-06T04:31:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.181+0000 7f356142a640 1 -- 172.21.15.45:0/3266590981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355c1a1b40 con 0x7f355c102560 2024-06-06T04:31:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.181+0000 7f3543fff640 1 -- 172.21.15.45:0/3266590981 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554030d40 con 0x7f355c102560 2024-06-06T04:31:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.181+0000 7f356142a640 1 -- 172.21.15.45:0/3266590981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355c1a20e0 con 0x7f355c102560 2024-06-06T04:31:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.183+0000 7f3543fff640 1 -- 172.21.15.45:0/3266590981 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f3554004040 con 0x7f355c102560 2024-06-06T04:31:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f3564211740 1 Processor -- start 2024-06-06T04:31:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f3564211740 1 -- start start 2024-06-06T04:31:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f3564211740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683f010 0x55a14683f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f3564211740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a14683fa00 0x55a146844500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f3564211740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a146844b70 0x55a146847000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f3564211740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a1467a24f0 con 0x55a14683fa00 2024-06-06T04:31:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f3564211740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a146740870 con 0x55a14683f010 2024-06-06T04:31:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f3564211740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a14679e4b0 con 0x55a146844b70 2024-06-06T04:31:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f355affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a14683fa00 0x55a146844500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f355affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a14683fa00 0x55a146844500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56112/0 (socket says 172.21.15.45:56112) 2024-06-06T04:31:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f355affd640 1 -- 172.21.15.45:0/1355654404 learned_addr learned my addr 172.21.15.45:0/1355654404 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f355a7fc640 1 --2- 172.21.15.45:0/1355654404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683f010 0x55a14683f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f3543fff640 1 --2- 172.21.15.45:0/3266590981 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f35440a2ef0 0x7f35440a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f355a7fc640 1 --2- 172.21.15.45:0/1355654404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a146844b70 0x55a146847000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f355affd640 1 -- 172.21.15.45:0/1355654404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a146844b70 msgr2=0x55a146847000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f355affd640 1 --2- 172.21.15.45:0/1355654404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a146844b70 0x55a146847000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f355affd640 1 -- 172.21.15.45:0/1355654404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683f010 msgr2=0x55a14683f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f355affd640 1 --2- 172.21.15.45:0/1355654404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683f010 0x55a14683f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f355affd640 1 -- 172.21.15.45:0/1355654404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a14675c9b0 con 0x55a14683fa00 2024-06-06T04:31:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f355b7fe640 1 --2- 172.21.15.45:0/3266590981 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f35440a2ef0 0x7f35440a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.184+0000 7f3543fff640 1 -- 172.21.15.45:0/3266590981 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f35540f0a10 con 0x7f355c102560 2024-06-06T04:31:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.185+0000 7f355affd640 1 --2- 172.21.15.45:0/1355654404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a14683fa00 0x55a146844500 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f354c008c20 tx=0x7f354c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8702bcbf6d97b78 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.185+0000 7f35417fa640 1 -- 172.21.15.45:0/1355654404 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c014070 con 0x55a14683fa00 2024-06-06T04:31:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.185+0000 7f35417fa640 1 -- 172.21.15.45:0/1355654404 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f354c0054d0 con 0x55a14683fa00 2024-06-06T04:31:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.185+0000 7f35417fa640 1 -- 172.21.15.45:0/1355654404 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c004bd0 con 0x55a14683fa00 2024-06-06T04:31:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.185+0000 7f3564211740 1 -- 172.21.15.45:0/1355654404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a14683fa00 msgr2=0x55a146844500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.185+0000 7f355b7fe640 1 --2- 172.21.15.45:0/3266590981 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f35440a2ef0 0x7f35440a53b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f35540bcb60 tx=0x7f355408ff20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.185+0000 7f3564211740 1 --2- 172.21.15.45:0/1355654404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a14683fa00 0x55a146844500 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f354c008c20 tx=0x7f354c00e720 comp rx=0 tx=0).stop 2024-06-06T04:31:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f3564211740 1 -- 172.21.15.45:0/1355654404 shutdown_connections 2024-06-06T04:31:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f3564211740 1 --2- 172.21.15.45:0/1355654404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a146844b70 0x55a146847000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f3564211740 1 --2- 172.21.15.45:0/1355654404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a14683fa00 0x55a146844500 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f3564211740 1 --2- 172.21.15.45:0/1355654404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683f010 0x55a14683f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f3564211740 1 -- 172.21.15.45:0/1355654404 >> 172.21.15.45:0/1355654404 conn(0x55a14683a680 msgr2=0x55a14683aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f3564211740 1 -- 172.21.15.45:0/1355654404 shutdown_connections 2024-06-06T04:31:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f3564211740 1 -- 172.21.15.45:0/1355654404 wait complete. 2024-06-06T04:31:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f3564211740 1 -- start start 2024-06-06T04:31:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f3564211740 1 Processor -- start 2024-06-06T04:31:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f3564211740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a14683f010 0x55a146849250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f3564211740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683fa00 0x55a146849790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f3564211740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a146844b70 0x55a1468507d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f3564211740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a146847b20 con 0x55a14683f010 2024-06-06T04:31:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f3564211740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a14672bae0 con 0x55a14683fa00 2024-06-06T04:31:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f3564211740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a146847ca0 con 0x55a146844b70 2024-06-06T04:31:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f3564211740 10 client.0 _ll_register_callbacks cb 0x55a1467649b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:31:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f355a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a14683f010 0x55a146849250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f355a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a14683f010 0x55a146849250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56128/0 (socket says 172.21.15.45:56128) 2024-06-06T04:31:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.186+0000 7f355a7fc640 1 -- 172.21.15.45:0/447374749 learned_addr learned my addr 172.21.15.45:0/447374749 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.187+0000 7f355affd640 1 --2- 172.21.15.45:0/447374749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683fa00 0x55a146849790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.187+0000 7f355a7fc640 1 --2- 172.21.15.45:0/447374749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a146844b70 0x55a1468507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.187+0000 7f355a7fc640 1 -- 172.21.15.45:0/447374749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a146844b70 msgr2=0x55a1468507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.187+0000 7f355a7fc640 1 --2- 172.21.15.45:0/447374749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a146844b70 0x55a1468507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.187+0000 7f355a7fc640 1 -- 172.21.15.45:0/447374749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683fa00 msgr2=0x55a146849790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.187+0000 7f355a7fc640 1 --2- 172.21.15.45:0/447374749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683fa00 0x55a146849790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.187+0000 7f355a7fc640 1 -- 172.21.15.45:0/447374749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1464b6210 con 0x55a14683f010 2024-06-06T04:31:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.187+0000 7f355affd640 1 --2- 172.21.15.45:0/447374749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683fa00 0x55a146849790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:31:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.187+0000 7f355a7fc640 1 --2- 172.21.15.45:0/447374749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a14683f010 0x55a146849250 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3550012740 tx=0x7f3550005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977ba7101040307 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.187+0000 7f353eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:31:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.187+0000 7f353eff5640 1 -- 172.21.15.45:0/447374749 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550004220 con 0x55a14683f010 2024-06-06T04:31:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.187+0000 7f353eff5640 1 -- 172.21.15.45:0/447374749 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f35500043c0 con 0x55a14683f010 2024-06-06T04:31:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.187+0000 7f3564211740 1 -- 172.21.15.45:0/447374749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f354c0054d0 con 0x55a14683f010 2024-06-06T04:31:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.187+0000 7f353eff5640 1 -- 172.21.15.45:0/447374749 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3550028050 con 0x55a14683f010 2024-06-06T04:31:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.187+0000 7f3564211740 10 client.15366 fetch_fsmap learned FSMap version 27 2024-06-06T04:31:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.187+0000 7f3564211740 1 -- 172.21.15.45:0/447374749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a146582620 con 0x55a14683f010 2024-06-06T04:31:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.188+0000 7f353eff5640 1 -- 172.21.15.45:0/447374749 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f354c0054d0 con 0x55a14683f010 2024-06-06T04:31:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.188+0000 7f353eff5640 1 -- 172.21.15.45:0/447374749 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f35500047a0 con 0x55a14683f010 2024-06-06T04:31:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.188+0000 7f3564211740 10 client.15366 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:31:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.188+0000 7f3564211740 10 client.15366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:31:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.188+0000 7f3564211740 20 client.15366 populate_metadata read hostname 'smithi045' 2024-06-06T04:31:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.188+0000 7f3564211740 1 --2- 172.21.15.45:0/447374749 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55a146855060 0x55a1468574f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.188+0000 7f3564211740 4 client.15366 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:31:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.188+0000 7f3564211740 1 -- 172.21.15.45:0/447374749 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1467b5f10 con 0x55a146855060 2024-06-06T04:31:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.188+0000 7f355affd640 1 --2- 172.21.15.45:0/447374749 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55a146855060 0x55a1468574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.189+0000 7f355affd640 1 --2- 172.21.15.45:0/447374749 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55a146855060 0x55a1468574f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2368e4a4e98b990 server_cookie=234895a4d78f168a in_seq=0 out_seq=0 2024-06-06T04:31:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.189+0000 7f353eff5640 10 client.15366 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.189+0000 7f353eff5640 1 -- 172.21.15.45:0/447374749 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a1467b5f10 con 0x55a146855060 2024-06-06T04:31:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.189+0000 7f353eff5640 10 client.15366 handle_command_reply: tid=0 2024-06-06T04:31:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.189+0000 7f3564211740 1 client.15366 shutdown 2024-06-06T04:31:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.189+0000 7f3564211740 1 -- 172.21.15.45:0/447374749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a14683f010 msgr2=0x55a146849250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.189+0000 7f3564211740 1 --2- 172.21.15.45:0/447374749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a14683f010 0x55a146849250 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3550012740 tx=0x7f3550005850 comp rx=0 tx=0).stop 2024-06-06T04:31:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.190+0000 7f3564211740 1 -- 172.21.15.45:0/447374749 shutdown_connections 2024-06-06T04:31:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.190+0000 7f3564211740 1 --2- 172.21.15.45:0/447374749 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55a146855060 0x55a1468574f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.190+0000 7f3564211740 1 --2- 172.21.15.45:0/447374749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683fa00 0x55a146849790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.190+0000 7f3564211740 1 --2- 172.21.15.45:0/447374749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a146844b70 0x55a1468507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.190+0000 7f3564211740 1 --2- 172.21.15.45:0/447374749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a14683f010 0x55a146849250 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.190+0000 7f353eff5640 0 client.15366 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.190+0000 7f3564211740 1 -- 172.21.15.45:0/447374749 >> 172.21.15.45:0/447374749 conn(0x55a14683a680 msgr2=0x55a1468397e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.190+0000 7f3564211740 1 -- 172.21.15.45:0/447374749 shutdown_connections 2024-06-06T04:31:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.190+0000 7f3564211740 1 -- 172.21.15.45:0/447374749 wait complete. 2024-06-06T04:31:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.190+0000 7f3564211740 20 client.15366 trim_cache size 0 max 16384 2024-06-06T04:31:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.205+0000 7f3564211740 1 Processor -- start 2024-06-06T04:31:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.205+0000 7f3564211740 1 -- start start 2024-06-06T04:31:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.205+0000 7f3564211740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683f010 0x55a14683d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.205+0000 7f3564211740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a14683fa00 0x55a146850790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.205+0000 7f3564211740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a146844b70 0x55a14683db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.205+0000 7f3564211740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f355c1aef50 con 0x55a146844b70 2024-06-06T04:31:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.205+0000 7f3564211740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f355c1ac820 con 0x55a14683f010 2024-06-06T04:31:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.205+0000 7f3564211740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f355c1aa670 con 0x55a14683fa00 2024-06-06T04:31:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.206+0000 7f355a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a146844b70 0x55a14683db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.206+0000 7f355a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a146844b70 0x55a14683db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56142/0 (socket says 172.21.15.45:56142) 2024-06-06T04:31:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.206+0000 7f355a7fc640 1 -- 172.21.15.45:0/4080886593 learned_addr learned my addr 172.21.15.45:0/4080886593 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.206+0000 7f355a7fc640 1 --2- 172.21.15.45:0/4080886593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683f010 0x55a14683d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.206+0000 7f355affd640 1 --2- 172.21.15.45:0/4080886593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a14683fa00 0x55a146850790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.206+0000 7f355a7fc640 1 -- 172.21.15.45:0/4080886593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a14683fa00 msgr2=0x55a146850790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.206+0000 7f355a7fc640 1 --2- 172.21.15.45:0/4080886593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a14683fa00 0x55a146850790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.206+0000 7f355a7fc640 1 -- 172.21.15.45:0/4080886593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683f010 msgr2=0x55a14683d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.206+0000 7f355a7fc640 1 --2- 172.21.15.45:0/4080886593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683f010 0x55a14683d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.206+0000 7f355a7fc640 1 -- 172.21.15.45:0/4080886593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a14683a4b0 con 0x55a146844b70 2024-06-06T04:31:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.206+0000 7f355a7fc640 1 --2- 172.21.15.45:0/4080886593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a146844b70 0x55a14683db90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f35500124d0 tx=0x7f3550012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fc4b2e21680483 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.206+0000 7f3540ff9640 1 -- 172.21.15.45:0/4080886593 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550012030 con 0x55a146844b70 2024-06-06T04:31:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.206+0000 7f3540ff9640 1 -- 172.21.15.45:0/4080886593 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35500121d0 con 0x55a146844b70 2024-06-06T04:31:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.206+0000 7f3540ff9640 1 -- 172.21.15.45:0/4080886593 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550005460 con 0x55a146844b70 2024-06-06T04:31:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 -- 172.21.15.45:0/4080886593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a146844b70 msgr2=0x55a14683db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 --2- 172.21.15.45:0/4080886593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a146844b70 0x55a14683db90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f35500124d0 tx=0x7f3550012f90 comp rx=0 tx=0).stop 2024-06-06T04:31:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 -- 172.21.15.45:0/4080886593 shutdown_connections 2024-06-06T04:31:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 --2- 172.21.15.45:0/4080886593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a146844b70 0x55a14683db90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 --2- 172.21.15.45:0/4080886593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a14683fa00 0x55a146850790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 --2- 172.21.15.45:0/4080886593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683f010 0x55a14683d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 -- 172.21.15.45:0/4080886593 >> 172.21.15.45:0/4080886593 conn(0x55a14683a680 msgr2=0x55a146840be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 -- 172.21.15.45:0/4080886593 shutdown_connections 2024-06-06T04:31:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 -- 172.21.15.45:0/4080886593 wait complete. 2024-06-06T04:31:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 -- start start 2024-06-06T04:31:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 Processor -- start 2024-06-06T04:31:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a14683f010 0x55a14684d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683fa00 0x55a14684dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a146844b70 0x55a1468bddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f355c1aace0 con 0x55a146844b70 2024-06-06T04:31:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f355c1ac150 con 0x55a14683fa00 2024-06-06T04:31:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f355c1ab3b0 con 0x55a14683f010 2024-06-06T04:31:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.207+0000 7f3564211740 10 client.0 _ll_register_callbacks cb 0x7f3550012590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:31:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f355a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a146844b70 0x55a1468bddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f355a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a146844b70 0x55a1468bddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56154/0 (socket says 172.21.15.45:56154) 2024-06-06T04:31:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f355a7fc640 1 -- 172.21.15.45:0/2934927078 learned_addr learned my addr 172.21.15.45:0/2934927078 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:31:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f355affd640 1 --2- 172.21.15.45:0/2934927078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683fa00 0x55a14684dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f355a7fc640 1 --2- 172.21.15.45:0/2934927078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a14683f010 0x55a14684d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f355a7fc640 1 -- 172.21.15.45:0/2934927078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a14683f010 msgr2=0x55a14684d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f355a7fc640 1 --2- 172.21.15.45:0/2934927078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a14683f010 0x55a14684d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f355a7fc640 1 -- 172.21.15.45:0/2934927078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683fa00 msgr2=0x55a14684dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:31:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f355a7fc640 1 --2- 172.21.15.45:0/2934927078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683fa00 0x55a14684dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f355a7fc640 1 -- 172.21.15.45:0/2934927078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1468406d0 con 0x55a146844b70 2024-06-06T04:31:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f355a7fc640 1 --2- 172.21.15.45:0/2934927078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a146844b70 0x55a1468bddc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3550005980 tx=0x7f355000bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a8f37dd7714b4c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:31:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f353eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:31:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f353eff5640 1 -- 172.21.15.45:0/2934927078 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35500053f0 con 0x55a146844b70 2024-06-06T04:31:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f353eff5640 1 -- 172.21.15.45:0/2934927078 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f3550004020 con 0x55a146844b70 2024-06-06T04:31:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f3564211740 1 -- 172.21.15.45:0/2934927078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35500121d0 con 0x55a146844b70 2024-06-06T04:31:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f353eff5640 1 -- 172.21.15.45:0/2934927078 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3550004220 con 0x55a146844b70 2024-06-06T04:31:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f3564211740 10 client.15372 fetch_fsmap learned FSMap version 27 2024-06-06T04:31:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.208+0000 7f3564211740 1 -- 172.21.15.45:0/2934927078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a1468bf2b0 con 0x55a146844b70 2024-06-06T04:31:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.209+0000 7f353eff5640 1 -- 172.21.15.45:0/2934927078 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35500121d0 con 0x55a146844b70 2024-06-06T04:31:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.209+0000 7f353eff5640 1 -- 172.21.15.45:0/2934927078 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3550003680 con 0x55a146844b70 2024-06-06T04:31:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.209+0000 7f3564211740 10 client.15372 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:31:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.209+0000 7f3564211740 10 client.15372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:31:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.209+0000 7f3564211740 20 client.15372 populate_metadata read hostname 'smithi045' 2024-06-06T04:31:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.209+0000 7f3564211740 1 --2- 172.21.15.45:0/2934927078 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55a1468bfb20 0x55a1468c1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:31:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.209+0000 7f3564211740 4 client.15372 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T04:31:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.209+0000 7f3564211740 1 -- 172.21.15.45:0/2934927078 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1467afc20 con 0x55a1468bfb20 2024-06-06T04:31:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.209+0000 7f355affd640 1 --2- 172.21.15.45:0/2934927078 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55a1468bfb20 0x55a1468c1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:31:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.209+0000 7f355affd640 1 --2- 172.21.15.45:0/2934927078 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55a1468bfb20 0x55a1468c1f40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e933ed6b7b8464eb server_cookie=91099cec5816c22e in_seq=0 out_seq=0 2024-06-06T04:31:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.209+0000 7f353eff5640 10 client.15372 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.213+0000 7f353eff5640 1 -- 172.21.15.45:0/2934927078 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a1467afc20 con 0x55a1468bfb20 2024-06-06T04:31:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.213+0000 7f353eff5640 10 client.15372 handle_command_reply: tid=0 2024-06-06T04:31:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.213+0000 7f3564211740 1 client.15372 shutdown 2024-06-06T04:31:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.213+0000 7f3564211740 1 -- 172.21.15.45:0/2934927078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a146844b70 msgr2=0x55a1468bddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.213+0000 7f3564211740 1 --2- 172.21.15.45:0/2934927078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a146844b70 0x55a1468bddc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3550005980 tx=0x7f355000bf30 comp rx=0 tx=0).stop 2024-06-06T04:31:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.213+0000 7f3564211740 1 -- 172.21.15.45:0/2934927078 shutdown_connections 2024-06-06T04:31:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.213+0000 7f3564211740 1 --2- 172.21.15.45:0/2934927078 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55a1468bfb20 0x55a1468c1f40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.213+0000 7f3564211740 1 --2- 172.21.15.45:0/2934927078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a14683fa00 0x55a14684dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.213+0000 7f3564211740 1 --2- 172.21.15.45:0/2934927078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a146844b70 0x55a1468bddc0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.213+0000 7f353eff5640 0 client.15372 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:31:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.213+0000 7f3564211740 1 --2- 172.21.15.45:0/2934927078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a14683f010 0x55a14684d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.213+0000 7f3564211740 1 -- 172.21.15.45:0/2934927078 >> 172.21.15.45:0/2934927078 conn(0x55a14683a680 msgr2=0x55a146840be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.213+0000 7f3564211740 1 -- 172.21.15.45:0/2934927078 shutdown_connections 2024-06-06T04:31:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.213+0000 7f3564211740 1 -- 172.21.15.45:0/2934927078 wait complete. 2024-06-06T04:31:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.213+0000 7f3564211740 20 client.15372 trim_cache size 0 max 16384 2024-06-06T04:31:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-06-06T04:31:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-06-06T04:31:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-06-06T04:31:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.214+0000 7f356142a640 1 -- 172.21.15.45:0/3266590981 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f35440a2ef0 msgr2=0x7f35440a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.214+0000 7f356142a640 1 --2- 172.21.15.45:0/3266590981 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f35440a2ef0 0x7f35440a53b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f35540bcb60 tx=0x7f355408ff20 comp rx=0 tx=0).stop 2024-06-06T04:31:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.214+0000 7f356142a640 1 -- 172.21.15.45:0/3266590981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c102560 msgr2=0x7f355c1a1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:31:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.214+0000 7f356142a640 1 --2- 172.21.15.45:0/3266590981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c102560 0x7f355c1a1210 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3554009dc0 tx=0x7f3554003ba0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.214+0000 7f356142a640 1 -- 172.21.15.45:0/3266590981 shutdown_connections 2024-06-06T04:31:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.214+0000 7f356142a640 1 --2- 172.21.15.45:0/3266590981 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f35440a2ef0 0x7f35440a53b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.214+0000 7f356142a640 1 --2- 172.21.15.45:0/3266590981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c102560 0x7f355c1a1210 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.214+0000 7f356142a640 1 --2- 172.21.15.45:0/3266590981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f355c101ba0 0x7f355c19a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.214+0000 7f356142a640 1 --2- 172.21.15.45:0/3266590981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f355c1009b0 0x7f355c199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:31:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.214+0000 7f356142a640 1 -- 172.21.15.45:0/3266590981 >> 172.21.15.45:0/3266590981 conn(0x7f355c0fbf20 msgr2=0x7f355c0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:31:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.214+0000 7f356142a640 1 -- 172.21.15.45:0/3266590981 shutdown_connections 2024-06-06T04:31:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:31:57.214+0000 7f356142a640 1 -- 172.21.15.45:0/3266590981 wait complete. 2024-06-06T04:31:58.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:31:58 smithi086 ceph-mon[36053]: pgmap v446: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-06-06T04:31:58.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:31:58 smithi154 ceph-mon[36428]: pgmap v446: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-06-06T04:31:58.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:31:58 smithi045 ceph-mon[32299]: pgmap v446: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-06-06T04:32:00.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:00 smithi086 ceph-mon[36053]: pgmap v447: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-06T04:32:00.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:00 smithi154 ceph-mon[36428]: pgmap v447: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-06T04:32:00.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:00 smithi045 ceph-mon[32299]: pgmap v447: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-06T04:32:02.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:02 smithi086 ceph-mon[36053]: pgmap v448: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T04:32:02.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:02 smithi154 ceph-mon[36428]: pgmap v448: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T04:32:02.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:02 smithi045 ceph-mon[32299]: pgmap v448: 129 pgs: 129 active+clean; 664 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T04:32:04.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:04 smithi086 ceph-mon[36053]: pgmap v449: 129 pgs: 129 active+clean; 665 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:04.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:32:04.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:32:04.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:32:04.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:32:04.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:04 smithi154 ceph-mon[36428]: pgmap v449: 129 pgs: 129 active+clean; 665 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:04.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:32:04.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:32:04.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:32:04.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:32:04.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:04 smithi045 ceph-mon[32299]: pgmap v449: 129 pgs: 129 active+clean; 665 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:04.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:32:04.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:32:04.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:32:04.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:32:06.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:06 smithi086 ceph-mon[36053]: pgmap v450: 129 pgs: 129 active+clean; 665 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T04:32:06.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:06 smithi154 ceph-mon[36428]: pgmap v450: 129 pgs: 129 active+clean; 665 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T04:32:06.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:06 smithi045 ceph-mon[32299]: pgmap v450: 129 pgs: 129 active+clean; 665 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T04:32:06.711 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T04:32:06.967 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:32:08.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:08 smithi086 ceph-mon[36053]: pgmap v451: 129 pgs: 129 active+clean; 665 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:08.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:08 smithi154 ceph-mon[36428]: pgmap v451: 129 pgs: 129 active+clean; 665 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:08.433 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:08 smithi045 ceph-mon[32299]: pgmap v451: 129 pgs: 129 active+clean; 665 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:08.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.792+0000 7f82c1e7d700 1 -- 172.21.15.45:0/2289009659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1024e0 msgr2=0x7f82bc102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.792+0000 7f82c1e7d700 1 --2- 172.21.15.45:0/2289009659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1024e0 0x7f82bc102960 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009990 tx=0x7f82b0009ca0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.792+0000 7f82c1e7d700 1 -- 172.21.15.45:0/2289009659 shutdown_connections 2024-06-06T04:32:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.792+0000 7f82c1e7d700 1 --2- 172.21.15.45:0/2289009659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82bc102ea0 0x7f82bc10f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.792+0000 7f82c1e7d700 1 --2- 172.21.15.45:0/2289009659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82bc1085f0 0x7f82bc1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.792+0000 7f82c1e7d700 1 --2- 172.21.15.45:0/2289009659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1024e0 0x7f82bc102960 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.792+0000 7f82c1e7d700 1 -- 172.21.15.45:0/2289009659 >> 172.21.15.45:0/2289009659 conn(0x7f82bc079640 msgr2=0x7f82bc079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.792+0000 7f82c1e7d700 1 -- 172.21.15.45:0/2289009659 shutdown_connections 2024-06-06T04:32:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.792+0000 7f82c1e7d700 1 -- 172.21.15.45:0/2289009659 wait complete. 2024-06-06T04:32:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.793+0000 7f82c1e7d700 1 Processor -- start 2024-06-06T04:32:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.793+0000 7f82c1e7d700 1 -- start start 2024-06-06T04:32:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.793+0000 7f82c1e7d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82bc1024e0 0x7f82bc19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.793+0000 7f82c1e7d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82bc102ea0 0x7f82bc19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.793+0000 7f82c1e7d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1085f0 0x7f82bc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.793+0000 7f82c1e7d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc112640 con 0x7f82bc1085f0 2024-06-06T04:32:08.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.793+0000 7f82c1e7d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc1124c0 con 0x7f82bc102ea0 2024-06-06T04:32:08.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.793+0000 7f82c1e7d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc1127c0 con 0x7f82bc1024e0 2024-06-06T04:32:08.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.793+0000 7f82bbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1085f0 0x7f82bc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.793+0000 7f82bbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1085f0 0x7f82bc1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48666/0 (socket says 172.21.15.45:48666) 2024-06-06T04:32:08.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.793+0000 7f82bbfff700 1 -- 172.21.15.45:0/2378400715 learned_addr learned my addr 172.21.15.45:0/2378400715 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:08.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.793+0000 7f82baffd700 1 --2- 172.21.15.45:0/2378400715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82bc102ea0 0x7f82bc19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.794+0000 7f82bb7fe700 1 --2- 172.21.15.45:0/2378400715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82bc1024e0 0x7f82bc19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.794+0000 7f82baffd700 1 -- 172.21.15.45:0/2378400715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82bc1024e0 msgr2=0x7f82bc19e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:08.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.794+0000 7f82baffd700 1 --2- 172.21.15.45:0/2378400715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82bc1024e0 0x7f82bc19e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.794+0000 7f82baffd700 1 -- 172.21.15.45:0/2378400715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1085f0 msgr2=0x7f82bc1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.794+0000 7f82baffd700 1 --2- 172.21.15.45:0/2378400715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1085f0 0x7f82bc1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.794+0000 7f82baffd700 1 -- 172.21.15.45:0/2378400715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc199040 con 0x7f82bc102ea0 2024-06-06T04:32:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.794+0000 7f82bbfff700 1 --2- 172.21.15.45:0/2378400715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1085f0 0x7f82bc1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.794+0000 7f82baffd700 1 --2- 172.21.15.45:0/2378400715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82bc102ea0 0x7f82bc19ec60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82b00059d0 tx=0x7f82b00047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.794+0000 7f82b8ff9700 1 -- 172.21.15.45:0/2378400715 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b0027070 con 0x7f82bc102ea0 2024-06-06T04:32:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.794+0000 7f82c1e7d700 1 -- 172.21.15.45:0/2378400715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82bc1992d0 con 0x7f82bc102ea0 2024-06-06T04:32:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.794+0000 7f82b8ff9700 1 -- 172.21.15.45:0/2378400715 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82b0003f10 con 0x7f82bc102ea0 2024-06-06T04:32:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.794+0000 7f82b8ff9700 1 -- 172.21.15.45:0/2378400715 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b0022ba0 con 0x7f82bc102ea0 2024-06-06T04:32:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.794+0000 7f82c1e7d700 1 -- 172.21.15.45:0/2378400715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82bc1a55f0 con 0x7f82bc102ea0 2024-06-06T04:32:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.797+0000 7f82b8ff9700 1 -- 172.21.15.45:0/2378400715 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f82b002a070 con 0x7f82bc102ea0 2024-06-06T04:32:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.797+0000 7f82b8ff9700 1 --2- 172.21.15.45:0/2378400715 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f82a40a2d30 0x7f82a40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.798+0000 7f82bb7fe700 1 --2- 172.21.15.45:0/2378400715 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f82a40a2d30 0x7f82a40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.798+0000 7f82b8ff9700 1 -- 172.21.15.45:0/2378400715 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f82b00d0400 con 0x7f82bc102ea0 2024-06-06T04:32:08.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.798+0000 7f82bb7fe700 1 --2- 172.21.15.45:0/2378400715 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f82a40a2d30 0x7f82a40a51f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f82bc199a20 tx=0x7f82ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.798+0000 7f82cc700b80 1 Processor -- start 2024-06-06T04:32:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.798+0000 7f82cc700b80 1 -- start start 2024-06-06T04:32:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82cc700b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8640 0x5639cbed8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82cc700b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8f60 0x5639cbed29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82cc700b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbed3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82cc700b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5639cbe24810 con 0x5639cbed8640 2024-06-06T04:32:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82cc700b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5639cbdcc720 con 0x5639cbed2f20 2024-06-06T04:32:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82cc700b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5639cbe207d0 con 0x5639cbed8f60 2024-06-06T04:32:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82bb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8640 0x5639cbed8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82bb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8640 0x5639cbed8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48674/0 (socket says 172.21.15.45:48674) 2024-06-06T04:32:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82bb7fe700 1 -- 172.21.15.45:0/2574796430 learned_addr learned my addr 172.21.15.45:0/2574796430 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82bbfff700 1 --2- 172.21.15.45:0/2574796430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbed3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82baffd700 1 --2- 172.21.15.45:0/2574796430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8f60 0x5639cbed29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82bb7fe700 1 -- 172.21.15.45:0/2574796430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8f60 msgr2=0x5639cbed29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82bb7fe700 1 --2- 172.21.15.45:0/2574796430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8f60 0x5639cbed29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82bb7fe700 1 -- 172.21.15.45:0/2574796430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 msgr2=0x5639cbed3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82bb7fe700 1 --2- 172.21.15.45:0/2574796430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbed3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82bb7fe700 1 -- 172.21.15.45:0/2574796430 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639cbdc2f50 con 0x5639cbed8640 2024-06-06T04:32:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.799+0000 7f82bb7fe700 1 --2- 172.21.15.45:0/2574796430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8640 0x5639cbed8a20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f82ac010f70 tx=0x7f82ac013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8947f27f83564d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.800+0000 7f82a1ffb700 1 -- 172.21.15.45:0/2574796430 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac00be60 con 0x5639cbed8640 2024-06-06T04:32:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.800+0000 7f82a1ffb700 1 -- 172.21.15.45:0/2574796430 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82ac011db0 con 0x5639cbed8640 2024-06-06T04:32:08.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.800+0000 7f82a1ffb700 1 -- 172.21.15.45:0/2574796430 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac01c6b0 con 0x5639cbed8640 2024-06-06T04:32:08.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.801+0000 7f82cc700b80 1 -- 172.21.15.45:0/2574796430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8640 msgr2=0x5639cbed8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:08.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.801+0000 7f82cc700b80 1 --2- 172.21.15.45:0/2574796430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8640 0x5639cbed8a20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f82ac010f70 tx=0x7f82ac013000 comp rx=0 tx=0).stop 2024-06-06T04:32:08.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.801+0000 7f82cc700b80 1 -- 172.21.15.45:0/2574796430 shutdown_connections 2024-06-06T04:32:08.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.801+0000 7f82cc700b80 1 --2- 172.21.15.45:0/2574796430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbed3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.801+0000 7f82cc700b80 1 --2- 172.21.15.45:0/2574796430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8640 0x5639cbed8a20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.801+0000 7f82cc700b80 1 --2- 172.21.15.45:0/2574796430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8f60 0x5639cbed29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.801+0000 7f82cc700b80 1 -- 172.21.15.45:0/2574796430 >> 172.21.15.45:0/2574796430 conn(0x5639cbdfdcc0 msgr2=0x5639cbecd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:08.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.801+0000 7f82cc700b80 1 -- 172.21.15.45:0/2574796430 shutdown_connections 2024-06-06T04:32:08.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.801+0000 7f82cc700b80 1 -- 172.21.15.45:0/2574796430 wait complete. 2024-06-06T04:32:08.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.801+0000 7f82cc700b80 1 -- start start 2024-06-06T04:32:08.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.802+0000 7f82cc700b80 1 Processor -- start 2024-06-06T04:32:08.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.802+0000 7f82cc700b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbedecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.802+0000 7f82cc700b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8640 0x5639cbedbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.802+0000 7f82cc700b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8f60 0x5639cbedc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.802+0000 7f82cc700b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5639cbedf9d0 con 0x5639cbed8f60 2024-06-06T04:32:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.802+0000 7f82cc700b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5639cbdc4bf0 con 0x5639cbed2f20 2024-06-06T04:32:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.802+0000 7f82cc700b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5639cbedfb50 con 0x5639cbed8640 2024-06-06T04:32:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.802+0000 7f82bbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8f60 0x5639cbedc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.802+0000 7f82bb7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbedecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.802+0000 7f82bb7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbedecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:40328/0 (socket says 172.21.15.45:40328) 2024-06-06T04:32:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.802+0000 7f82bb7fe700 1 -- 172.21.15.45:0/3313866600 learned_addr learned my addr 172.21.15.45:0/3313866600 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.802+0000 7f82bbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8f60 0x5639cbedc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48682/0 (socket says 172.21.15.45:48682) 2024-06-06T04:32:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.802+0000 7f82cc700b80 10 client.0 _ll_register_callbacks cb 0x5639cbdbe920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.802+0000 7f82baffd700 1 --2- 172.21.15.45:0/3313866600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8640 0x5639cbedbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.802+0000 7f82bbfff700 1 -- 172.21.15.45:0/3313866600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8640 msgr2=0x5639cbedbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.803+0000 7f82bbfff700 1 --2- 172.21.15.45:0/3313866600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8640 0x5639cbedbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.803+0000 7f82bbfff700 1 -- 172.21.15.45:0/3313866600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 msgr2=0x5639cbedecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.803+0000 7f82bbfff700 1 --2- 172.21.15.45:0/3313866600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbedecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.803+0000 7f82bbfff700 1 -- 172.21.15.45:0/3313866600 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639cbd738d0 con 0x5639cbed8f60 2024-06-06T04:32:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.803+0000 7f82bb7fe700 1 --2- 172.21.15.45:0/3313866600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbedecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.803+0000 7f82baffd700 1 --2- 172.21.15.45:0/3313866600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8640 0x5639cbedbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.803+0000 7f82bbfff700 1 --2- 172.21.15.45:0/3313866600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8f60 0x5639cbedc500 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f82b4007ad0 tx=0x7f82b4007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d2b634aa28e82f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.803+0000 7f829affd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.803+0000 7f829affd700 1 -- 172.21.15.45:0/3313866600 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b4004bb0 con 0x5639cbed8f60 2024-06-06T04:32:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.803+0000 7f829affd700 1 -- 172.21.15.45:0/3313866600 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f82b4004d50 con 0x5639cbed8f60 2024-06-06T04:32:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.803+0000 7f82cc700b80 1 -- 172.21.15.45:0/3313866600 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82ac011db0 con 0x5639cbed8f60 2024-06-06T04:32:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.803+0000 7f829affd700 1 -- 172.21.15.45:0/3313866600 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82b402c050 con 0x5639cbed8f60 2024-06-06T04:32:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.803+0000 7f82cc700b80 10 client.15384 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.803+0000 7f82cc700b80 1 -- 172.21.15.45:0/3313866600 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5639cbeea330 con 0x5639cbed8f60 2024-06-06T04:32:08.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.804+0000 7f829affd700 1 -- 172.21.15.45:0/3313866600 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82ac011db0 con 0x5639cbed8f60 2024-06-06T04:32:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.804+0000 7f829affd700 1 -- 172.21.15.45:0/3313866600 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f82b4023960 con 0x5639cbed8f60 2024-06-06T04:32:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.804+0000 7f82cc700b80 10 client.15384 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.804+0000 7f82cc700b80 10 client.15384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.804+0000 7f82cc700b80 20 client.15384 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.804+0000 7f82cc700b80 1 --2- 172.21.15.45:0/3313866600 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5639cbeec970 0x5639cbeeed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.804+0000 7f82cc700b80 4 client.15384 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:32:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.804+0000 7f82cc700b80 1 -- 172.21.15.45:0/3313866600 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639cbe31d10 con 0x5639cbeec970 2024-06-06T04:32:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.804+0000 7f82bb7fe700 1 --2- 172.21.15.45:0/3313866600 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5639cbeec970 0x5639cbeeed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.805+0000 7f82bb7fe700 1 --2- 172.21.15.45:0/3313866600 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5639cbeec970 0x5639cbeeed60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43e1bd382315a66a server_cookie=5d46b2da23f6e8f0 in_seq=0 out_seq=0 2024-06-06T04:32:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.805+0000 7f829affd700 10 client.15384 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.805+0000 7f829affd700 1 -- 172.21.15.45:0/3313866600 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5639cbe31d10 con 0x5639cbeec970 2024-06-06T04:32:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.805+0000 7f829affd700 10 client.15384 handle_command_reply: tid=0 2024-06-06T04:32:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.805+0000 7f82cc700b80 1 client.15384 shutdown 2024-06-06T04:32:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.805+0000 7f82cc700b80 1 -- 172.21.15.45:0/3313866600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8f60 msgr2=0x5639cbedc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:08.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.805+0000 7f82cc700b80 1 --2- 172.21.15.45:0/3313866600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8f60 0x5639cbedc500 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f82b4007ad0 tx=0x7f82b4007de0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.806+0000 7f82cc700b80 1 -- 172.21.15.45:0/3313866600 shutdown_connections 2024-06-06T04:32:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.806+0000 7f82cc700b80 1 --2- 172.21.15.45:0/3313866600 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5639cbeec970 0x5639cbeeed60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.806+0000 7f82cc700b80 1 --2- 172.21.15.45:0/3313866600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8f60 0x5639cbedc500 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.806+0000 7f82cc700b80 1 --2- 172.21.15.45:0/3313866600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbedecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.806+0000 7f82cc700b80 1 --2- 172.21.15.45:0/3313866600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8640 0x5639cbedbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.806+0000 7f829affd700 0 client.15384 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.806+0000 7f82cc700b80 1 -- 172.21.15.45:0/3313866600 >> 172.21.15.45:0/3313866600 conn(0x5639cbdfdcc0 msgr2=0x5639cbecd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.806+0000 7f82cc700b80 1 -- 172.21.15.45:0/3313866600 shutdown_connections 2024-06-06T04:32:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.806+0000 7f82cc700b80 1 -- 172.21.15.45:0/3313866600 wait complete. 2024-06-06T04:32:08.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.806+0000 7f82cc700b80 20 client.15384 trim_cache size 0 max 16384 2024-06-06T04:32:08.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.827+0000 7f82cc700b80 1 Processor -- start 2024-06-06T04:32:08.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.827+0000 7f82cc700b80 1 -- start start 2024-06-06T04:32:08.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.827+0000 7f82cc700b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbed29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.827+0000 7f82cc700b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8640 0x5639cbed6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.827+0000 7f82cc700b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8f60 0x5639cbec7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.827+0000 7f82cc700b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc1ae850 con 0x5639cbed8640 2024-06-06T04:32:08.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.827+0000 7f82cc700b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc1ac0e0 con 0x5639cbed2f20 2024-06-06T04:32:08.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.827+0000 7f82cc700b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc1a9f30 con 0x5639cbed8f60 2024-06-06T04:32:08.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.827+0000 7f82baffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8640 0x5639cbed6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.827+0000 7f82baffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8640 0x5639cbed6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48690/0 (socket says 172.21.15.45:48690) 2024-06-06T04:32:08.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.827+0000 7f82baffd700 1 -- 172.21.15.45:0/4208030124 learned_addr learned my addr 172.21.15.45:0/4208030124 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:08.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.827+0000 7f82bb7fe700 1 --2- 172.21.15.45:0/4208030124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbed29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.828+0000 7f82baffd700 1 -- 172.21.15.45:0/4208030124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8f60 msgr2=0x5639cbec7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:08.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.828+0000 7f82baffd700 1 --2- 172.21.15.45:0/4208030124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8f60 0x5639cbec7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.828+0000 7f82baffd700 1 -- 172.21.15.45:0/4208030124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 msgr2=0x5639cbed29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:08.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.828+0000 7f82baffd700 1 --2- 172.21.15.45:0/4208030124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbed29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.828+0000 7f82baffd700 1 -- 172.21.15.45:0/4208030124 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639cbeda530 con 0x5639cbed8640 2024-06-06T04:32:08.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.828+0000 7f82bb7fe700 1 --2- 172.21.15.45:0/4208030124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbed29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:08.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.828+0000 7f82baffd700 1 --2- 172.21.15.45:0/4208030124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8640 0x5639cbed6730 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f82b0005f50 tx=0x7f82b001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564f402380bc13e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:08.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.828+0000 7f82a17fa700 1 -- 172.21.15.45:0/4208030124 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b009ceb0 con 0x5639cbed8640 2024-06-06T04:32:08.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.828+0000 7f82a17fa700 1 -- 172.21.15.45:0/4208030124 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82b00040c0 con 0x5639cbed8640 2024-06-06T04:32:08.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.828+0000 7f82a17fa700 1 -- 172.21.15.45:0/4208030124 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b0098e20 con 0x5639cbed8640 2024-06-06T04:32:08.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.829+0000 7f82cc700b80 1 -- 172.21.15.45:0/4208030124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8640 msgr2=0x5639cbed6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:08.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.829+0000 7f82cc700b80 1 --2- 172.21.15.45:0/4208030124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8640 0x5639cbed6730 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f82b0005f50 tx=0x7f82b001a040 comp rx=0 tx=0).stop 2024-06-06T04:32:08.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.829+0000 7f82cc700b80 1 -- 172.21.15.45:0/4208030124 shutdown_connections 2024-06-06T04:32:08.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.829+0000 7f82cc700b80 1 --2- 172.21.15.45:0/4208030124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8f60 0x5639cbec7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.829+0000 7f82cc700b80 1 --2- 172.21.15.45:0/4208030124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbed29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.829+0000 7f82cc700b80 1 --2- 172.21.15.45:0/4208030124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8640 0x5639cbed6730 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.829+0000 7f82cc700b80 1 -- 172.21.15.45:0/4208030124 >> 172.21.15.45:0/4208030124 conn(0x5639cbdfdcc0 msgr2=0x5639cbedd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:08.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.829+0000 7f82cc700b80 1 -- 172.21.15.45:0/4208030124 shutdown_connections 2024-06-06T04:32:08.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.829+0000 7f82cc700b80 1 -- 172.21.15.45:0/4208030124 wait complete. 2024-06-06T04:32:08.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.829+0000 7f82cc700b80 1 -- start start 2024-06-06T04:32:08.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82cc700b80 1 Processor -- start 2024-06-06T04:32:08.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82cc700b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbee4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82cc700b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8640 0x5639cbee5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82cc700b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8f60 0x5639cbee1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82cc700b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc1aa5a0 con 0x5639cbed8f60 2024-06-06T04:32:08.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82cc700b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc1aba10 con 0x5639cbed2f20 2024-06-06T04:32:08.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82cc700b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc1aac70 con 0x5639cbed8640 2024-06-06T04:32:08.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82cc700b80 10 client.0 _ll_register_callbacks cb 0x5639cbdbe920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:08.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82bbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8f60 0x5639cbee1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82baffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8640 0x5639cbee5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82bbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8f60 0x5639cbee1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48704/0 (socket says 172.21.15.45:48704) 2024-06-06T04:32:08.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82bb7fe700 1 --2- 172.21.15.45:0/1254477830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbee4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82bbfff700 1 -- 172.21.15.45:0/1254477830 learned_addr learned my addr 172.21.15.45:0/1254477830 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:08.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82baffd700 1 -- 172.21.15.45:0/1254477830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 msgr2=0x5639cbee4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:08.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82baffd700 1 --2- 172.21.15.45:0/1254477830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbee4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82baffd700 1 -- 172.21.15.45:0/1254477830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8f60 msgr2=0x5639cbee1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:08.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82baffd700 1 --2- 172.21.15.45:0/1254477830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8f60 0x5639cbee1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82baffd700 1 -- 172.21.15.45:0/1254477830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639cbee2320 con 0x5639cbed8640 2024-06-06T04:32:08.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82bbfff700 1 --2- 172.21.15.45:0/1254477830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8f60 0x5639cbee1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:08.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.830+0000 7f82bb7fe700 1 --2- 172.21.15.45:0/1254477830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbee4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:08.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.831+0000 7f82baffd700 1 --2- 172.21.15.45:0/1254477830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8640 0x5639cbee5350 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f82b009c970 tx=0x7f82b0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e106264d31643236 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:08.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.831+0000 7f829affd700 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-06-06T04:32:08.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.831+0000 7f829affd700 1 -- 172.21.15.45:0/1254477830 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b00ba060 con 0x5639cbed8640 2024-06-06T04:32:08.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.831+0000 7f82cc700b80 1 -- 172.21.15.45:0/1254477830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82b00040c0 con 0x5639cbed8640 2024-06-06T04:32:08.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.831+0000 7f829affd700 1 -- 172.21.15.45:0/1254477830 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f82b00a4f00 con 0x5639cbed8640 2024-06-06T04:32:08.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.831+0000 7f829affd700 1 -- 172.21.15.45:0/1254477830 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82b00b4070 con 0x5639cbed8640 2024-06-06T04:32:08.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.831+0000 7f82cc700b80 10 client.25030 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:08.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.831+0000 7f82cc700b80 1 -- 172.21.15.45:0/1254477830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5639cbf57de0 con 0x5639cbed8640 2024-06-06T04:32:08.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.831+0000 7f829affd700 1 -- 172.21.15.45:0/1254477830 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82b00040c0 con 0x5639cbed8640 2024-06-06T04:32:08.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.832+0000 7f829affd700 1 -- 172.21.15.45:0/1254477830 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f82b009bd20 con 0x5639cbed8640 2024-06-06T04:32:08.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.832+0000 7f82cc700b80 10 client.25030 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:08.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.832+0000 7f82cc700b80 10 client.25030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:08.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.832+0000 7f82cc700b80 20 client.25030 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:08.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.832+0000 7f82cc700b80 1 --2- 172.21.15.45:0/1254477830 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5639cbf58420 0x5639cbf5a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:08.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.832+0000 7f82cc700b80 4 client.25030 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T04:32:08.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.832+0000 7f82cc700b80 1 -- 172.21.15.45:0/1254477830 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5639cbe2c290 con 0x5639cbf58420 2024-06-06T04:32:08.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.832+0000 7f82bb7fe700 1 --2- 172.21.15.45:0/1254477830 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5639cbf58420 0x5639cbf5a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:08.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.832+0000 7f82bb7fe700 1 --2- 172.21.15.45:0/1254477830 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5639cbf58420 0x5639cbf5a840 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d6730952db2a6a8 server_cookie=e33422105a83c3d0 in_seq=0 out_seq=0 2024-06-06T04:32:08.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.832+0000 7f829affd700 10 client.25030 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:08.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.833+0000 7f829affd700 1 -- 172.21.15.45:0/1254477830 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5639cbe2c290 con 0x5639cbf58420 2024-06-06T04:32:08.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.833+0000 7f829affd700 10 client.25030 handle_command_reply: tid=0 2024-06-06T04:32:08.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.833+0000 7f82cc700b80 1 client.25030 shutdown 2024-06-06T04:32:08.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.833+0000 7f82cc700b80 1 -- 172.21.15.45:0/1254477830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8640 msgr2=0x5639cbee5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:08.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.833+0000 7f82cc700b80 1 --2- 172.21.15.45:0/1254477830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8640 0x5639cbee5350 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f82b009c970 tx=0x7f82b0009310 comp rx=0 tx=0).stop 2024-06-06T04:32:08.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.833+0000 7f82cc700b80 1 -- 172.21.15.45:0/1254477830 shutdown_connections 2024-06-06T04:32:08.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.833+0000 7f82cc700b80 1 --2- 172.21.15.45:0/1254477830 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5639cbf58420 0x5639cbf5a840 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.833+0000 7f82cc700b80 1 --2- 172.21.15.45:0/1254477830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639cbed8f60 0x5639cbee1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.833+0000 7f82cc700b80 1 --2- 172.21.15.45:0/1254477830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639cbed2f20 0x5639cbee4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.833+0000 7f82cc700b80 1 --2- 172.21.15.45:0/1254477830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cbed8640 0x5639cbee5350 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.833+0000 7f82cc700b80 1 -- 172.21.15.45:0/1254477830 >> 172.21.15.45:0/1254477830 conn(0x5639cbdfdcc0 msgr2=0x5639cbede080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:08.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.833+0000 7f829affd700 0 client.25030 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:08.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.833+0000 7f82cc700b80 1 -- 172.21.15.45:0/1254477830 shutdown_connections 2024-06-06T04:32:08.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.833+0000 7f82cc700b80 1 -- 172.21.15.45:0/1254477830 wait complete. 2024-06-06T04:32:08.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.833+0000 7f82cc700b80 20 client.25030 trim_cache size 0 max 16384 2024-06-06T04:32:08.839 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:32:08.839 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-06-06T04:32:08.839 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-06-06T04:32:08.839 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:32:08.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.834+0000 7f82c1e7d700 1 -- 172.21.15.45:0/2378400715 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f82a40a2d30 msgr2=0x7f82a40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:08.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.834+0000 7f82c1e7d700 1 --2- 172.21.15.45:0/2378400715 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f82a40a2d30 0x7f82a40a51f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f82bc199a20 tx=0x7f82ac00b410 comp rx=0 tx=0).stop 2024-06-06T04:32:08.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.834+0000 7f82c1e7d700 1 -- 172.21.15.45:0/2378400715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82bc102ea0 msgr2=0x7f82bc19ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:08.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.834+0000 7f82c1e7d700 1 --2- 172.21.15.45:0/2378400715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82bc102ea0 0x7f82bc19ec60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82b00059d0 tx=0x7f82b00047b0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.834+0000 7f82c1e7d700 1 -- 172.21.15.45:0/2378400715 shutdown_connections 2024-06-06T04:32:08.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.834+0000 7f82c1e7d700 1 --2- 172.21.15.45:0/2378400715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1085f0 0x7f82bc1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.834+0000 7f82c1e7d700 1 --2- 172.21.15.45:0/2378400715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82bc1024e0 0x7f82bc19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.834+0000 7f82c1e7d700 1 --2- 172.21.15.45:0/2378400715 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f82a40a2d30 0x7f82a40a51f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.834+0000 7f82c1e7d700 1 --2- 172.21.15.45:0/2378400715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82bc102ea0 0x7f82bc19ec60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:08.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.834+0000 7f82c1e7d700 1 -- 172.21.15.45:0/2378400715 >> 172.21.15.45:0/2378400715 conn(0x7f82bc079640 msgr2=0x7f82bc1010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:08.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.834+0000 7f82c1e7d700 1 -- 172.21.15.45:0/2378400715 shutdown_connections 2024-06-06T04:32:08.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:08.834+0000 7f82c1e7d700 1 -- 172.21.15.45:0/2378400715 wait complete. 2024-06-06T04:32:09.700 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-06T04:32:09.700 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:32:09.989 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:32:10.259 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:10 smithi045 ceph-mon[32299]: pgmap v452: 129 pgs: 129 active+clean; 665 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.314+0000 7f03e52d4640 1 -- 172.21.15.45:0/659194785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e01002d0 msgr2=0x7f03e01006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.314+0000 7f03e52d4640 1 --2- 172.21.15.45:0/659194785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e01002d0 0x7f03e01006f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009900 tx=0x7f03d002a410 comp rx=0 tx=0).stop 2024-06-06T04:32:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.314+0000 7f03e52d4640 1 -- 172.21.15.45:0/659194785 shutdown_connections 2024-06-06T04:32:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.314+0000 7f03e52d4640 1 --2- 172.21.15.45:0/659194785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03e0100c30 0x7f03e0101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.314+0000 7f03e52d4640 1 --2- 172.21.15.45:0/659194785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e01015d0 0x7f03e0101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.314+0000 7f03e52d4640 1 --2- 172.21.15.45:0/659194785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e01002d0 0x7f03e01006f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.314+0000 7f03e52d4640 1 -- 172.21.15.45:0/659194785 >> 172.21.15.45:0/659194785 conn(0x7f03e00fbe70 msgr2=0x7f03e00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.314+0000 7f03e52d4640 1 -- 172.21.15.45:0/659194785 shutdown_connections 2024-06-06T04:32:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.314+0000 7f03e52d4640 1 -- 172.21.15.45:0/659194785 wait complete. 2024-06-06T04:32:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.315+0000 7f03e52d4640 1 Processor -- start 2024-06-06T04:32:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.315+0000 7f03e52d4640 1 -- start start 2024-06-06T04:32:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.315+0000 7f03e52d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03e01002d0 0x7f03e0199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.315+0000 7f03e52d4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e0100c30 0x7f03e019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.315+0000 7f03e52d4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e01015d0 0x7f03e01a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.315+0000 7f03e52d4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03e010f710 con 0x7f03e0100c30 2024-06-06T04:32:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.315+0000 7f03e52d4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03e010f590 con 0x7f03e01002d0 2024-06-06T04:32:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.315+0000 7f03e52d4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f03e010f890 con 0x7f03e01015d0 2024-06-06T04:32:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.315+0000 7f03de7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e0100c30 0x7f03e019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.315+0000 7f03de7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e0100c30 0x7f03e019a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48712/0 (socket says 172.21.15.45:48712) 2024-06-06T04:32:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.315+0000 7f03de7fc640 1 -- 172.21.15.45:0/710250478 learned_addr learned my addr 172.21.15.45:0/710250478 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.315+0000 7f03deffd640 1 --2- 172.21.15.45:0/710250478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03e01002d0 0x7f03e0199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.315+0000 7f03df7fe640 1 --2- 172.21.15.45:0/710250478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e01015d0 0x7f03e01a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.316+0000 7f03de7fc640 1 -- 172.21.15.45:0/710250478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e01015d0 msgr2=0x7f03e01a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.316+0000 7f03de7fc640 1 --2- 172.21.15.45:0/710250478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e01015d0 0x7f03e01a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.316+0000 7f03de7fc640 1 -- 172.21.15.45:0/710250478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03e01002d0 msgr2=0x7f03e0199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.316+0000 7f03de7fc640 1 --2- 172.21.15.45:0/710250478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03e01002d0 0x7f03e0199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.316+0000 7f03de7fc640 1 -- 172.21.15.45:0/710250478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e01a19b0 con 0x7f03e0100c30 2024-06-06T04:32:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.316+0000 7f03de7fc640 1 --2- 172.21.15.45:0/710250478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e0100c30 0x7f03e019a2e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f03d400d950 tx=0x7f03d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.316+0000 7f03c7fff640 1 -- 172.21.15.45:0/710250478 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4017070 con 0x7f03e0100c30 2024-06-06T04:32:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.316+0000 7f03c7fff640 1 -- 172.21.15.45:0/710250478 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03d4009c30 con 0x7f03e0100c30 2024-06-06T04:32:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.316+0000 7f03e52d4640 1 -- 172.21.15.45:0/710250478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e01a1ca0 con 0x7f03e0100c30 2024-06-06T04:32:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.316+0000 7f03c7fff640 1 -- 172.21.15.45:0/710250478 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d400f620 con 0x7f03e0100c30 2024-06-06T04:32:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.316+0000 7f03e52d4640 1 -- 172.21.15.45:0/710250478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e01a21a0 con 0x7f03e0100c30 2024-06-06T04:32:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.319+0000 7f03c7fff640 1 -- 172.21.15.45:0/710250478 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f03d40109a0 con 0x7f03e0100c30 2024-06-06T04:32:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.319+0000 7f03e7fe5740 1 Processor -- start 2024-06-06T04:32:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.319+0000 7f03e7fe5740 1 -- start start 2024-06-06T04:32:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.319+0000 7f03e7fe5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809010 0x564eb5809430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.319+0000 7f03e7fe5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809a00 0x564eb580e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.319+0000 7f03e7fe5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb580eb70 0x564eb5811000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.319+0000 7f03e7fe5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564eb576c4f0 con 0x564eb5809010 2024-06-06T04:32:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.319+0000 7f03e7fe5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564eb570a870 con 0x564eb5809a00 2024-06-06T04:32:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.319+0000 7f03e7fe5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564eb57684b0 con 0x564eb580eb70 2024-06-06T04:32:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.320+0000 7f03df7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809010 0x564eb5809430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.320+0000 7f03df7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809010 0x564eb5809430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48726/0 (socket says 172.21.15.45:48726) 2024-06-06T04:32:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.320+0000 7f03df7fe640 1 -- 172.21.15.45:0/3047778572 learned_addr learned my addr 172.21.15.45:0/3047778572 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.320+0000 7f03deffd640 1 --2- 172.21.15.45:0/3047778572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809a00 0x564eb580e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.320+0000 7f03de7fc640 1 --2- 172.21.15.45:0/3047778572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb580eb70 0x564eb5811000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.320+0000 7f03c7fff640 1 --2- 172.21.15.45:0/710250478 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f03c80a2cd0 0x7f03c80a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.320+0000 7f03deffd640 1 -- 172.21.15.45:0/3047778572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb580eb70 msgr2=0x564eb5811000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.320+0000 7f03deffd640 1 --2- 172.21.15.45:0/3047778572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb580eb70 0x564eb5811000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.320+0000 7f03deffd640 1 -- 172.21.15.45:0/3047778572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809010 msgr2=0x564eb5809430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.320+0000 7f03deffd640 1 --2- 172.21.15.45:0/3047778572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809010 0x564eb5809430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.320+0000 7f03deffd640 1 -- 172.21.15.45:0/3047778572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eb57269b0 con 0x564eb5809a00 2024-06-06T04:32:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.320+0000 7f03df7fe640 1 --2- 172.21.15.45:0/710250478 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f03c80a2cd0 0x7f03c80a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.320+0000 7f03c7fff640 1 -- 172.21.15.45:0/710250478 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f03d40cf270 con 0x7f03e0100c30 2024-06-06T04:32:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.320+0000 7f03deffd640 1 --2- 172.21.15.45:0/3047778572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809a00 0x564eb580e500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f03d00314e0 tx=0x7f03d0005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=111a6821f4ebd306 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.321+0000 7f03c57fa640 1 -- 172.21.15.45:0/3047778572 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0035030 con 0x564eb5809a00 2024-06-06T04:32:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.321+0000 7f03c57fa640 1 -- 172.21.15.45:0/3047778572 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03d0040070 con 0x564eb5809a00 2024-06-06T04:32:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.321+0000 7f03c57fa640 1 -- 172.21.15.45:0/3047778572 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0003a80 con 0x564eb5809a00 2024-06-06T04:32:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.321+0000 7f03df7fe640 1 --2- 172.21.15.45:0/710250478 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f03c80a2cd0 0x7f03c80a5190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x564eb580a260 tx=0x7f03d8004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.321+0000 7f03e7fe5740 1 -- 172.21.15.45:0/3047778572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809a00 msgr2=0x564eb580e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.321+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/3047778572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809a00 0x564eb580e500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f03d00314e0 tx=0x7f03d0005c00 comp rx=0 tx=0).stop 2024-06-06T04:32:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.321+0000 7f03e7fe5740 1 -- 172.21.15.45:0/3047778572 shutdown_connections 2024-06-06T04:32:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.321+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/3047778572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809a00 0x564eb580e500 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.321+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/3047778572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb580eb70 0x564eb5811000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.321+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/3047778572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809010 0x564eb5809430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.321+0000 7f03e7fe5740 1 -- 172.21.15.45:0/3047778572 >> 172.21.15.45:0/3047778572 conn(0x564eb5804680 msgr2=0x564eb5804a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.322+0000 7f03e7fe5740 1 -- 172.21.15.45:0/3047778572 shutdown_connections 2024-06-06T04:32:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.322+0000 7f03e7fe5740 1 -- 172.21.15.45:0/3047778572 wait complete. 2024-06-06T04:32:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.322+0000 7f03e7fe5740 1 -- start start 2024-06-06T04:32:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.322+0000 7f03e7fe5740 1 Processor -- start 2024-06-06T04:32:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.322+0000 7f03e7fe5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809010 0x564eb5813310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.322+0000 7f03e7fe5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809a00 0x564eb5813850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.322+0000 7f03e7fe5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb580eb70 0x564eb581a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.322+0000 7f03e7fe5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564eb5811ac0 con 0x564eb5809a00 2024-06-06T04:32:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.322+0000 7f03e7fe5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564eb56f5ae0 con 0x564eb5809010 2024-06-06T04:32:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.322+0000 7f03e7fe5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564eb5811c40 con 0x564eb580eb70 2024-06-06T04:32:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.322+0000 7f03e7fe5740 10 client.0 _ll_register_callbacks cb 0x564eb572e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.322+0000 7f03de7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809a00 0x564eb5813850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.322+0000 7f03de7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809a00 0x564eb5813850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48734/0 (socket says 172.21.15.45:48734) 2024-06-06T04:32:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.322+0000 7f03de7fc640 1 -- 172.21.15.45:0/4117536049 learned_addr learned my addr 172.21.15.45:0/4117536049 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.322+0000 7f03deffd640 1 --2- 172.21.15.45:0/4117536049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809010 0x564eb5813310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03de7fc640 1 -- 172.21.15.45:0/4117536049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb580eb70 msgr2=0x564eb581a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03df7fe640 1 --2- 172.21.15.45:0/4117536049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb580eb70 0x564eb581a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03de7fc640 1 --2- 172.21.15.45:0/4117536049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb580eb70 0x564eb581a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03de7fc640 1 -- 172.21.15.45:0/4117536049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809010 msgr2=0x564eb5813310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03de7fc640 1 --2- 172.21.15.45:0/4117536049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809010 0x564eb5813310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03de7fc640 1 -- 172.21.15.45:0/4117536049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eb5480210 con 0x564eb5809a00 2024-06-06T04:32:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03df7fe640 1 --2- 172.21.15.45:0/4117536049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb580eb70 0x564eb581a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03deffd640 1 --2- 172.21.15.45:0/4117536049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809010 0x564eb5813310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03de7fc640 1 --2- 172.21.15.45:0/4117536049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809a00 0x564eb5813850 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f03d409b2b0 tx=0x7f03d409b0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98151a72c2a4e5c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03c2ff5640 1 -- 172.21.15.45:0/4117536049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4020030 con 0x564eb5809a00 2024-06-06T04:32:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03c2ff5640 1 -- 172.21.15.45:0/4117536049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f03d4010460 con 0x564eb5809a00 2024-06-06T04:32:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03e7fe5740 1 -- 172.21.15.45:0/4117536049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03d0040070 con 0x564eb5809a00 2024-06-06T04:32:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03c2ff5640 1 -- 172.21.15.45:0/4117536049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03d409b5b0 con 0x564eb5809a00 2024-06-06T04:32:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03e7fe5740 10 client.15402 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03e7fe5740 1 -- 172.21.15.45:0/4117536049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564eb554c620 con 0x564eb5809a00 2024-06-06T04:32:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.323+0000 7f03c2ff5640 1 -- 172.21.15.45:0/4117536049 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03d0040070 con 0x564eb5809a00 2024-06-06T04:32:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.324+0000 7f03c2ff5640 1 -- 172.21.15.45:0/4117536049 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f03d4097270 con 0x564eb5809a00 2024-06-06T04:32:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.324+0000 7f03e7fe5740 10 client.15402 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.324+0000 7f03e7fe5740 10 client.15402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.324+0000 7f03e7fe5740 20 client.15402 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.324+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/4117536049 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x564eb581f120 0x564eb5821570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.324+0000 7f03e7fe5740 4 client.15402 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:32:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.324+0000 7f03e7fe5740 1 -- 172.21.15.45:0/4117536049 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564eb577ff10 con 0x564eb581f120 2024-06-06T04:32:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.324+0000 7f03deffd640 1 --2- 172.21.15.45:0/4117536049 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x564eb581f120 0x564eb5821570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.325+0000 7f03c2ff5640 10 client.15402 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.325+0000 7f03deffd640 1 --2- 172.21.15.45:0/4117536049 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x564eb581f120 0x564eb5821570 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3dc6d2d89994175 server_cookie=f722f71b54925016 in_seq=0 out_seq=0 2024-06-06T04:32:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.325+0000 7f03c2ff5640 1 -- 172.21.15.45:0/4117536049 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564eb577ff10 con 0x564eb581f120 2024-06-06T04:32:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.325+0000 7f03c2ff5640 10 client.15402 handle_command_reply: tid=0 2024-06-06T04:32:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.326+0000 7f03e7fe5740 1 client.15402 shutdown 2024-06-06T04:32:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.326+0000 7f03e7fe5740 1 -- 172.21.15.45:0/4117536049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809a00 msgr2=0x564eb5813850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.326+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/4117536049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809a00 0x564eb5813850 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f03d409b2b0 tx=0x7f03d409b0f0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.326+0000 7f03e7fe5740 1 -- 172.21.15.45:0/4117536049 shutdown_connections 2024-06-06T04:32:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.326+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/4117536049 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x564eb581f120 0x564eb5821570 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.326+0000 7f03c2ff5640 0 client.15402 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.326+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/4117536049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb580eb70 0x564eb581a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.326+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/4117536049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809a00 0x564eb5813850 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.326+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/4117536049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809010 0x564eb5813310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.326+0000 7f03e7fe5740 1 -- 172.21.15.45:0/4117536049 >> 172.21.15.45:0/4117536049 conn(0x564eb5804680 msgr2=0x564eb58037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.326+0000 7f03e7fe5740 1 -- 172.21.15.45:0/4117536049 shutdown_connections 2024-06-06T04:32:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.326+0000 7f03e7fe5740 1 -- 172.21.15.45:0/4117536049 wait complete. 2024-06-06T04:32:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.326+0000 7f03e7fe5740 20 client.15402 trim_cache size 0 max 16384 2024-06-06T04:32:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.342+0000 7f03e7fe5740 1 Processor -- start 2024-06-06T04:32:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.342+0000 7f03e7fe5740 1 -- start start 2024-06-06T04:32:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.342+0000 7f03e7fe5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809010 0x564eb5810f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.342+0000 7f03e7fe5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb580eb70 0x564eb581a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.342+0000 7f03e7fe5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb581f120 0x564eb580ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.342+0000 7f03e7fe5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01aeff0 con 0x564eb580eb70 2024-06-06T04:32:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.342+0000 7f03e7fe5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01ac8c0 con 0x564eb5809010 2024-06-06T04:32:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.342+0000 7f03e7fe5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01aa710 con 0x564eb581f120 2024-06-06T04:32:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.343+0000 7f03de7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb580eb70 0x564eb581a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.343+0000 7f03de7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb580eb70 0x564eb581a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48736/0 (socket says 172.21.15.45:48736) 2024-06-06T04:32:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.343+0000 7f03de7fc640 1 -- 172.21.15.45:0/1380081620 learned_addr learned my addr 172.21.15.45:0/1380081620 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.343+0000 7f03deffd640 1 --2- 172.21.15.45:0/1380081620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809010 0x564eb5810f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.343+0000 7f03df7fe640 1 --2- 172.21.15.45:0/1380081620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb581f120 0x564eb580ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.343+0000 7f03de7fc640 1 -- 172.21.15.45:0/1380081620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb581f120 msgr2=0x564eb580ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.343+0000 7f03de7fc640 1 --2- 172.21.15.45:0/1380081620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb581f120 0x564eb580ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.343+0000 7f03de7fc640 1 -- 172.21.15.45:0/1380081620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809010 msgr2=0x564eb5810f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.343+0000 7f03de7fc640 1 --2- 172.21.15.45:0/1380081620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809010 0x564eb5810f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.343+0000 7f03de7fc640 1 -- 172.21.15.45:0/1380081620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eb58093f0 con 0x564eb580eb70 2024-06-06T04:32:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.343+0000 7f03deffd640 1 --2- 172.21.15.45:0/1380081620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809010 0x564eb5810f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.343+0000 7f03de7fc640 1 --2- 172.21.15.45:0/1380081620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb580eb70 0x564eb581a850 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f03d4000c00 tx=0x7f03d4097720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1958453582ec65 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.343+0000 7f03c4ff9640 1 -- 172.21.15.45:0/1380081620 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4014020 con 0x564eb580eb70 2024-06-06T04:32:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.343+0000 7f03c4ff9640 1 -- 172.21.15.45:0/1380081620 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03d409b810 con 0x564eb580eb70 2024-06-06T04:32:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.343+0000 7f03c4ff9640 1 -- 172.21.15.45:0/1380081620 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4020040 con 0x564eb580eb70 2024-06-06T04:32:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.344+0000 7f03e7fe5740 1 -- 172.21.15.45:0/1380081620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb580eb70 msgr2=0x564eb581a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.344+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/1380081620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb580eb70 0x564eb581a850 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f03d4000c00 tx=0x7f03d4097720 comp rx=0 tx=0).stop 2024-06-06T04:32:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.344+0000 7f03e7fe5740 1 -- 172.21.15.45:0/1380081620 shutdown_connections 2024-06-06T04:32:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.344+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/1380081620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb581f120 0x564eb580ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.344+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/1380081620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb580eb70 0x564eb581a850 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.344+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/1380081620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb5809010 0x564eb5810f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.344+0000 7f03e7fe5740 1 -- 172.21.15.45:0/1380081620 >> 172.21.15.45:0/1380081620 conn(0x564eb5804680 msgr2=0x564eb5813310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.344+0000 7f03e7fe5740 1 -- 172.21.15.45:0/1380081620 shutdown_connections 2024-06-06T04:32:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.344+0000 7f03e7fe5740 1 -- 172.21.15.45:0/1380081620 wait complete. 2024-06-06T04:32:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03e7fe5740 1 -- start start 2024-06-06T04:32:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03e7fe5740 1 Processor -- start 2024-06-06T04:32:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03e7fe5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809010 0x564eb581a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03e7fe5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb580eb70 0x564eb5816e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03e7fe5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb581f120 0x564eb5817370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03e7fe5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01aad80 con 0x564eb5809010 2024-06-06T04:32:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03e7fe5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01ac1f0 con 0x564eb580eb70 2024-06-06T04:32:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03e7fe5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f03e01ab450 con 0x564eb581f120 2024-06-06T04:32:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03e7fe5740 10 client.0 _ll_register_callbacks cb 0x7f03d4097120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03deffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809010 0x564eb581a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03deffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809010 0x564eb581a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48752/0 (socket says 172.21.15.45:48752) 2024-06-06T04:32:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03deffd640 1 -- 172.21.15.45:0/1071469044 learned_addr learned my addr 172.21.15.45:0/1071469044 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03de7fc640 1 --2- 172.21.15.45:0/1071469044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb580eb70 0x564eb5816e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03df7fe640 1 --2- 172.21.15.45:0/1071469044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb581f120 0x564eb5817370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03deffd640 1 -- 172.21.15.45:0/1071469044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb581f120 msgr2=0x564eb5817370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03deffd640 1 --2- 172.21.15.45:0/1071469044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb581f120 0x564eb5817370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03deffd640 1 -- 172.21.15.45:0/1071469044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb580eb70 msgr2=0x564eb5816e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03deffd640 1 --2- 172.21.15.45:0/1071469044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb580eb70 0x564eb5816e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03deffd640 1 -- 172.21.15.45:0/1071469044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eb581a9c0 con 0x564eb5809010 2024-06-06T04:32:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03de7fc640 1 --2- 172.21.15.45:0/1071469044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb580eb70 0x564eb5816e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.345+0000 7f03df7fe640 1 --2- 172.21.15.45:0/1071469044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb581f120 0x564eb5817370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.346+0000 7f03deffd640 1 --2- 172.21.15.45:0/1071469044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809010 0x564eb581a390 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f03d0005ec0 tx=0x7f03d0031c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d896f5c0db683b72 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.346+0000 7f03c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.346+0000 7f03c2ff5640 1 -- 172.21.15.45:0/1071469044 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0035040 con 0x564eb5809010 2024-06-06T04:32:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.346+0000 7f03c2ff5640 1 -- 172.21.15.45:0/1071469044 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f03d0004440 con 0x564eb5809010 2024-06-06T04:32:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.346+0000 7f03e7fe5740 1 -- 172.21.15.45:0/1071469044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564eb5779c20 con 0x564eb5809010 2024-06-06T04:32:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.346+0000 7f03c2ff5640 1 -- 172.21.15.45:0/1071469044 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03d00035f0 con 0x564eb5809010 2024-06-06T04:32:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.346+0000 7f03e7fe5740 10 client.15408 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.346+0000 7f03e7fe5740 1 -- 172.21.15.45:0/1071469044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564eb5888dc0 con 0x564eb5809010 2024-06-06T04:32:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.346+0000 7f03c2ff5640 1 -- 172.21.15.45:0/1071469044 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564eb5779c20 con 0x564eb5809010 2024-06-06T04:32:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.347+0000 7f03c2ff5640 1 -- 172.21.15.45:0/1071469044 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f03d003c7f0 con 0x564eb5809010 2024-06-06T04:32:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.347+0000 7f03e7fe5740 10 client.15408 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.347+0000 7f03e7fe5740 10 client.15408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.347+0000 7f03e7fe5740 20 client.15408 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.347+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/1071469044 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x564eb5889330 0x564eb588b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.347+0000 7f03e7fe5740 4 client.15408 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T04:32:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.347+0000 7f03e7fe5740 1 -- 172.21.15.45:0/1071469044 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564eb5773f70 con 0x564eb5889330 2024-06-06T04:32:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.347+0000 7f03de7fc640 1 --2- 172.21.15.45:0/1071469044 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x564eb5889330 0x564eb588b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.347+0000 7f03de7fc640 1 --2- 172.21.15.45:0/1071469044 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x564eb5889330 0x564eb588b720 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22c4c834bf56abf server_cookie=be018204b1faf489 in_seq=0 out_seq=0 2024-06-06T04:32:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.347+0000 7f03c2ff5640 10 client.15408 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.351+0000 7f03c2ff5640 1 -- 172.21.15.45:0/1071469044 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564eb5773f70 con 0x564eb5889330 2024-06-06T04:32:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.351+0000 7f03c2ff5640 10 client.15408 handle_command_reply: tid=0 2024-06-06T04:32:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.351+0000 7f03e7fe5740 1 client.15408 shutdown 2024-06-06T04:32:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.351+0000 7f03e7fe5740 1 -- 172.21.15.45:0/1071469044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809010 msgr2=0x564eb581a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.351+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/1071469044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809010 0x564eb581a390 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f03d0005ec0 tx=0x7f03d0031c10 comp rx=0 tx=0).stop 2024-06-06T04:32:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.351+0000 7f03e7fe5740 1 -- 172.21.15.45:0/1071469044 shutdown_connections 2024-06-06T04:32:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.351+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/1071469044 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x564eb5889330 0x564eb588b720 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.351+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/1071469044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564eb580eb70 0x564eb5816e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.351+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/1071469044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564eb581f120 0x564eb5817370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.351+0000 7f03e7fe5740 1 --2- 172.21.15.45:0/1071469044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564eb5809010 0x564eb581a390 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.351+0000 7f03c2ff5640 0 client.15408 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.351+0000 7f03e7fe5740 1 -- 172.21.15.45:0/1071469044 >> 172.21.15.45:0/1071469044 conn(0x564eb5804680 msgr2=0x564eb580ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.351+0000 7f03e7fe5740 1 -- 172.21.15.45:0/1071469044 shutdown_connections 2024-06-06T04:32:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.351+0000 7f03e7fe5740 1 -- 172.21.15.45:0/1071469044 wait complete. 2024-06-06T04:32:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.351+0000 7f03e7fe5740 20 client.15408 trim_cache size 0 max 16384 2024-06-06T04:32:10.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-06-06T04:32:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-06-06T04:32:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-06-06T04:32:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.352+0000 7f03e52d4640 1 -- 172.21.15.45:0/710250478 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f03c80a2cd0 msgr2=0x7f03c80a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.352+0000 7f03e52d4640 1 --2- 172.21.15.45:0/710250478 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f03c80a2cd0 0x7f03c80a5190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x564eb580a260 tx=0x7f03d8004ab0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.352+0000 7f03e52d4640 1 -- 172.21.15.45:0/710250478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e0100c30 msgr2=0x7f03e019a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.352+0000 7f03e52d4640 1 --2- 172.21.15.45:0/710250478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e0100c30 0x7f03e019a2e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f03d400d950 tx=0x7f03d400de10 comp rx=0 tx=0).stop 2024-06-06T04:32:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.352+0000 7f03e52d4640 1 -- 172.21.15.45:0/710250478 shutdown_connections 2024-06-06T04:32:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.352+0000 7f03e52d4640 1 --2- 172.21.15.45:0/710250478 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f03c80a2cd0 0x7f03c80a5190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.352+0000 7f03e52d4640 1 --2- 172.21.15.45:0/710250478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e01015d0 0x7f03e01a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.352+0000 7f03e52d4640 1 --2- 172.21.15.45:0/710250478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e0100c30 0x7f03e019a2e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.352+0000 7f03e52d4640 1 --2- 172.21.15.45:0/710250478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03e01002d0 0x7f03e0199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.352+0000 7f03e52d4640 1 -- 172.21.15.45:0/710250478 >> 172.21.15.45:0/710250478 conn(0x7f03e00fbe70 msgr2=0x7f03e00ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.352+0000 7f03e52d4640 1 -- 172.21.15.45:0/710250478 shutdown_connections 2024-06-06T04:32:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:10.352+0000 7f03e52d4640 1 -- 172.21.15.45:0/710250478 wait complete. 2024-06-06T04:32:10.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:10 smithi086 ceph-mon[36053]: pgmap v452: 129 pgs: 129 active+clean; 665 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:10.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:10 smithi154 ceph-mon[36428]: pgmap v452: 129 pgs: 129 active+clean; 665 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.785+0000 7f6add9d9700 1 -- 172.21.15.45:0/1291654401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad8108510 msgr2=0x7f6ad81088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.785+0000 7f6add9d9700 1 --2- 172.21.15.45:0/1291654401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad8108510 0x7f6ad81088f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009990 tx=0x7f6ac8009ca0 comp rx=0 tx=0).stop 2024-06-06T04:32:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.785+0000 7f6add9d9700 1 -- 172.21.15.45:0/1291654401 shutdown_connections 2024-06-06T04:32:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.785+0000 7f6add9d9700 1 --2- 172.21.15.45:0/1291654401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ad8101440 0x7f6ad810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.785+0000 7f6add9d9700 1 --2- 172.21.15.45:0/1291654401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad8108510 0x7f6ad81088f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.785+0000 7f6add9d9700 1 --2- 172.21.15.45:0/1291654401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8100a80 0x7f6ad8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.785+0000 7f6add9d9700 1 -- 172.21.15.45:0/1291654401 >> 172.21.15.45:0/1291654401 conn(0x7f6ad8077a00 msgr2=0x7f6ad80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.785+0000 7f6add9d9700 1 -- 172.21.15.45:0/1291654401 shutdown_connections 2024-06-06T04:32:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.785+0000 7f6add9d9700 1 -- 172.21.15.45:0/1291654401 wait complete. 2024-06-06T04:32:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.785+0000 7f6add9d9700 1 Processor -- start 2024-06-06T04:32:11.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6add9d9700 1 -- start start 2024-06-06T04:32:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6add9d9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8100a80 0x7f6ad819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6add9d9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ad8101440 0x7f6ad819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6add9d9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad8108510 0x7f6ad8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6add9d9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad8112620 con 0x7f6ad8108510 2024-06-06T04:32:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6add9d9700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad81124a0 con 0x7f6ad8100a80 2024-06-06T04:32:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6add9d9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad81127a0 con 0x7f6ad8101440 2024-06-06T04:32:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6ad77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad8108510 0x7f6ad8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6ad77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad8108510 0x7f6ad8198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48774/0 (socket says 172.21.15.45:48774) 2024-06-06T04:32:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6ad77fe700 1 -- 172.21.15.45:0/2788734540 learned_addr learned my addr 172.21.15.45:0/2788734540 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6ad6ffd700 1 --2- 172.21.15.45:0/2788734540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8100a80 0x7f6ad819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6ad67fc700 1 --2- 172.21.15.45:0/2788734540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ad8101440 0x7f6ad819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:11.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6ad77fe700 1 -- 172.21.15.45:0/2788734540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ad8101440 msgr2=0x7f6ad819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6ad77fe700 1 --2- 172.21.15.45:0/2788734540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ad8101440 0x7f6ad819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6ad77fe700 1 -- 172.21.15.45:0/2788734540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8100a80 msgr2=0x7f6ad819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6ad77fe700 1 --2- 172.21.15.45:0/2788734540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8100a80 0x7f6ad819e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6ad77fe700 1 -- 172.21.15.45:0/2788734540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad81990c0 con 0x7f6ad8108510 2024-06-06T04:32:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.786+0000 7f6ad67fc700 1 --2- 172.21.15.45:0/2788734540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ad8101440 0x7f6ad819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.787+0000 7f6ad77fe700 1 --2- 172.21.15.45:0/2788734540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad8108510 0x7f6ad8198960 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0007ee0 tx=0x7f6ad000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.787+0000 7f6adc9d7700 1 -- 172.21.15.45:0/2788734540 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad000dbd0 con 0x7f6ad8108510 2024-06-06T04:32:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.787+0000 7f6add9d9700 1 -- 172.21.15.45:0/2788734540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad81993b0 con 0x7f6ad8108510 2024-06-06T04:32:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.787+0000 7f6adc9d7700 1 -- 172.21.15.45:0/2788734540 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ad000dd70 con 0x7f6ad8108510 2024-06-06T04:32:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.787+0000 7f6adc9d7700 1 -- 172.21.15.45:0/2788734540 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0005990 con 0x7f6ad8108510 2024-06-06T04:32:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.787+0000 7f6add9d9700 1 -- 172.21.15.45:0/2788734540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad81a5670 con 0x7f6ad8108510 2024-06-06T04:32:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.788+0000 7f6add9d9700 1 -- 172.21.15.45:0/2788734540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad810a9f0 con 0x7f6ad8108510 2024-06-06T04:32:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.789+0000 7f6adc9d7700 1 -- 172.21.15.45:0/2788734540 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f6ad001a020 con 0x7f6ad8108510 2024-06-06T04:32:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.790+0000 7f6adc9d7700 1 --2- 172.21.15.45:0/2788734540 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6ac00a2cb0 0x7f6ac00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.790+0000 7f6ad6ffd700 1 --2- 172.21.15.45:0/2788734540 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6ac00a2cb0 0x7f6ac00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.790+0000 7f6adc9d7700 1 -- 172.21.15.45:0/2788734540 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f6ad00cecf0 con 0x7f6ad8108510 2024-06-06T04:32:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.790+0000 7f6ad6ffd700 1 --2- 172.21.15.45:0/2788734540 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6ac00a2cb0 0x7f6ac00a5170 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800b5c0 tx=0x7f6ac8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:11.795+0000 7f6adc9d7700 1 -- 172.21.15.45:0/2788734540 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6ad0010070 con 0x7f6ad8108510 2024-06-06T04:32:12.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:12.072+0000 7f6add9d9700 1 -- 172.21.15.45:0/2788734540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ad8069180 con 0x7f6ad8108510 2024-06-06T04:32:12.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:12.073+0000 7f6adc9d7700 1 -- 172.21.15.45:0/2788734540 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11858 (secure 0 0 0) 0x7f6ad009b6d0 con 0x7f6ad8108510 2024-06-06T04:32:12.073 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:32:12.073 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T04:27:24.462897+0000","modified":"2024-06-06T04:31:17.956589+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24451,24463,24473]},"id":1}]} 2024-06-06T04:32:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:12.075+0000 7f6add9d9700 1 -- 172.21.15.45:0/2788734540 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6ac00a2cb0 msgr2=0x7f6ac00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:12.075+0000 7f6add9d9700 1 --2- 172.21.15.45:0/2788734540 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6ac00a2cb0 0x7f6ac00a5170 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800b5c0 tx=0x7f6ac8005960 comp rx=0 tx=0).stop 2024-06-06T04:32:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:12.075+0000 7f6add9d9700 1 -- 172.21.15.45:0/2788734540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad8108510 msgr2=0x7f6ad8198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:12.075+0000 7f6add9d9700 1 --2- 172.21.15.45:0/2788734540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad8108510 0x7f6ad8198960 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0007ee0 tx=0x7f6ad000d3b0 comp rx=0 tx=0).stop 2024-06-06T04:32:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:12.075+0000 7f6add9d9700 1 -- 172.21.15.45:0/2788734540 shutdown_connections 2024-06-06T04:32:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:12.075+0000 7f6add9d9700 1 --2- 172.21.15.45:0/2788734540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad8108510 0x7f6ad8198960 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:12.075+0000 7f6add9d9700 1 --2- 172.21.15.45:0/2788734540 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6ac00a2cb0 0x7f6ac00a5170 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:12.075+0000 7f6add9d9700 1 --2- 172.21.15.45:0/2788734540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8100a80 0x7f6ad819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:12.075+0000 7f6add9d9700 1 --2- 172.21.15.45:0/2788734540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ad8101440 0x7f6ad819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:12.075+0000 7f6add9d9700 1 -- 172.21.15.45:0/2788734540 >> 172.21.15.45:0/2788734540 conn(0x7f6ad8077a00 msgr2=0x7f6ad80ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:12.075+0000 7f6add9d9700 1 -- 172.21.15.45:0/2788734540 shutdown_connections 2024-06-06T04:32:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:12.075+0000 7f6add9d9700 1 -- 172.21.15.45:0/2788734540 wait complete. 2024-06-06T04:32:12.077 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-06-06T04:32:12.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:12 smithi045 ceph-mon[32299]: pgmap v453: 129 pgs: 129 active+clean; 665 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:12.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:12 smithi086 ceph-mon[36053]: pgmap v453: 129 pgs: 129 active+clean; 665 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:12.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:12 smithi154 ceph-mon[36428]: pgmap v453: 129 pgs: 129 active+clean; 665 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:13.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:13 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2788734540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:32:13.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:13 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2788734540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:32:13.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:13 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2788734540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:32:14.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:14 smithi086 ceph-mon[36053]: pgmap v454: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-06-06T04:32:14.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:14 smithi154 ceph-mon[36428]: pgmap v454: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-06-06T04:32:14.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:14 smithi045 ceph-mon[32299]: pgmap v454: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-06-06T04:32:16.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:16 smithi086 ceph-mon[36053]: pgmap v455: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T04:32:16.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:16 smithi154 ceph-mon[36428]: pgmap v455: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T04:32:16.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:16 smithi045 ceph-mon[32299]: pgmap v455: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-06T04:32:16.751 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-06T04:32:17.074 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:32:18.329 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:18 smithi045 ceph-mon[32299]: pgmap v456: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:18.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:18 smithi086 ceph-mon[36053]: pgmap v456: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:18.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:18 smithi154 ceph-mon[36428]: pgmap v456: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:18.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.891+0000 7f4a8afd3700 1 -- 172.21.15.45:0/997324115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a84101d50 msgr2=0x7f4a8410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.891+0000 7f4a8afd3700 1 --2- 172.21.15.45:0/997324115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a84101d50 0x7f4a8410e240 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00b0a0 tx=0x7f4a7c00b3b0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.891+0000 7f4a8afd3700 1 -- 172.21.15.45:0/997324115 shutdown_connections 2024-06-06T04:32:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.891+0000 7f4a8afd3700 1 --2- 172.21.15.45:0/997324115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a84101d50 0x7f4a8410e240 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.891+0000 7f4a8afd3700 1 --2- 172.21.15.45:0/997324115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a84107400 0x7f4a841077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.891+0000 7f4a8afd3700 1 --2- 172.21.15.45:0/997324115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a84101390 0x7f4a84101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.891+0000 7f4a8afd3700 1 -- 172.21.15.45:0/997324115 >> 172.21.15.45:0/997324115 conn(0x7f4a84078340 msgr2=0x7f4a84078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.891+0000 7f4a8afd3700 1 -- 172.21.15.45:0/997324115 shutdown_connections 2024-06-06T04:32:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.891+0000 7f4a8afd3700 1 -- 172.21.15.45:0/997324115 wait complete. 2024-06-06T04:32:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.892+0000 7f4a8afd3700 1 Processor -- start 2024-06-06T04:32:18.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.892+0000 7f4a8afd3700 1 -- start start 2024-06-06T04:32:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.892+0000 7f4a8afd3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a84101390 0x7f4a841a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.892+0000 7f4a8afd3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a84101d50 0x7f4a841a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.893+0000 7f4a8afd3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a84107400 0x7f4a8419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.893+0000 7f4a8afd3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a84111520 con 0x7f4a84101390 2024-06-06T04:32:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.893+0000 7f4a8afd3700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841113a0 con 0x7f4a84107400 2024-06-06T04:32:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.893+0000 7f4a8afd3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841116a0 con 0x7f4a84101d50 2024-06-06T04:32:18.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.893+0000 7f4a88d6f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a84101390 0x7f4a841a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.893+0000 7f4a88d6f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a84101390 0x7f4a841a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57356/0 (socket says 172.21.15.45:57356) 2024-06-06T04:32:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.893+0000 7f4a88d6f700 1 -- 172.21.15.45:0/2105898500 learned_addr learned my addr 172.21.15.45:0/2105898500 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.893+0000 7f4a83fff700 1 --2- 172.21.15.45:0/2105898500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a84101d50 0x7f4a841a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.893+0000 7f4a89570700 1 --2- 172.21.15.45:0/2105898500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a84107400 0x7f4a8419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.893+0000 7f4a83fff700 1 -- 172.21.15.45:0/2105898500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a84107400 msgr2=0x7f4a8419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.893+0000 7f4a83fff700 1 --2- 172.21.15.45:0/2105898500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a84107400 0x7f4a8419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.893+0000 7f4a83fff700 1 -- 172.21.15.45:0/2105898500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a84101390 msgr2=0x7f4a841a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:18.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.893+0000 7f4a83fff700 1 --2- 172.21.15.45:0/2105898500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a84101390 0x7f4a841a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.893+0000 7f4a83fff700 1 -- 172.21.15.45:0/2105898500 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8419b410 con 0x7f4a84101d50 2024-06-06T04:32:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.893+0000 7f4a88d6f700 1 --2- 172.21.15.45:0/2105898500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a84101390 0x7f4a841a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.893+0000 7f4a83fff700 1 --2- 172.21.15.45:0/2105898500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a84101d50 0x7f4a841a0fa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000d950 tx=0x7f4a7000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.894+0000 7f4a81ffb700 1 -- 172.21.15.45:0/2105898500 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70017070 con 0x7f4a84101d50 2024-06-06T04:32:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.894+0000 7f4a81ffb700 1 -- 172.21.15.45:0/2105898500 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a70009de0 con 0x7f4a84101d50 2024-06-06T04:32:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.894+0000 7f4a81ffb700 1 -- 172.21.15.45:0/2105898500 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70010d40 con 0x7f4a84101d50 2024-06-06T04:32:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.894+0000 7f4a8afd3700 1 -- 172.21.15.45:0/2105898500 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8419b700 con 0x7f4a84101d50 2024-06-06T04:32:18.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.894+0000 7f4a8afd3700 1 -- 172.21.15.45:0/2105898500 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a841a7810 con 0x7f4a84101d50 2024-06-06T04:32:18.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.897+0000 7f4a81ffb700 1 -- 172.21.15.45:0/2105898500 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f4a7001a070 con 0x7f4a84101d50 2024-06-06T04:32:18.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.898+0000 7f4a95856b80 1 Processor -- start 2024-06-06T04:32:18.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.898+0000 7f4a95856b80 1 -- start start 2024-06-06T04:32:18.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.898+0000 7f4a81ffb700 1 --2- 172.21.15.45:0/2105898500 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4a740a2d30 0x7f4a740a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.898+0000 7f4a95856b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21f2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.898+0000 7f4a95856b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21f26c0 0x55fdb21ed1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.898+0000 7f4a95856b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21ed730 0x55fdb21edb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.898+0000 7f4a95856b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fdb2136810 con 0x55fdb21ed730 2024-06-06T04:32:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.898+0000 7f4a95856b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fdb20de720 con 0x55fdb21f26c0 2024-06-06T04:32:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.898+0000 7f4a95856b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fdb21327d0 con 0x55fdb21f1da0 2024-06-06T04:32:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.898+0000 7f4a81ffb700 1 -- 172.21.15.45:0/2105898500 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f4a700cf010 con 0x7f4a84101d50 2024-06-06T04:32:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.899+0000 7f4a89570700 1 --2- 172.21.15.45:0/2105898500 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4a740a2d30 0x7f4a740a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.899+0000 7f4a88d6f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.899+0000 7f4a88d6f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21f2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:53528/0 (socket says 172.21.15.45:53528) 2024-06-06T04:32:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.899+0000 7f4a88d6f700 1 -- 172.21.15.45:0/98650573 learned_addr learned my addr 172.21.15.45:0/98650573 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.899+0000 7f4a89570700 1 --2- 172.21.15.45:0/98650573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21ed730 0x55fdb21edb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.899+0000 7f4a83fff700 1 --2- 172.21.15.45:0/98650573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21f26c0 0x55fdb21ed1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.899+0000 7f4a88d6f700 1 -- 172.21.15.45:0/98650573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21f26c0 msgr2=0x55fdb21ed1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:18.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.899+0000 7f4a88d6f700 1 --2- 172.21.15.45:0/98650573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21f26c0 0x55fdb21ed1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.899+0000 7f4a88d6f700 1 -- 172.21.15.45:0/98650573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21ed730 msgr2=0x55fdb21edb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:18.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.899+0000 7f4a88d6f700 1 --2- 172.21.15.45:0/98650573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21ed730 0x55fdb21edb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.899+0000 7f4a88d6f700 1 -- 172.21.15.45:0/98650573 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdb20d4f50 con 0x55fdb21f1da0 2024-06-06T04:32:18.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.899+0000 7f4a89570700 1 --2- 172.21.15.45:0/2105898500 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4a740a2d30 0x7f4a740a51f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009610 tx=0x7f4a7c009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:18.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.899+0000 7f4a88d6f700 1 --2- 172.21.15.45:0/98650573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21f2180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4a78011650 tx=0x7f4a78011960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19d5eaaeeeedbfbd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:18.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.900+0000 7f4a6effd700 1 -- 172.21.15.45:0/98650573 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7801e070 con 0x55fdb21f1da0 2024-06-06T04:32:18.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.900+0000 7f4a6effd700 1 -- 172.21.15.45:0/98650573 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a78019ae0 con 0x55fdb21f1da0 2024-06-06T04:32:18.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.900+0000 7f4a6effd700 1 -- 172.21.15.45:0/98650573 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7800e9f0 con 0x55fdb21f1da0 2024-06-06T04:32:18.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.901+0000 7f4a95856b80 1 -- 172.21.15.45:0/98650573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 msgr2=0x55fdb21f2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:18.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.901+0000 7f4a95856b80 1 --2- 172.21.15.45:0/98650573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21f2180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4a78011650 tx=0x7f4a78011960 comp rx=0 tx=0).stop 2024-06-06T04:32:18.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.901+0000 7f4a95856b80 1 -- 172.21.15.45:0/98650573 shutdown_connections 2024-06-06T04:32:18.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.901+0000 7f4a95856b80 1 --2- 172.21.15.45:0/98650573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21ed730 0x55fdb21edb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.901+0000 7f4a95856b80 1 --2- 172.21.15.45:0/98650573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21f2180 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.901+0000 7f4a95856b80 1 --2- 172.21.15.45:0/98650573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21f26c0 0x55fdb21ed1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.901+0000 7f4a95856b80 1 -- 172.21.15.45:0/98650573 >> 172.21.15.45:0/98650573 conn(0x55fdb210fcc0 msgr2=0x55fdb21df850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:18.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.901+0000 7f4a95856b80 1 -- 172.21.15.45:0/98650573 shutdown_connections 2024-06-06T04:32:18.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.901+0000 7f4a95856b80 1 -- 172.21.15.45:0/98650573 wait complete. 2024-06-06T04:32:18.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.901+0000 7f4a95856b80 1 -- start start 2024-06-06T04:32:18.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.902+0000 7f4a95856b80 1 Processor -- start 2024-06-06T04:32:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.902+0000 7f4a95856b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 0x55fdb21f9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.902+0000 7f4a95856b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21f6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.902+0000 7f4a95856b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb21f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.902+0000 7f4a95856b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fdb21fa200 con 0x55fdb21f26c0 2024-06-06T04:32:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.902+0000 7f4a95856b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fdb20d6bf0 con 0x55fdb21ed730 2024-06-06T04:32:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.902+0000 7f4a95856b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fdb21fa380 con 0x55fdb21f1da0 2024-06-06T04:32:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.902+0000 7f4a95856b80 10 client.0 _ll_register_callbacks cb 0x55fdb20d0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:18.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.902+0000 7f4a83fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.902+0000 7f4a89570700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb21f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.903+0000 7f4a88d6f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 0x55fdb21f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.903+0000 7f4a83fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21f6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:53542/0 (socket says 172.21.15.45:53542) 2024-06-06T04:32:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.903+0000 7f4a83fff700 1 -- 172.21.15.45:0/3189397625 learned_addr learned my addr 172.21.15.45:0/3189397625 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.903+0000 7f4a88d6f700 1 -- 172.21.15.45:0/3189397625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 msgr2=0x55fdb21f6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.903+0000 7f4a88d6f700 1 --2- 172.21.15.45:0/3189397625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21f6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.903+0000 7f4a88d6f700 1 -- 172.21.15.45:0/3189397625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 msgr2=0x55fdb21f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.903+0000 7f4a88d6f700 1 --2- 172.21.15.45:0/3189397625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb21f6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.903+0000 7f4a88d6f700 1 -- 172.21.15.45:0/3189397625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdb20858d0 con 0x55fdb21ed730 2024-06-06T04:32:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.903+0000 7f4a89570700 1 --2- 172.21.15.45:0/3189397625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb21f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.903+0000 7f4a88d6f700 1 --2- 172.21.15.45:0/3189397625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 0x55fdb21f9440 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4a78000c00 tx=0x7f4a78019660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd067a023451ff6e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.903+0000 7f4a63fff700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-06-06T04:32:18.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.903+0000 7f4a63fff700 1 -- 172.21.15.45:0/3189397625 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7801e050 con 0x55fdb21ed730 2024-06-06T04:32:18.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.904+0000 7f4a63fff700 1 -- 172.21.15.45:0/3189397625 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f4a78004040 con 0x55fdb21ed730 2024-06-06T04:32:18.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.904+0000 7f4a95856b80 1 -- 172.21.15.45:0/3189397625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a78019ae0 con 0x55fdb21ed730 2024-06-06T04:32:18.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.904+0000 7f4a63fff700 1 -- 172.21.15.45:0/3189397625 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4a78004240 con 0x55fdb21ed730 2024-06-06T04:32:18.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.904+0000 7f4a95856b80 10 client.25088 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:18.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.904+0000 7f4a95856b80 1 -- 172.21.15.45:0/3189397625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fdb2204a50 con 0x55fdb21ed730 2024-06-06T04:32:18.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.904+0000 7f4a63fff700 1 -- 172.21.15.45:0/3189397625 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a78019ae0 con 0x55fdb21ed730 2024-06-06T04:32:18.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.904+0000 7f4a63fff700 1 -- 172.21.15.45:0/3189397625 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4a7800e670 con 0x55fdb21ed730 2024-06-06T04:32:18.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.904+0000 7f4a95856b80 10 client.25088 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:18.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.904+0000 7f4a95856b80 10 client.25088 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-06T04:32:18.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.904+0000 7f4a95856b80 20 client.25088 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:18.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.904+0000 7f4a95856b80 1 --2- 172.21.15.45:0/3189397625 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55fdb2206fd0 0x55fdb22093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.905+0000 7f4a95856b80 4 client.25088 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:32:18.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.905+0000 7f4a95856b80 1 -- 172.21.15.45:0/3189397625 --> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdb2143d10 con 0x55fdb2206fd0 2024-06-06T04:32:18.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.905+0000 7f4a83fff700 1 --2- 172.21.15.45:0/3189397625 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55fdb2206fd0 0x55fdb22093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.905+0000 7f4a83fff700 1 --2- 172.21.15.45:0/3189397625 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55fdb2206fd0 0x55fdb22093c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4f68d74b9ab2dc2 server_cookie=aafc6c731878eed2 in_seq=0 out_seq=0 2024-06-06T04:32:18.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.905+0000 7f4a63fff700 10 client.25088 ms_handle_connect on v2:172.21.15.154:6836/1140071744 2024-06-06T04:32:18.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.906+0000 7f4a63fff700 1 -- 172.21.15.45:0/3189397625 <== mds.2 v2:172.21.15.154:6836/1140071744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fdb2143d10 con 0x55fdb2206fd0 2024-06-06T04:32:18.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.906+0000 7f4a63fff700 10 client.25088 handle_command_reply: tid=0 2024-06-06T04:32:18.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.906+0000 7f4a95856b80 1 client.25088 shutdown 2024-06-06T04:32:18.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.906+0000 7f4a95856b80 1 -- 172.21.15.45:0/3189397625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 msgr2=0x55fdb21f9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:18.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.906+0000 7f4a95856b80 1 --2- 172.21.15.45:0/3189397625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 0x55fdb21f9440 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4a78000c00 tx=0x7f4a78019660 comp rx=0 tx=0).stop 2024-06-06T04:32:18.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.906+0000 7f4a95856b80 1 -- 172.21.15.45:0/3189397625 shutdown_connections 2024-06-06T04:32:18.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.906+0000 7f4a95856b80 1 --2- 172.21.15.45:0/3189397625 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55fdb2206fd0 0x55fdb22093c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.906+0000 7f4a95856b80 1 --2- 172.21.15.45:0/3189397625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb21f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.906+0000 7f4a95856b80 1 --2- 172.21.15.45:0/3189397625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 0x55fdb21f9440 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.906+0000 7f4a95856b80 1 --2- 172.21.15.45:0/3189397625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21f6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.906+0000 7f4a95856b80 1 -- 172.21.15.45:0/3189397625 >> 172.21.15.45:0/3189397625 conn(0x55fdb210fcc0 msgr2=0x55fdb21edb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:18.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.906+0000 7f4a63fff700 0 client.25088 ms_handle_reset on v2:172.21.15.154:6836/1140071744 2024-06-06T04:32:18.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.907+0000 7f4a95856b80 1 -- 172.21.15.45:0/3189397625 shutdown_connections 2024-06-06T04:32:18.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.907+0000 7f4a95856b80 1 -- 172.21.15.45:0/3189397625 wait complete. 2024-06-06T04:32:18.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.907+0000 7f4a95856b80 20 client.25088 trim_cache size 0 max 16384 2024-06-06T04:32:18.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.937+0000 7f4a95856b80 1 Processor -- start 2024-06-06T04:32:18.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.937+0000 7f4a95856b80 1 -- start start 2024-06-06T04:32:18.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a95856b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 0x55fdb21f9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a95856b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21ef980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a95856b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb21efec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a95856b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841aea20 con 0x55fdb21f26c0 2024-06-06T04:32:18.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a95856b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841ac2b0 con 0x55fdb21ed730 2024-06-06T04:32:18.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a95856b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841aa100 con 0x55fdb21f1da0 2024-06-06T04:32:18.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a89570700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb21efec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a88d6f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 0x55fdb21f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a89570700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb21efec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57372/0 (socket says 172.21.15.45:57372) 2024-06-06T04:32:18.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a89570700 1 -- 172.21.15.45:0/1625562471 learned_addr learned my addr 172.21.15.45:0/1625562471 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:18.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a83fff700 1 --2- 172.21.15.45:0/1625562471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21ef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a89570700 1 -- 172.21.15.45:0/1625562471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 msgr2=0x55fdb21ef980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:18.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a89570700 1 --2- 172.21.15.45:0/1625562471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21ef980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a89570700 1 -- 172.21.15.45:0/1625562471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 msgr2=0x55fdb21f9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:18.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a89570700 1 --2- 172.21.15.45:0/1625562471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 0x55fdb21f9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a89570700 1 -- 172.21.15.45:0/1625562471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdb21f9080 con 0x55fdb21f26c0 2024-06-06T04:32:18.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.938+0000 7f4a88d6f700 1 --2- 172.21.15.45:0/1625562471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 0x55fdb21f9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:18.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.939+0000 7f4a89570700 1 --2- 172.21.15.45:0/1625562471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb21efec0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00b070 tx=0x7f4a7c028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582965a92afe8087 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:18.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.939+0000 7f4a6e7fc700 1 -- 172.21.15.45:0/1625562471 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c007940 con 0x55fdb21f26c0 2024-06-06T04:32:18.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.939+0000 7f4a6e7fc700 1 -- 172.21.15.45:0/1625562471 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a7c032070 con 0x55fdb21f26c0 2024-06-06T04:32:18.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.939+0000 7f4a6e7fc700 1 -- 172.21.15.45:0/1625562471 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c02dd30 con 0x55fdb21f26c0 2024-06-06T04:32:18.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.940+0000 7f4a95856b80 1 -- 172.21.15.45:0/1625562471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 msgr2=0x55fdb21efec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:18.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.940+0000 7f4a95856b80 1 --2- 172.21.15.45:0/1625562471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb21efec0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00b070 tx=0x7f4a7c028000 comp rx=0 tx=0).stop 2024-06-06T04:32:18.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.940+0000 7f4a95856b80 1 -- 172.21.15.45:0/1625562471 shutdown_connections 2024-06-06T04:32:18.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.940+0000 7f4a95856b80 1 --2- 172.21.15.45:0/1625562471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb21efec0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.940+0000 7f4a95856b80 1 --2- 172.21.15.45:0/1625562471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 0x55fdb21f9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.940+0000 7f4a95856b80 1 --2- 172.21.15.45:0/1625562471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21ef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.940+0000 7f4a95856b80 1 -- 172.21.15.45:0/1625562471 >> 172.21.15.45:0/1625562471 conn(0x55fdb210fcc0 msgr2=0x55fdb21df5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:18.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.940+0000 7f4a95856b80 1 -- 172.21.15.45:0/1625562471 shutdown_connections 2024-06-06T04:32:18.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.940+0000 7f4a95856b80 1 -- 172.21.15.45:0/1625562471 wait complete. 2024-06-06T04:32:18.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.941+0000 7f4a95856b80 1 -- start start 2024-06-06T04:32:18.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.941+0000 7f4a95856b80 1 Processor -- start 2024-06-06T04:32:18.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.941+0000 7f4a95856b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 0x55fdb21ff640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.941+0000 7f4a95856b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.941+0000 7f4a95856b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb220dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.941+0000 7f4a89570700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb220dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.941+0000 7f4a83fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.941+0000 7f4a89570700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb220dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57384/0 (socket says 172.21.15.45:57384) 2024-06-06T04:32:18.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.941+0000 7f4a83fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21ffb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:53566/0 (socket says 172.21.15.45:53566) 2024-06-06T04:32:18.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.941+0000 7f4a83fff700 1 -- 172.21.15.45:0/674373273 learned_addr learned my addr 172.21.15.45:0/674373273 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:18.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.942+0000 7f4a95856b80 1 -- 172.21.15.45:0/674373273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841aa770 con 0x55fdb21f26c0 2024-06-06T04:32:18.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.942+0000 7f4a95856b80 1 -- 172.21.15.45:0/674373273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841abbe0 con 0x55fdb21ed730 2024-06-06T04:32:18.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.942+0000 7f4a95856b80 1 -- 172.21.15.45:0/674373273 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841aae40 con 0x55fdb21f1da0 2024-06-06T04:32:18.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.942+0000 7f4a88d6f700 1 --2- 172.21.15.45:0/674373273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 0x55fdb21ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.942+0000 7f4a95856b80 10 client.0 _ll_register_callbacks cb 0x55fdb20d0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:18.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.942+0000 7f4a88d6f700 1 -- 172.21.15.45:0/674373273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 msgr2=0x55fdb21ffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:18.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.942+0000 7f4a88d6f700 1 --2- 172.21.15.45:0/674373273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21ffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.942+0000 7f4a88d6f700 1 -- 172.21.15.45:0/674373273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 msgr2=0x55fdb220dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:18.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.942+0000 7f4a88d6f700 1 --2- 172.21.15.45:0/674373273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb220dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.942+0000 7f4a88d6f700 1 -- 172.21.15.45:0/674373273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdb220f510 con 0x55fdb21ed730 2024-06-06T04:32:18.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.942+0000 7f4a83fff700 1 --2- 172.21.15.45:0/674373273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.942+0000 7f4a89570700 1 --2- 172.21.15.45:0/674373273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb220dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.942+0000 7f4a88d6f700 1 --2- 172.21.15.45:0/674373273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 0x55fdb21ff640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800ebe0 tx=0x7f4a780115f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=395c165ab40fc571 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.942+0000 7f4a63fff700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-06-06T04:32:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.943+0000 7f4a63fff700 1 -- 172.21.15.45:0/674373273 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78036020 con 0x55fdb21ed730 2024-06-06T04:32:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.943+0000 7f4a63fff700 1 -- 172.21.15.45:0/674373273 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f4a7800e580 con 0x55fdb21ed730 2024-06-06T04:32:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.943+0000 7f4a95856b80 1 -- 172.21.15.45:0/674373273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a7c032070 con 0x55fdb21ed730 2024-06-06T04:32:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.943+0000 7f4a63fff700 1 -- 172.21.15.45:0/674373273 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4a7801e040 con 0x55fdb21ed730 2024-06-06T04:32:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.943+0000 7f4a95856b80 10 client.25094 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.943+0000 7f4a95856b80 1 -- 172.21.15.45:0/674373273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fdb2270df0 con 0x55fdb21ed730 2024-06-06T04:32:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.943+0000 7f4a63fff700 1 -- 172.21.15.45:0/674373273 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a7c032070 con 0x55fdb21ed730 2024-06-06T04:32:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.944+0000 7f4a63fff700 1 -- 172.21.15.45:0/674373273 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4a78005320 con 0x55fdb21ed730 2024-06-06T04:32:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.944+0000 7f4a95856b80 10 client.25094 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.944+0000 7f4a95856b80 10 client.25094 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-06T04:32:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.944+0000 7f4a95856b80 20 client.25094 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:18.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.944+0000 7f4a95856b80 1 --2- 172.21.15.45:0/674373273 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55fdb22714a0 0x55fdb22738c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.944+0000 7f4a95856b80 4 client.25094 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T04:32:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.944+0000 7f4a95856b80 1 -- 172.21.15.45:0/674373273 --> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fdb213e290 con 0x55fdb22714a0 2024-06-06T04:32:18.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.945+0000 7f4a83fff700 1 --2- 172.21.15.45:0/674373273 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55fdb22714a0 0x55fdb22738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.945+0000 7f4a83fff700 1 --2- 172.21.15.45:0/674373273 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55fdb22714a0 0x55fdb22738c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c95c1cf0578aa76 server_cookie=b31b91fbe1aeebc1 in_seq=0 out_seq=0 2024-06-06T04:32:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.945+0000 7f4a63fff700 10 client.25094 ms_handle_connect on v2:172.21.15.154:6836/1140071744 2024-06-06T04:32:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.945+0000 7f4a63fff700 1 -- 172.21.15.45:0/674373273 <== mds.2 v2:172.21.15.154:6836/1140071744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fdb213e290 con 0x55fdb22714a0 2024-06-06T04:32:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.945+0000 7f4a63fff700 10 client.25094 handle_command_reply: tid=0 2024-06-06T04:32:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.945+0000 7f4a95856b80 1 client.25094 shutdown 2024-06-06T04:32:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.945+0000 7f4a95856b80 1 -- 172.21.15.45:0/674373273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 msgr2=0x55fdb21ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.945+0000 7f4a95856b80 1 --2- 172.21.15.45:0/674373273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 0x55fdb21ff640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800ebe0 tx=0x7f4a780115f0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.946+0000 7f4a95856b80 1 -- 172.21.15.45:0/674373273 shutdown_connections 2024-06-06T04:32:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.946+0000 7f4a95856b80 1 --2- 172.21.15.45:0/674373273 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55fdb22714a0 0x55fdb22738c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.946+0000 7f4a63fff700 0 client.25094 ms_handle_reset on v2:172.21.15.154:6836/1140071744 2024-06-06T04:32:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.946+0000 7f4a95856b80 1 --2- 172.21.15.45:0/674373273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fdb21f26c0 0x55fdb220dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.946+0000 7f4a95856b80 1 --2- 172.21.15.45:0/674373273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fdb21ed730 0x55fdb21ff640 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.946+0000 7f4a95856b80 1 --2- 172.21.15.45:0/674373273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fdb21f1da0 0x55fdb21ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.946+0000 7f4a95856b80 1 -- 172.21.15.45:0/674373273 >> 172.21.15.45:0/674373273 conn(0x55fdb210fcc0 msgr2=0x55fdb21d8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.946+0000 7f4a95856b80 1 -- 172.21.15.45:0/674373273 shutdown_connections 2024-06-06T04:32:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.946+0000 7f4a95856b80 1 -- 172.21.15.45:0/674373273 wait complete. 2024-06-06T04:32:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.946+0000 7f4a95856b80 20 client.25094 trim_cache size 0 max 16384 2024-06-06T04:32:18.950 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-06-06T04:32:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.947+0000 7f4a8afd3700 1 -- 172.21.15.45:0/2105898500 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4a740a2d30 msgr2=0x7f4a740a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.947+0000 7f4a8afd3700 1 --2- 172.21.15.45:0/2105898500 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4a740a2d30 0x7f4a740a51f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009610 tx=0x7f4a7c009360 comp rx=0 tx=0).stop 2024-06-06T04:32:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.947+0000 7f4a8afd3700 1 -- 172.21.15.45:0/2105898500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a84101d50 msgr2=0x7f4a841a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.947+0000 7f4a8afd3700 1 --2- 172.21.15.45:0/2105898500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a84101d50 0x7f4a841a0fa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000d950 tx=0x7f4a7000dc60 comp rx=0 tx=0).stop 2024-06-06T04:32:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.947+0000 7f4a8afd3700 1 -- 172.21.15.45:0/2105898500 shutdown_connections 2024-06-06T04:32:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.947+0000 7f4a8afd3700 1 --2- 172.21.15.45:0/2105898500 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4a740a2d30 0x7f4a740a51f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.947+0000 7f4a8afd3700 1 --2- 172.21.15.45:0/2105898500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a84107400 0x7f4a8419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.947+0000 7f4a8afd3700 1 --2- 172.21.15.45:0/2105898500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a84101390 0x7f4a841a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.947+0000 7f4a8afd3700 1 --2- 172.21.15.45:0/2105898500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a84101d50 0x7f4a841a0fa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.947+0000 7f4a8afd3700 1 -- 172.21.15.45:0/2105898500 >> 172.21.15.45:0/2105898500 conn(0x7f4a84078340 msgr2=0x7f4a8406a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.947+0000 7f4a8afd3700 1 -- 172.21.15.45:0/2105898500 shutdown_connections 2024-06-06T04:32:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:18.947+0000 7f4a8afd3700 1 -- 172.21.15.45:0/2105898500 wait complete. 2024-06-06T04:32:19.766 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-06T04:32:20.041 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:32:20.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:20 smithi045 ceph-mon[32299]: pgmap v457: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:20.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:20 smithi086 ceph-mon[36053]: pgmap v457: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:20.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:20 smithi154 ceph-mon[36428]: pgmap v457: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.965+0000 7ff3afa13700 1 -- 172.21.15.45:0/4284277669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a8107400 msgr2=0x7ff3a81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.965+0000 7ff3afa13700 1 --2- 172.21.15.45:0/4284277669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a8107400 0x7ff3a81077e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009990 tx=0x7ff3a4009ca0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.965+0000 7ff3afa13700 1 -- 172.21.15.45:0/4284277669 shutdown_connections 2024-06-06T04:32:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.965+0000 7ff3afa13700 1 --2- 172.21.15.45:0/4284277669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3a8101d50 0x7ff3a810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.965+0000 7ff3afa13700 1 --2- 172.21.15.45:0/4284277669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a8107400 0x7ff3a81077e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.965+0000 7ff3afa13700 1 --2- 172.21.15.45:0/4284277669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a8101390 0x7ff3a8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.965+0000 7ff3afa13700 1 -- 172.21.15.45:0/4284277669 >> 172.21.15.45:0/4284277669 conn(0x7ff3a8078340 msgr2=0x7ff3a8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.966+0000 7ff3afa13700 1 -- 172.21.15.45:0/4284277669 shutdown_connections 2024-06-06T04:32:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.966+0000 7ff3afa13700 1 -- 172.21.15.45:0/4284277669 wait complete. 2024-06-06T04:32:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.966+0000 7ff3afa13700 1 Processor -- start 2024-06-06T04:32:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.966+0000 7ff3afa13700 1 -- start start 2024-06-06T04:32:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.967+0000 7ff3afa13700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a8101390 0x7ff3a819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.967+0000 7ff3afa13700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3a8101d50 0x7ff3a819dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.967+0000 7ff3afa13700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a8107400 0x7ff3a81978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.967+0000 7ff3afa13700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81115c0 con 0x7ff3a8107400 2024-06-06T04:32:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.967+0000 7ff3afa13700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a8111440 con 0x7ff3a8101d50 2024-06-06T04:32:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.967+0000 7ff3afa13700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a8111740 con 0x7ff3a8101390 2024-06-06T04:32:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.967+0000 7ff3ad7af700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a8101390 0x7ff3a819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.967+0000 7ff3acfae700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3a8101d50 0x7ff3a819dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.967+0000 7ff3ad7af700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a8101390 0x7ff3a819d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:53594/0 (socket says 172.21.15.45:53594) 2024-06-06T04:32:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.967+0000 7ff3acfae700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3a8101d50 0x7ff3a819dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:42026/0 (socket says 172.21.15.45:42026) 2024-06-06T04:32:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.967+0000 7ff3adfb0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a8107400 0x7ff3a81978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.967+0000 7ff3ad7af700 1 -- 172.21.15.45:0/886670965 learned_addr learned my addr 172.21.15.45:0/886670965 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.967+0000 7ff3ad7af700 1 -- 172.21.15.45:0/886670965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3a8101d50 msgr2=0x7ff3a819dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.967+0000 7ff3ad7af700 1 --2- 172.21.15.45:0/886670965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3a8101d50 0x7ff3a819dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.967+0000 7ff3ad7af700 1 -- 172.21.15.45:0/886670965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a8107400 msgr2=0x7ff3a81978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.968+0000 7ff3ad7af700 1 --2- 172.21.15.45:0/886670965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a8107400 0x7ff3a81978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.968+0000 7ff3ad7af700 1 -- 172.21.15.45:0/886670965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a8198000 con 0x7ff3a8101390 2024-06-06T04:32:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.968+0000 7ff3ad7af700 1 --2- 172.21.15.45:0/886670965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a8101390 0x7ff3a819d6e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4005910 tx=0x7ff3a40047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.968+0000 7ff3adfb0700 1 --2- 172.21.15.45:0/886670965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a8107400 0x7ff3a81978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.968+0000 7ff39e7fc700 1 -- 172.21.15.45:0/886670965 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4026070 con 0x7ff3a8101390 2024-06-06T04:32:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.968+0000 7ff39e7fc700 1 -- 172.21.15.45:0/886670965 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3a4003f10 con 0x7ff3a8101390 2024-06-06T04:32:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.968+0000 7ff39e7fc700 1 -- 172.21.15.45:0/886670965 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a400fb70 con 0x7ff3a8101390 2024-06-06T04:32:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.968+0000 7ff3afa13700 1 -- 172.21.15.45:0/886670965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a8198290 con 0x7ff3a8101390 2024-06-06T04:32:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.968+0000 7ff3afa13700 1 -- 172.21.15.45:0/886670965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a806a170 con 0x7ff3a8101390 2024-06-06T04:32:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.972+0000 7ff39e7fc700 1 -- 172.21.15.45:0/886670965 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7ff3a4029070 con 0x7ff3a8101390 2024-06-06T04:32:21.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.972+0000 7ff3ba296b80 1 Processor -- start 2024-06-06T04:32:21.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.972+0000 7ff3ba296b80 1 -- start start 2024-06-06T04:32:21.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.973+0000 7ff3ba296b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585976aef60 0x5585976af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.973+0000 7ff3ba296b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976af880 0x5585976a9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.973+0000 7ff3ba296b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585976a97a0 0x5585976a9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.973+0000 7ff39e7fc700 1 --2- 172.21.15.45:0/886670965 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff3940a2cf0 0x7ff3940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.973+0000 7ff3ba296b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55859759bf60 con 0x5585976af880 2024-06-06T04:32:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.973+0000 7ff3ba296b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5585975fa850 con 0x5585976a97a0 2024-06-06T04:32:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.973+0000 7ff3ba296b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558597562460 con 0x5585976aef60 2024-06-06T04:32:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.973+0000 7ff3ad7af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976af880 0x5585976a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.973+0000 7ff3adfb0700 1 --2- 172.21.15.45:0/886670965 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff3940a2cf0 0x7ff3940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.973+0000 7ff3ad7af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976af880 0x5585976a9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57408/0 (socket says 172.21.15.45:57408) 2024-06-06T04:32:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.973+0000 7ff3ad7af700 1 -- 172.21.15.45:0/879388858 learned_addr learned my addr 172.21.15.45:0/879388858 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.973+0000 7ff3adfb0700 1 --2- 172.21.15.45:0/879388858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585976aef60 0x5585976af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.973+0000 7ff3acfae700 1 --2- 172.21.15.45:0/879388858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585976a97a0 0x5585976a9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.973+0000 7ff39e7fc700 1 -- 172.21.15.45:0/886670965 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7ff3a40cfe00 con 0x7ff3a8101390 2024-06-06T04:32:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.974+0000 7ff3adfb0700 1 -- 172.21.15.45:0/879388858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585976a97a0 msgr2=0x5585976a9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.974+0000 7ff3adfb0700 1 --2- 172.21.15.45:0/879388858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585976a97a0 0x5585976a9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.974+0000 7ff3adfb0700 1 -- 172.21.15.45:0/879388858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976af880 msgr2=0x5585976a9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.974+0000 7ff3adfb0700 1 --2- 172.21.15.45:0/879388858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976af880 0x5585976a9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.974+0000 7ff3adfb0700 1 -- 172.21.15.45:0/879388858 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585975498d0 con 0x5585976aef60 2024-06-06T04:32:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.974+0000 7ff3acfae700 1 --2- 172.21.15.45:0/879388858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585976a97a0 0x5585976a9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.974+0000 7ff3ad7af700 1 --2- 172.21.15.45:0/879388858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976af880 0x5585976a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.974+0000 7ff3adfb0700 1 --2- 172.21.15.45:0/886670965 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff3940a2cf0 0x7ff3940a51b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0007b90 tx=0x7ff3a00112d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:21.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.974+0000 7ff3adfb0700 1 --2- 172.21.15.45:0/879388858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585976aef60 0x5585976af340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0004a50 tx=0x7ff3a001b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f4106f6af5d7ea5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.974+0000 7ff3937fe700 1 -- 172.21.15.45:0/879388858 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0004bd0 con 0x5585976aef60 2024-06-06T04:32:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.974+0000 7ff3937fe700 1 -- 172.21.15.45:0/879388858 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3a0019dc0 con 0x5585976aef60 2024-06-06T04:32:21.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.974+0000 7ff3937fe700 1 -- 172.21.15.45:0/879388858 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00327a0 con 0x5585976aef60 2024-06-06T04:32:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.975+0000 7ff3ba296b80 1 -- 172.21.15.45:0/879388858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585976aef60 msgr2=0x5585976af340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.975+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/879388858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585976aef60 0x5585976af340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0004a50 tx=0x7ff3a001b000 comp rx=0 tx=0).stop 2024-06-06T04:32:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.975+0000 7ff3ba296b80 1 -- 172.21.15.45:0/879388858 shutdown_connections 2024-06-06T04:32:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.975+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/879388858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585976a97a0 0x5585976a9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.975+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/879388858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585976aef60 0x5585976af340 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.975+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/879388858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976af880 0x5585976a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.975+0000 7ff3ba296b80 1 -- 172.21.15.45:0/879388858 >> 172.21.15.45:0/879388858 conn(0x5585975d6080 msgr2=0x5585976a4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.975+0000 7ff3ba296b80 1 -- 172.21.15.45:0/879388858 shutdown_connections 2024-06-06T04:32:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.975+0000 7ff3ba296b80 1 -- 172.21.15.45:0/879388858 wait complete. 2024-06-06T04:32:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.976+0000 7ff3ba296b80 1 -- start start 2024-06-06T04:32:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.976+0000 7ff3ba296b80 1 Processor -- start 2024-06-06T04:32:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.976+0000 7ff3ba296b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976a97a0 0x5585976b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:21.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.976+0000 7ff3ba296b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585976aef60 0x5585976b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.976+0000 7ff3ba296b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585976af880 0x5585976b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.976+0000 7ff3ba296b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55859759afc0 con 0x5585976a97a0 2024-06-06T04:32:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.976+0000 7ff3ba296b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5585975f6810 con 0x5585976af880 2024-06-06T04:32:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.976+0000 7ff3ba296b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5585976b6250 con 0x5585976aef60 2024-06-06T04:32:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.976+0000 7ff3ba296b80 10 client.0 _ll_register_callbacks cb 0x5585975905b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.976+0000 7ff3acfae700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585976aef60 0x5585976b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.976+0000 7ff3ad7af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976a97a0 0x5585976b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.976+0000 7ff3ad7af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976a97a0 0x5585976b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57414/0 (socket says 172.21.15.45:57414) 2024-06-06T04:32:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.976+0000 7ff3ad7af700 1 -- 172.21.15.45:0/2254283371 learned_addr learned my addr 172.21.15.45:0/2254283371 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.977+0000 7ff3adfb0700 1 --2- 172.21.15.45:0/2254283371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585976af880 0x5585976b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.977+0000 7ff3ad7af700 1 -- 172.21.15.45:0/2254283371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585976aef60 msgr2=0x5585976b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.977+0000 7ff3ad7af700 1 --2- 172.21.15.45:0/2254283371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585976aef60 0x5585976b2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.977+0000 7ff3ad7af700 1 -- 172.21.15.45:0/2254283371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585976af880 msgr2=0x5585976b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.977+0000 7ff3ad7af700 1 --2- 172.21.15.45:0/2254283371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585976af880 0x5585976b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.977+0000 7ff3ad7af700 1 -- 172.21.15.45:0/2254283371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585975c0750 con 0x5585976a97a0 2024-06-06T04:32:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.977+0000 7ff3ad7af700 1 --2- 172.21.15.45:0/2254283371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976a97a0 0x5585976b56d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40985e0 tx=0x7ff3a409bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb208adbde7a1871 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.977+0000 7ff390ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.977+0000 7ff390ff9700 1 -- 172.21.15.45:0/2254283371 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a409c8d0 con 0x5585976a97a0 2024-06-06T04:32:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.977+0000 7ff390ff9700 1 -- 172.21.15.45:0/2254283371 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7ff3a40bc0d0 con 0x5585976a97a0 2024-06-06T04:32:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.977+0000 7ff3ba296b80 1 -- 172.21.15.45:0/2254283371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3a0019dc0 con 0x5585976a97a0 2024-06-06T04:32:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.978+0000 7ff390ff9700 1 -- 172.21.15.45:0/2254283371 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff3a409c1d0 con 0x5585976a97a0 2024-06-06T04:32:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.978+0000 7ff3ba296b80 10 client.15441 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.978+0000 7ff3ba296b80 1 -- 172.21.15.45:0/2254283371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5585976c07a0 con 0x5585976a97a0 2024-06-06T04:32:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.978+0000 7ff390ff9700 1 -- 172.21.15.45:0/2254283371 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3a0019dc0 con 0x5585976a97a0 2024-06-06T04:32:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.978+0000 7ff390ff9700 1 -- 172.21.15.45:0/2254283371 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff3a40a0180 con 0x5585976a97a0 2024-06-06T04:32:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.978+0000 7ff3ba296b80 10 client.15441 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.979+0000 7ff3ba296b80 10 client.15441 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T04:32:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.979+0000 7ff3ba296b80 20 client.15441 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.979+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/2254283371 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x5585976c2e00 0x5585976c5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.979+0000 7ff3ba296b80 4 client.15441 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:32:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.979+0000 7ff3ba296b80 1 -- 172.21.15.45:0/2254283371 --> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558597607d50 con 0x5585976c2e00 2024-06-06T04:32:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.979+0000 7ff3acfae700 1 --2- 172.21.15.45:0/2254283371 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x5585976c2e00 0x5585976c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.979+0000 7ff390ff9700 10 client.15441 ms_handle_connect on v2:172.21.15.154:6838/982197202 2024-06-06T04:32:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.979+0000 7ff3acfae700 1 --2- 172.21.15.45:0/2254283371 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x5585976c2e00 0x5585976c5220 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34aa65a3eb4506ba server_cookie=dabfadb6d1504cc5 in_seq=0 out_seq=0 2024-06-06T04:32:21.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.980+0000 7ff390ff9700 1 -- 172.21.15.45:0/2254283371 <== mds.1 v2:172.21.15.154:6838/982197202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558597607d50 con 0x5585976c2e00 2024-06-06T04:32:21.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.980+0000 7ff390ff9700 10 client.15441 handle_command_reply: tid=0 2024-06-06T04:32:21.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.980+0000 7ff3ba296b80 1 client.15441 shutdown 2024-06-06T04:32:21.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.980+0000 7ff3ba296b80 1 -- 172.21.15.45:0/2254283371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976a97a0 msgr2=0x5585976b56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:21.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.980+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/2254283371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976a97a0 0x5585976b56d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40985e0 tx=0x7ff3a409bd90 comp rx=0 tx=0).stop 2024-06-06T04:32:21.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.980+0000 7ff3ba296b80 1 -- 172.21.15.45:0/2254283371 shutdown_connections 2024-06-06T04:32:21.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.980+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/2254283371 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x5585976c2e00 0x5585976c5220 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.980+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/2254283371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585976af880 0x5585976b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.980+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/2254283371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976a97a0 0x5585976b56d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.980+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/2254283371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585976aef60 0x5585976b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.980+0000 7ff3ba296b80 1 -- 172.21.15.45:0/2254283371 >> 172.21.15.45:0/2254283371 conn(0x5585975d6080 msgr2=0x5585976a3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:21.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.980+0000 7ff390ff9700 0 client.15441 ms_handle_reset on v2:172.21.15.154:6838/982197202 2024-06-06T04:32:21.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.981+0000 7ff3ba296b80 1 -- 172.21.15.45:0/2254283371 shutdown_connections 2024-06-06T04:32:21.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.981+0000 7ff3ba296b80 1 -- 172.21.15.45:0/2254283371 wait complete. 2024-06-06T04:32:21.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.981+0000 7ff3ba296b80 20 client.15441 trim_cache size 0 max 16384 2024-06-06T04:32:21.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.996+0000 7ff3ba296b80 1 Processor -- start 2024-06-06T04:32:21.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.996+0000 7ff3ba296b80 1 -- start start 2024-06-06T04:32:21.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.996+0000 7ff3ba296b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585975d6080 0x5585976a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:21.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.997+0000 7ff3ba296b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976a97a0 0x5585976a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:21.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.997+0000 7ff3ba296b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585976aef60 0x5585976b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:21.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.997+0000 7ff3ba296b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81ae6c0 con 0x5585976a97a0 2024-06-06T04:32:21.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.997+0000 7ff3ba296b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81abf50 con 0x5585975d6080 2024-06-06T04:32:21.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.997+0000 7ff3ba296b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81a9da0 con 0x5585976aef60 2024-06-06T04:32:21.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.997+0000 7ff3acfae700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976a97a0 0x5585976a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:21.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.997+0000 7ff3acfae700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976a97a0 0x5585976a43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57416/0 (socket says 172.21.15.45:57416) 2024-06-06T04:32:21.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.997+0000 7ff3acfae700 1 -- 172.21.15.45:0/1136621846 learned_addr learned my addr 172.21.15.45:0/1136621846 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:21.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.997+0000 7ff3ad7af700 1 --2- 172.21.15.45:0/1136621846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585975d6080 0x5585976a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:21.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.997+0000 7ff3acfae700 1 -- 172.21.15.45:0/1136621846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585976aef60 msgr2=0x5585976b2a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T04:32:21.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.997+0000 7ff3acfae700 1 --2- 172.21.15.45:0/1136621846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585976aef60 0x5585976b2a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.997+0000 7ff3acfae700 1 -- 172.21.15.45:0/1136621846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585975d6080 msgr2=0x5585976a3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:21.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.997+0000 7ff3acfae700 1 --2- 172.21.15.45:0/1136621846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585975d6080 0x5585976a3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:21.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.997+0000 7ff3acfae700 1 -- 172.21.15.45:0/1136621846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585976b0f90 con 0x5585976a97a0 2024-06-06T04:32:21.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.997+0000 7ff3acfae700 1 --2- 172.21.15.45:0/1136621846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976a97a0 0x5585976a43c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff398005950 tx=0x7ff39800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad0000df02b0174 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.998+0000 7ff392ffd700 1 -- 172.21.15.45:0/1136621846 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff398023020 con 0x5585976a97a0 2024-06-06T04:32:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.998+0000 7ff392ffd700 1 -- 172.21.15.45:0/1136621846 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff398016640 con 0x5585976a97a0 2024-06-06T04:32:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.998+0000 7ff392ffd700 1 -- 172.21.15.45:0/1136621846 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff398003ce0 con 0x5585976a97a0 2024-06-06T04:32:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.998+0000 7ff3ba296b80 1 -- 172.21.15.45:0/1136621846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976a97a0 msgr2=0x5585976a43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.998+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/1136621846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976a97a0 0x5585976a43c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff398005950 tx=0x7ff39800b410 comp rx=0 tx=0).stop 2024-06-06T04:32:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.998+0000 7ff3ba296b80 1 -- 172.21.15.45:0/1136621846 shutdown_connections 2024-06-06T04:32:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.998+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/1136621846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585976aef60 0x5585976b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.998+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/1136621846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585975d6080 0x5585976a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.998+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/1136621846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976a97a0 0x5585976a43c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.998+0000 7ff3ba296b80 1 -- 172.21.15.45:0/1136621846 >> 172.21.15.45:0/1136621846 conn(0x5585976b07f0 msgr2=0x5585976b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.999+0000 7ff3ba296b80 1 -- 172.21.15.45:0/1136621846 shutdown_connections 2024-06-06T04:32:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.999+0000 7ff3ba296b80 1 -- 172.21.15.45:0/1136621846 wait complete. 2024-06-06T04:32:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.999+0000 7ff3ba296b80 1 -- start start 2024-06-06T04:32:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:21.999+0000 7ff3ba296b80 1 Processor -- start 2024-06-06T04:32:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3ba296b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585975d6080 0x5585976bb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3ba296b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976aef60 0x5585976bbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3acfae700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976aef60 0x5585976bbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3acfae700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976aef60 0x5585976bbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57426/0 (socket says 172.21.15.45:57426) 2024-06-06T04:32:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3acfae700 1 -- 172.21.15.45:0/624590632 learned_addr learned my addr 172.21.15.45:0/624590632 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/624590632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585976af880 0x5585976b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3ba296b80 1 -- 172.21.15.45:0/624590632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81aa410 con 0x5585976aef60 2024-06-06T04:32:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3ba296b80 1 -- 172.21.15.45:0/624590632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81ab880 con 0x5585976af880 2024-06-06T04:32:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3ba296b80 1 -- 172.21.15.45:0/624590632 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a81aaae0 con 0x5585975d6080 2024-06-06T04:32:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3ad7af700 1 --2- 172.21.15.45:0/624590632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585975d6080 0x5585976bb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3adfb0700 1 --2- 172.21.15.45:0/624590632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585976af880 0x5585976b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3acfae700 1 -- 172.21.15.45:0/624590632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585975d6080 msgr2=0x5585976bb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3acfae700 1 --2- 172.21.15.45:0/624590632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585975d6080 0x5585976bb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3acfae700 1 -- 172.21.15.45:0/624590632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585976af880 msgr2=0x5585976b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3acfae700 1 --2- 172.21.15.45:0/624590632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585976af880 0x5585976b6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3acfae700 1 -- 172.21.15.45:0/624590632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585976b72f0 con 0x5585976aef60 2024-06-06T04:32:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.000+0000 7ff3ad7af700 1 --2- 172.21.15.45:0/624590632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585975d6080 0x5585976bb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.001+0000 7ff3acfae700 1 --2- 172.21.15.45:0/624590632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976aef60 0x5585976bbb00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x5585976aec00 tx=0x7ff398003830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf098b151afa82e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.001+0000 7ff390ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.001+0000 7ff3ba296b80 10 client.0 _ll_register_callbacks cb 0x5585975905b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.001+0000 7ff3ba296b80 1 -- 172.21.15.45:0/624590632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff398016640 con 0x5585976aef60 2024-06-06T04:32:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.001+0000 7ff390ff9700 1 -- 172.21.15.45:0/624590632 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff398016190 con 0x5585976aef60 2024-06-06T04:32:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.001+0000 7ff390ff9700 1 -- 172.21.15.45:0/624590632 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7ff398016330 con 0x5585976aef60 2024-06-06T04:32:22.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.001+0000 7ff390ff9700 1 -- 172.21.15.45:0/624590632 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff398003980 con 0x5585976aef60 2024-06-06T04:32:22.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.001+0000 7ff3ba296b80 10 client.15447 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:22.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.001+0000 7ff3ba296b80 1 -- 172.21.15.45:0/624590632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55859772d2f0 con 0x5585976aef60 2024-06-06T04:32:22.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.002+0000 7ff390ff9700 1 -- 172.21.15.45:0/624590632 <== mon.0 v2:172.21.15.45:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff398003b20 con 0x5585976aef60 2024-06-06T04:32:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.002+0000 7ff3ba296b80 10 client.15447 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.002+0000 7ff3ba296b80 10 client.15447 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T04:32:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.002+0000 7ff3ba296b80 20 client.15447 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.002+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/624590632 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55859772d8b0 0x55859772fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.002+0000 7ff3ba296b80 4 client.15447 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T04:32:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.002+0000 7ff3ba296b80 1 -- 172.21.15.45:0/624590632 --> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5585976022d0 con 0x55859772d8b0 2024-06-06T04:32:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.002+0000 7ff3ad7af700 1 --2- 172.21.15.45:0/624590632 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55859772d8b0 0x55859772fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.003+0000 7ff3ad7af700 1 --2- 172.21.15.45:0/624590632 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55859772d8b0 0x55859772fd70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8c37c7b11351a789 server_cookie=c00e68bb63c54c8b in_seq=0 out_seq=0 2024-06-06T04:32:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.003+0000 7ff390ff9700 10 client.15447 ms_handle_connect on v2:172.21.15.154:6838/982197202 2024-06-06T04:32:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.003+0000 7ff390ff9700 1 -- 172.21.15.45:0/624590632 <== mds.1 v2:172.21.15.154:6838/982197202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5585976022d0 con 0x55859772d8b0 2024-06-06T04:32:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.003+0000 7ff390ff9700 10 client.15447 handle_command_reply: tid=0 2024-06-06T04:32:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.003+0000 7ff3ba296b80 1 client.15447 shutdown 2024-06-06T04:32:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.003+0000 7ff3ba296b80 1 -- 172.21.15.45:0/624590632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976aef60 msgr2=0x5585976bbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.003+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/624590632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976aef60 0x5585976bbb00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x5585976aec00 tx=0x7ff398003830 comp rx=0 tx=0).stop 2024-06-06T04:32:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.003+0000 7ff3ba296b80 1 -- 172.21.15.45:0/624590632 shutdown_connections 2024-06-06T04:32:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.003+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/624590632 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55859772d8b0 0x55859772fd70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.003+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/624590632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585976af880 0x5585976b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.003+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/624590632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5585975d6080 0x5585976bb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.003+0000 7ff3ba296b80 1 --2- 172.21.15.45:0/624590632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585976aef60 0x5585976bbb00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.003+0000 7ff3ba296b80 1 -- 172.21.15.45:0/624590632 >> 172.21.15.45:0/624590632 conn(0x5585976b07f0 msgr2=0x5585976ae3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.003+0000 7ff390ff9700 0 client.15447 ms_handle_reset on v2:172.21.15.154:6838/982197202 2024-06-06T04:32:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.004+0000 7ff3ba296b80 1 -- 172.21.15.45:0/624590632 shutdown_connections 2024-06-06T04:32:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.004+0000 7ff3ba296b80 1 -- 172.21.15.45:0/624590632 wait complete. 2024-06-06T04:32:22.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.004+0000 7ff3ba296b80 20 client.15447 trim_cache size 0 max 16384 2024-06-06T04:32:22.009 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-06-06T04:32:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.004+0000 7ff3afa13700 1 -- 172.21.15.45:0/886670965 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff3940a2cf0 msgr2=0x7ff3940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.005+0000 7ff3afa13700 1 --2- 172.21.15.45:0/886670965 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff3940a2cf0 0x7ff3940a51b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0007b90 tx=0x7ff3a00112d0 comp rx=0 tx=0).stop 2024-06-06T04:32:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.005+0000 7ff3afa13700 1 -- 172.21.15.45:0/886670965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a8101390 msgr2=0x7ff3a819d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.005+0000 7ff3afa13700 1 --2- 172.21.15.45:0/886670965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a8101390 0x7ff3a819d6e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4005910 tx=0x7ff3a40047b0 comp rx=0 tx=0).stop 2024-06-06T04:32:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.005+0000 7ff3afa13700 1 -- 172.21.15.45:0/886670965 shutdown_connections 2024-06-06T04:32:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.005+0000 7ff3afa13700 1 --2- 172.21.15.45:0/886670965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3a8107400 0x7ff3a81978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.005+0000 7ff3afa13700 1 --2- 172.21.15.45:0/886670965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a8101390 0x7ff3a819d6e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.005+0000 7ff3afa13700 1 --2- 172.21.15.45:0/886670965 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7ff3940a2cf0 0x7ff3940a51b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.005+0000 7ff3afa13700 1 --2- 172.21.15.45:0/886670965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3a8101d50 0x7ff3a819dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.005+0000 7ff3afa13700 1 -- 172.21.15.45:0/886670965 >> 172.21.15.45:0/886670965 conn(0x7ff3a8078340 msgr2=0x7ff3a80fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.005+0000 7ff3afa13700 1 -- 172.21.15.45:0/886670965 shutdown_connections 2024-06-06T04:32:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:22.005+0000 7ff3afa13700 1 -- 172.21.15.45:0/886670965 wait complete. 2024-06-06T04:32:22.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:22 smithi045 ceph-mon[32299]: pgmap v458: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:22.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:22 smithi086 ceph-mon[36053]: pgmap v458: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:22.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:22 smithi154 ceph-mon[36428]: pgmap v458: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:22.725 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-06T04:32:23.012 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:32:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.495+0000 7f2e05be7640 1 -- 172.21.15.45:0/3034611111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e00101a50 msgr2=0x7f2e00101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.495+0000 7f2e05be7640 1 --2- 172.21.15.45:0/3034611111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e00101a50 0x7f2e00101e70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009900 tx=0x7f2df002a410 comp rx=0 tx=0).stop 2024-06-06T04:32:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.495+0000 7f2e05be7640 1 -- 172.21.15.45:0/3034611111 shutdown_connections 2024-06-06T04:32:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.495+0000 7f2e05be7640 1 --2- 172.21.15.45:0/3034611111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e00103600 0x7f2e00109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.495+0000 7f2e05be7640 1 --2- 172.21.15.45:0/3034611111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e00102c40 0x7f2e001030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.495+0000 7f2e05be7640 1 --2- 172.21.15.45:0/3034611111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e00101a50 0x7f2e00101e70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.495+0000 7f2e05be7640 1 -- 172.21.15.45:0/3034611111 >> 172.21.15.45:0/3034611111 conn(0x7f2e000fcfc0 msgr2=0x7f2e000ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.495+0000 7f2e05be7640 1 -- 172.21.15.45:0/3034611111 shutdown_connections 2024-06-06T04:32:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.495+0000 7f2e05be7640 1 -- 172.21.15.45:0/3034611111 wait complete. 2024-06-06T04:32:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.496+0000 7f2e05be7640 1 Processor -- start 2024-06-06T04:32:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.496+0000 7f2e05be7640 1 -- start start 2024-06-06T04:32:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.497+0000 7f2e05be7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e00101a50 0x7f2e00197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.497+0000 7f2e05be7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e00102c40 0x7f2e00198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.497+0000 7f2e05be7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e00103600 0x7f2e0019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.497+0000 7f2e05be7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0010d570 con 0x7f2e00103600 2024-06-06T04:32:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.497+0000 7f2e05be7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0010d3f0 con 0x7f2e00101a50 2024-06-06T04:32:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.497+0000 7f2e05be7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0010d6f0 con 0x7f2e00102c40 2024-06-06T04:32:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.497+0000 7f2dfffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e00103600 0x7f2e0019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.497+0000 7f2dfffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e00103600 0x7f2e0019f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57448/0 (socket says 172.21.15.45:57448) 2024-06-06T04:32:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.497+0000 7f2dfffff640 1 -- 172.21.15.45:0/2569504726 learned_addr learned my addr 172.21.15.45:0/2569504726 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.497+0000 7f2dff7fe640 1 --2- 172.21.15.45:0/2569504726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e00101a50 0x7f2e00197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.497+0000 7f2dfeffd640 1 --2- 172.21.15.45:0/2569504726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e00102c40 0x7f2e00198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.497+0000 7f2dfffff640 1 -- 172.21.15.45:0/2569504726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e00102c40 msgr2=0x7f2e00198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.497+0000 7f2dfffff640 1 --2- 172.21.15.45:0/2569504726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e00102c40 0x7f2e00198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.497+0000 7f2dfffff640 1 -- 172.21.15.45:0/2569504726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e00101a50 msgr2=0x7f2e00197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.497+0000 7f2dfffff640 1 --2- 172.21.15.45:0/2569504726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e00101a50 0x7f2e00197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.497+0000 7f2dfffff640 1 -- 172.21.15.45:0/2569504726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0019f810 con 0x7f2e00103600 2024-06-06T04:32:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.498+0000 7f2dfffff640 1 --2- 172.21.15.45:0/2569504726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e00103600 0x7f2e0019f1a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2df800eb10 tx=0x7f2df800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.498+0000 7f2dfcff9640 1 -- 172.21.15.45:0/2569504726 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8004b20 con 0x7f2e00103600 2024-06-06T04:32:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.498+0000 7f2dfcff9640 1 -- 172.21.15.45:0/2569504726 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2df8004cc0 con 0x7f2e00103600 2024-06-06T04:32:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.498+0000 7f2e05be7640 1 -- 172.21.15.45:0/2569504726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0019fb00 con 0x7f2e00103600 2024-06-06T04:32:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.498+0000 7f2dfcff9640 1 -- 172.21.15.45:0/2569504726 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8004b20 con 0x7f2e00103600 2024-06-06T04:32:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.498+0000 7f2e05be7640 1 -- 172.21.15.45:0/2569504726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e001a0000 con 0x7f2e00103600 2024-06-06T04:32:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.501+0000 7f2e089d2740 1 Processor -- start 2024-06-06T04:32:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.502+0000 7f2dfcff9640 1 -- 172.21.15.45:0/2569504726 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f2df8005020 con 0x7f2e00103600 2024-06-06T04:32:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.503+0000 7f2dfcff9640 1 --2- 172.21.15.45:0/2569504726 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2de80a2ec0 0x7f2de80a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.503+0000 7f2dfcff9640 1 -- 172.21.15.45:0/2569504726 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f2df8098180 con 0x7f2e00103600 2024-06-06T04:32:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.503+0000 7f2e089d2740 1 -- start start 2024-06-06T04:32:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.503+0000 7f2e089d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef13f010 0x5559ef13f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.503+0000 7f2e089d2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef13fa00 0x5559ef144500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.503+0000 7f2e089d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef144b70 0x5559ef147000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.503+0000 7f2e089d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5559ef0a24f0 con 0x5559ef13f010 2024-06-06T04:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.503+0000 7f2e089d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5559ef040870 con 0x5559ef144b70 2024-06-06T04:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.503+0000 7f2e089d2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5559ef09e4b0 con 0x5559ef13fa00 2024-06-06T04:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.503+0000 7f2dfeffd640 1 --2- 172.21.15.45:0/2569504726 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2de80a2ec0 0x7f2de80a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.503+0000 7f2dff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef13f010 0x5559ef13f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.503+0000 7f2dff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef13f010 0x5559ef13f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57462/0 (socket says 172.21.15.45:57462) 2024-06-06T04:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.503+0000 7f2dff7fe640 1 -- 172.21.15.45:0/3791025445 learned_addr learned my addr 172.21.15.45:0/3791025445 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.504+0000 7f2dfffff640 1 --2- 172.21.15.45:0/3791025445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef144b70 0x5559ef147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.504+0000 7f2dff7fe640 1 -- 172.21.15.45:0/3791025445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef13fa00 msgr2=0x5559ef144500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T04:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.504+0000 7f2dff7fe640 1 --2- 172.21.15.45:0/3791025445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef13fa00 0x5559ef144500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.504+0000 7f2dff7fe640 1 -- 172.21.15.45:0/3791025445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef144b70 msgr2=0x5559ef147000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.504+0000 7f2dff7fe640 1 --2- 172.21.15.45:0/3791025445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef144b70 0x5559ef147000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.504+0000 7f2dff7fe640 1 -- 172.21.15.45:0/3791025445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ef05c9b0 con 0x5559ef13f010 2024-06-06T04:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.504+0000 7f2dfeffd640 1 --2- 172.21.15.45:0/2569504726 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2de80a2ec0 0x7f2de80a5380 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e00199100 tx=0x7f2df4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.504+0000 7f2dff7fe640 1 --2- 172.21.15.45:0/3791025445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef13f010 0x5559ef13f430 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009900 tx=0x7f2df0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ca1a2a55d21a86 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.504+0000 7f2de5ffb640 1 -- 172.21.15.45:0/3791025445 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0035030 con 0x5559ef13f010 2024-06-06T04:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.505+0000 7f2de5ffb640 1 -- 172.21.15.45:0/3791025445 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2df0040070 con 0x5559ef13f010 2024-06-06T04:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.505+0000 7f2de5ffb640 1 -- 172.21.15.45:0/3791025445 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df00039f0 con 0x5559ef13f010 2024-06-06T04:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.505+0000 7f2e089d2740 1 -- 172.21.15.45:0/3791025445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef13f010 msgr2=0x5559ef13f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.505+0000 7f2e089d2740 1 --2- 172.21.15.45:0/3791025445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef13f010 0x5559ef13f430 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009900 tx=0x7f2df0005c00 comp rx=0 tx=0).stop 2024-06-06T04:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.505+0000 7f2e089d2740 1 -- 172.21.15.45:0/3791025445 shutdown_connections 2024-06-06T04:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.505+0000 7f2e089d2740 1 --2- 172.21.15.45:0/3791025445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef144b70 0x5559ef147000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.505+0000 7f2e089d2740 1 --2- 172.21.15.45:0/3791025445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef13fa00 0x5559ef144500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.506+0000 7f2e089d2740 1 --2- 172.21.15.45:0/3791025445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef13f010 0x5559ef13f430 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.506+0000 7f2e089d2740 1 -- 172.21.15.45:0/3791025445 >> 172.21.15.45:0/3791025445 conn(0x5559ef13a680 msgr2=0x5559ef13aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.506+0000 7f2e089d2740 1 -- 172.21.15.45:0/3791025445 shutdown_connections 2024-06-06T04:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.506+0000 7f2e089d2740 1 -- 172.21.15.45:0/3791025445 wait complete. 2024-06-06T04:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.506+0000 7f2e089d2740 1 -- start start 2024-06-06T04:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.507+0000 7f2e089d2740 1 Processor -- start 2024-06-06T04:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.507+0000 7f2e089d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef13fa00 0x5559ef1494f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.507+0000 7f2e089d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef144b70 0x5559ef149a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.507+0000 7f2e089d2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef14ea60 0x5559ef150e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.507+0000 7f2e089d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5559ef147ac0 con 0x5559ef13fa00 2024-06-06T04:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.507+0000 7f2e089d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5559ef02bae0 con 0x5559ef144b70 2024-06-06T04:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.507+0000 7f2e089d2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5559ef147c40 con 0x5559ef14ea60 2024-06-06T04:32:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.507+0000 7f2dff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef13fa00 0x5559ef1494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.507+0000 7f2dff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef13fa00 0x5559ef1494f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57470/0 (socket says 172.21.15.45:57470) 2024-06-06T04:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.507+0000 7f2dff7fe640 1 -- 172.21.15.45:0/3284698576 learned_addr learned my addr 172.21.15.45:0/3284698576 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.507+0000 7f2dfffff640 1 --2- 172.21.15.45:0/3284698576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef14ea60 0x5559ef150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.507+0000 7f2e089d2740 10 client.0 _ll_register_callbacks cb 0x5559ef0649b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.508+0000 7f2dfeffd640 1 --2- 172.21.15.45:0/3284698576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef144b70 0x5559ef149a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.508+0000 7f2dff7fe640 1 -- 172.21.15.45:0/3284698576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef14ea60 msgr2=0x5559ef150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.508+0000 7f2dff7fe640 1 --2- 172.21.15.45:0/3284698576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef14ea60 0x5559ef150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.508+0000 7f2dff7fe640 1 -- 172.21.15.45:0/3284698576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef144b70 msgr2=0x5559ef149a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.508+0000 7f2dff7fe640 1 --2- 172.21.15.45:0/3284698576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef144b70 0x5559ef149a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.508+0000 7f2dff7fe640 1 -- 172.21.15.45:0/3284698576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559eedb6210 con 0x5559ef13fa00 2024-06-06T04:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.508+0000 7f2dff7fe640 1 --2- 172.21.15.45:0/3284698576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef13fa00 0x5559ef1494f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2df0036040 tx=0x7f2df0031830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bd285e005721aa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.508+0000 7f2ddeffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.508+0000 7f2ddeffd640 1 -- 172.21.15.45:0/3284698576 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df003c640 con 0x5559ef13fa00 2024-06-06T04:32:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.508+0000 7f2ddeffd640 1 -- 172.21.15.45:0/3284698576 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f2df003c7e0 con 0x5559ef13fa00 2024-06-06T04:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.508+0000 7f2e089d2740 1 -- 172.21.15.45:0/3284698576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2df0040070 con 0x5559ef13fa00 2024-06-06T04:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.509+0000 7f2ddeffd640 1 -- 172.21.15.45:0/3284698576 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2df005a050 con 0x5559ef13fa00 2024-06-06T04:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.509+0000 7f2e089d2740 10 client.15459 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.509+0000 7f2e089d2740 1 -- 172.21.15.45:0/3284698576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5559eee82620 con 0x5559ef13fa00 2024-06-06T04:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.509+0000 7f2ddeffd640 1 -- 172.21.15.45:0/3284698576 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2df0040070 con 0x5559ef13fa00 2024-06-06T04:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.509+0000 7f2ddeffd640 1 -- 172.21.15.45:0/3284698576 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2df004c8b0 con 0x5559ef13fa00 2024-06-06T04:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.509+0000 7f2e089d2740 10 client.15459 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.509+0000 7f2e089d2740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.509+0000 7f2e089d2740 20 client.15459 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.509+0000 7f2e089d2740 1 --2- 172.21.15.45:0/3284698576 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5559ef155320 0x5559ef157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.509+0000 7f2e089d2740 4 client.15459 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:32:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.509+0000 7f2e089d2740 1 -- 172.21.15.45:0/3284698576 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559ef0b5f10 con 0x5559ef155320 2024-06-06T04:32:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.510+0000 7f2dfeffd640 1 --2- 172.21.15.45:0/3284698576 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5559ef155320 0x5559ef157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.510+0000 7f2ddeffd640 10 client.15459 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.510+0000 7f2dfeffd640 1 --2- 172.21.15.45:0/3284698576 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5559ef155320 0x5559ef157710 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99210438a8b6cc18 server_cookie=76a60de1451c29a2 in_seq=0 out_seq=0 2024-06-06T04:32:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.511+0000 7f2ddeffd640 1 -- 172.21.15.45:0/3284698576 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5559ef0b5f10 con 0x5559ef155320 2024-06-06T04:32:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.511+0000 7f2ddeffd640 10 client.15459 handle_command_reply: tid=0 2024-06-06T04:32:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.511+0000 7f2e089d2740 1 client.15459 shutdown 2024-06-06T04:32:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.511+0000 7f2e089d2740 1 -- 172.21.15.45:0/3284698576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef13fa00 msgr2=0x5559ef1494f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.511+0000 7f2e089d2740 1 --2- 172.21.15.45:0/3284698576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef13fa00 0x5559ef1494f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2df0036040 tx=0x7f2df0031830 comp rx=0 tx=0).stop 2024-06-06T04:32:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.511+0000 7f2e089d2740 1 -- 172.21.15.45:0/3284698576 shutdown_connections 2024-06-06T04:32:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.511+0000 7f2e089d2740 1 --2- 172.21.15.45:0/3284698576 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5559ef155320 0x5559ef157710 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.511+0000 7f2e089d2740 1 --2- 172.21.15.45:0/3284698576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef144b70 0x5559ef149a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.511+0000 7f2e089d2740 1 --2- 172.21.15.45:0/3284698576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef14ea60 0x5559ef150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.511+0000 7f2e089d2740 1 --2- 172.21.15.45:0/3284698576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef13fa00 0x5559ef1494f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.511+0000 7f2e089d2740 1 -- 172.21.15.45:0/3284698576 >> 172.21.15.45:0/3284698576 conn(0x5559ef13a680 msgr2=0x5559ef1397e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.511+0000 7f2ddeffd640 0 client.15459 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.511+0000 7f2e089d2740 1 -- 172.21.15.45:0/3284698576 shutdown_connections 2024-06-06T04:32:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.511+0000 7f2e089d2740 1 -- 172.21.15.45:0/3284698576 wait complete. 2024-06-06T04:32:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.511+0000 7f2e089d2740 20 client.15459 trim_cache size 0 max 16384 2024-06-06T04:32:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.541+0000 7f2e089d2740 1 Processor -- start 2024-06-06T04:32:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.541+0000 7f2e089d2740 1 -- start start 2024-06-06T04:32:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.541+0000 7f2e089d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef144b70 0x5559ef13dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.541+0000 7f2e089d2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef14ea60 0x5559ef148b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.541+0000 7f2e089d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef149080 0x5559ef149530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.541+0000 7f2e089d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001acdd0 con 0x5559ef144b70 2024-06-06T04:32:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.541+0000 7f2e089d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001aa6a0 con 0x5559ef149080 2024-06-06T04:32:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.541+0000 7f2e089d2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001a84f0 con 0x5559ef14ea60 2024-06-06T04:32:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.542+0000 7f2dff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef144b70 0x5559ef13dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.542+0000 7f2dff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef144b70 0x5559ef13dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57478/0 (socket says 172.21.15.45:57478) 2024-06-06T04:32:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.542+0000 7f2dff7fe640 1 -- 172.21.15.45:0/2692026074 learned_addr learned my addr 172.21.15.45:0/2692026074 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.542+0000 7f2dfeffd640 1 --2- 172.21.15.45:0/2692026074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef14ea60 0x5559ef148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.542+0000 7f2dfffff640 1 --2- 172.21.15.45:0/2692026074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef149080 0x5559ef149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.542+0000 7f2dfeffd640 1 -- 172.21.15.45:0/2692026074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef149080 msgr2=0x5559ef149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.542+0000 7f2dfeffd640 1 --2- 172.21.15.45:0/2692026074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef149080 0x5559ef149530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.542+0000 7f2dfeffd640 1 -- 172.21.15.45:0/2692026074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef144b70 msgr2=0x5559ef13dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.542+0000 7f2dfeffd640 1 --2- 172.21.15.45:0/2692026074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef144b70 0x5559ef13dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.542+0000 7f2dfeffd640 1 -- 172.21.15.45:0/2692026074 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ef157ca0 con 0x5559ef14ea60 2024-06-06T04:32:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.542+0000 7f2dfffff640 1 --2- 172.21.15.45:0/2692026074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef149080 0x5559ef149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.542+0000 7f2dff7fe640 1 --2- 172.21.15.45:0/2692026074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef144b70 0x5559ef13dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.542+0000 7f2dfeffd640 1 --2- 172.21.15.45:0/2692026074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef14ea60 0x5559ef148b40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x5559ef157ed0 tx=0x7f2df4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5859407635caa6fa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.542+0000 7f2de57fa640 1 -- 172.21.15.45:0/2692026074 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df402c070 con 0x5559ef14ea60 2024-06-06T04:32:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.543+0000 7f2de57fa640 1 -- 172.21.15.45:0/2692026074 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2df4027980 con 0x5559ef14ea60 2024-06-06T04:32:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.543+0000 7f2de57fa640 1 -- 172.21.15.45:0/2692026074 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df400e9c0 con 0x5559ef14ea60 2024-06-06T04:32:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.543+0000 7f2e089d2740 1 -- 172.21.15.45:0/2692026074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef14ea60 msgr2=0x5559ef148b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.543+0000 7f2e089d2740 1 --2- 172.21.15.45:0/2692026074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef14ea60 0x5559ef148b40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x5559ef157ed0 tx=0x7f2df4014000 comp rx=0 tx=0).stop 2024-06-06T04:32:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.544+0000 7f2e089d2740 1 -- 172.21.15.45:0/2692026074 shutdown_connections 2024-06-06T04:32:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.544+0000 7f2e089d2740 1 --2- 172.21.15.45:0/2692026074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef149080 0x5559ef149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.544+0000 7f2e089d2740 1 --2- 172.21.15.45:0/2692026074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef14ea60 0x5559ef148b40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.544+0000 7f2e089d2740 1 --2- 172.21.15.45:0/2692026074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef144b70 0x5559ef13dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.544+0000 7f2e089d2740 1 -- 172.21.15.45:0/2692026074 >> 172.21.15.45:0/2692026074 conn(0x5559ef13a680 msgr2=0x5559ef13d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.544+0000 7f2e089d2740 1 -- 172.21.15.45:0/2692026074 shutdown_connections 2024-06-06T04:32:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.544+0000 7f2e089d2740 1 -- 172.21.15.45:0/2692026074 wait complete. 2024-06-06T04:32:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.544+0000 7f2e089d2740 1 -- start start 2024-06-06T04:32:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.544+0000 7f2e089d2740 1 Processor -- start 2024-06-06T04:32:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2e089d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef144b70 0x5559ef1bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2e089d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef149080 0x5559ef1bd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2e089d2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef14ea60 0x5559ef167e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2dfeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef149080 0x5559ef1bd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2dfeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef149080 0x5559ef1bd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57488/0 (socket says 172.21.15.45:57488) 2024-06-06T04:32:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2dfeffd640 1 -- 172.21.15.45:0/1006243041 learned_addr learned my addr 172.21.15.45:0/1006243041 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2e089d2740 1 -- 172.21.15.45:0/1006243041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001a8b60 con 0x5559ef149080 2024-06-06T04:32:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2e089d2740 1 -- 172.21.15.45:0/1006243041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001a9fd0 con 0x5559ef144b70 2024-06-06T04:32:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2e089d2740 1 -- 172.21.15.45:0/1006243041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001a9230 con 0x5559ef14ea60 2024-06-06T04:32:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2dff7fe640 1 --2- 172.21.15.45:0/1006243041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef144b70 0x5559ef1bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2e089d2740 10 client.0 _ll_register_callbacks cb 0x7f2df0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2dfffff640 1 --2- 172.21.15.45:0/1006243041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef14ea60 0x5559ef167e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2dfeffd640 1 -- 172.21.15.45:0/1006243041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef14ea60 msgr2=0x5559ef167e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2dfeffd640 1 --2- 172.21.15.45:0/1006243041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef14ea60 0x5559ef167e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2dfeffd640 1 -- 172.21.15.45:0/1006243041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef144b70 msgr2=0x5559ef1bcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2dfeffd640 1 --2- 172.21.15.45:0/1006243041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef144b70 0x5559ef1bcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2dfeffd640 1 -- 172.21.15.45:0/1006243041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559ef1684c0 con 0x5559ef149080 2024-06-06T04:32:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.545+0000 7f2dff7fe640 1 --2- 172.21.15.45:0/1006243041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef144b70 0x5559ef1bcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.546+0000 7f2dfeffd640 1 --2- 172.21.15.45:0/1006243041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef149080 0x5559ef1bd480 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2df4000c00 tx=0x7f2df4027db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cacc3c6921f691dc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.546+0000 7f2ddeffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.546+0000 7f2ddeffd640 1 -- 172.21.15.45:0/1006243041 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df402c040 con 0x5559ef149080 2024-06-06T04:32:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.546+0000 7f2e089d2740 1 -- 172.21.15.45:0/1006243041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2df4027980 con 0x5559ef149080 2024-06-06T04:32:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.546+0000 7f2ddeffd640 1 -- 172.21.15.45:0/1006243041 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f2df40275d0 con 0x5559ef149080 2024-06-06T04:32:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.546+0000 7f2ddeffd640 1 -- 172.21.15.45:0/1006243041 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2df403e050 con 0x5559ef149080 2024-06-06T04:32:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.546+0000 7f2e089d2740 10 client.15465 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.546+0000 7f2e089d2740 1 -- 172.21.15.45:0/1006243041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5559ef169730 con 0x5559ef149080 2024-06-06T04:32:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.546+0000 7f2ddeffd640 1 -- 172.21.15.45:0/1006243041 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2df4027980 con 0x5559ef149080 2024-06-06T04:32:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.546+0000 7f2ddeffd640 1 -- 172.21.15.45:0/1006243041 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2df402f070 con 0x5559ef149080 2024-06-06T04:32:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.547+0000 7f2e089d2740 10 client.15465 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.547+0000 7f2e089d2740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.547+0000 7f2e089d2740 20 client.15465 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.547+0000 7f2e089d2740 1 --2- 172.21.15.45:0/1006243041 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5559ef1c8fd0 0x5559ef1cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.547+0000 7f2e089d2740 4 client.15465 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T04:32:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.547+0000 7f2e089d2740 1 -- 172.21.15.45:0/1006243041 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5559ef0afc20 con 0x5559ef1c8fd0 2024-06-06T04:32:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.547+0000 7f2dff7fe640 1 --2- 172.21.15.45:0/1006243041 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5559ef1c8fd0 0x5559ef1cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.547+0000 7f2ddeffd640 10 client.15465 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.547+0000 7f2dff7fe640 1 --2- 172.21.15.45:0/1006243041 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5559ef1c8fd0 0x5559ef1cb3f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d25d5016914b45 server_cookie=ec1b56cdbd560439 in_seq=0 out_seq=0 2024-06-06T04:32:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.551+0000 7f2ddeffd640 1 -- 172.21.15.45:0/1006243041 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5559ef0afc20 con 0x5559ef1c8fd0 2024-06-06T04:32:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.551+0000 7f2ddeffd640 10 client.15465 handle_command_reply: tid=0 2024-06-06T04:32:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.551+0000 7f2e089d2740 1 client.15465 shutdown 2024-06-06T04:32:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.551+0000 7f2e089d2740 1 -- 172.21.15.45:0/1006243041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef149080 msgr2=0x5559ef1bd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.551+0000 7f2e089d2740 1 --2- 172.21.15.45:0/1006243041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef149080 0x5559ef1bd480 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2df4000c00 tx=0x7f2df4027db0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.552+0000 7f2e089d2740 1 -- 172.21.15.45:0/1006243041 shutdown_connections 2024-06-06T04:32:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.552+0000 7f2e089d2740 1 --2- 172.21.15.45:0/1006243041 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5559ef1c8fd0 0x5559ef1cb3f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.552+0000 7f2ddeffd640 0 client.15465 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.552+0000 7f2e089d2740 1 --2- 172.21.15.45:0/1006243041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559ef14ea60 0x5559ef167e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.552+0000 7f2e089d2740 1 --2- 172.21.15.45:0/1006243041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5559ef149080 0x5559ef1bd480 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.552+0000 7f2e089d2740 1 --2- 172.21.15.45:0/1006243041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559ef144b70 0x5559ef1bcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.552+0000 7f2e089d2740 1 -- 172.21.15.45:0/1006243041 >> 172.21.15.45:0/1006243041 conn(0x5559ef13a680 msgr2=0x5559ef13d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.552+0000 7f2e089d2740 1 -- 172.21.15.45:0/1006243041 shutdown_connections 2024-06-06T04:32:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.552+0000 7f2e089d2740 1 -- 172.21.15.45:0/1006243041 wait complete. 2024-06-06T04:32:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.552+0000 7f2e089d2740 20 client.15465 trim_cache size 0 max 16384 2024-06-06T04:32:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-06-06T04:32:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-06-06T04:32:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-06-06T04:32:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.553+0000 7f2e05be7640 1 -- 172.21.15.45:0/2569504726 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2de80a2ec0 msgr2=0x7f2de80a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.553+0000 7f2e05be7640 1 --2- 172.21.15.45:0/2569504726 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2de80a2ec0 0x7f2de80a5380 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e00199100 tx=0x7f2df4005f90 comp rx=0 tx=0).stop 2024-06-06T04:32:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.553+0000 7f2e05be7640 1 -- 172.21.15.45:0/2569504726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e00103600 msgr2=0x7f2e0019f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.553+0000 7f2e05be7640 1 --2- 172.21.15.45:0/2569504726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e00103600 0x7f2e0019f1a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2df800eb10 tx=0x7f2df800c2d0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.553+0000 7f2e05be7640 1 -- 172.21.15.45:0/2569504726 shutdown_connections 2024-06-06T04:32:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.553+0000 7f2e05be7640 1 --2- 172.21.15.45:0/2569504726 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2de80a2ec0 0x7f2de80a5380 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.553+0000 7f2e05be7640 1 --2- 172.21.15.45:0/2569504726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e00103600 0x7f2e0019f1a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.553+0000 7f2e05be7640 1 --2- 172.21.15.45:0/2569504726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e00102c40 0x7f2e00198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.553+0000 7f2e05be7640 1 --2- 172.21.15.45:0/2569504726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e00101a50 0x7f2e00197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.553+0000 7f2e05be7640 1 -- 172.21.15.45:0/2569504726 >> 172.21.15.45:0/2569504726 conn(0x7f2e000fcfc0 msgr2=0x7f2e00106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.553+0000 7f2e05be7640 1 -- 172.21.15.45:0/2569504726 shutdown_connections 2024-06-06T04:32:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:23.553+0000 7f2e05be7640 1 -- 172.21.15.45:0/2569504726 wait complete. 2024-06-06T04:32:24.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:24 smithi045 ceph-mon[32299]: pgmap v459: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:24.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:24 smithi086 ceph-mon[36053]: pgmap v459: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:24.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:24 smithi154 ceph-mon[36428]: pgmap v459: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.963+0000 7fcc646d2700 1 -- 172.21.15.45:0/3922542876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5c101d50 msgr2=0x7fcc5c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.963+0000 7fcc646d2700 1 --2- 172.21.15.45:0/3922542876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5c101d50 0x7fcc5c10e240 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400b1a0 tx=0x7fcc5400b4b0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.963+0000 7fcc646d2700 1 -- 172.21.15.45:0/3922542876 shutdown_connections 2024-06-06T04:32:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.963+0000 7fcc646d2700 1 --2- 172.21.15.45:0/3922542876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5c101d50 0x7fcc5c10e240 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.963+0000 7fcc646d2700 1 --2- 172.21.15.45:0/3922542876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc5c107400 0x7fcc5c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.963+0000 7fcc646d2700 1 --2- 172.21.15.45:0/3922542876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc5c101390 0x7fcc5c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.963+0000 7fcc646d2700 1 -- 172.21.15.45:0/3922542876 >> 172.21.15.45:0/3922542876 conn(0x7fcc5c078340 msgr2=0x7fcc5c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:24.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.963+0000 7fcc646d2700 1 -- 172.21.15.45:0/3922542876 shutdown_connections 2024-06-06T04:32:24.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.963+0000 7fcc646d2700 1 -- 172.21.15.45:0/3922542876 wait complete. 2024-06-06T04:32:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.964+0000 7fcc646d2700 1 Processor -- start 2024-06-06T04:32:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.964+0000 7fcc646d2700 1 -- start start 2024-06-06T04:32:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.964+0000 7fcc646d2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5c101390 0x7fcc5c196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.964+0000 7fcc646d2700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc5c101d50 0x7fcc5c196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.964+0000 7fcc646d2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc5c107400 0x7fcc5c19b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.964+0000 7fcc646d2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c1104e0 con 0x7fcc5c101390 2024-06-06T04:32:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.964+0000 7fcc646d2700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c110360 con 0x7fcc5c101d50 2024-06-06T04:32:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.964+0000 7fcc646d2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c110660 con 0x7fcc5c107400 2024-06-06T04:32:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.965+0000 7fcc6246e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5c101390 0x7fcc5c196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.965+0000 7fcc62c6f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc5c107400 0x7fcc5c19b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.965+0000 7fcc62c6f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc5c107400 0x7fcc5c19b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:53700/0 (socket says 172.21.15.45:53700) 2024-06-06T04:32:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.965+0000 7fcc62c6f700 1 -- 172.21.15.45:0/809593394 learned_addr learned my addr 172.21.15.45:0/809593394 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.965+0000 7fcc61c6d700 1 --2- 172.21.15.45:0/809593394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc5c101d50 0x7fcc5c196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.965+0000 7fcc62c6f700 1 -- 172.21.15.45:0/809593394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc5c101d50 msgr2=0x7fcc5c196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.965+0000 7fcc62c6f700 1 --2- 172.21.15.45:0/809593394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc5c101d50 0x7fcc5c196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.965+0000 7fcc62c6f700 1 -- 172.21.15.45:0/809593394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5c101390 msgr2=0x7fcc5c196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.965+0000 7fcc62c6f700 1 --2- 172.21.15.45:0/809593394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5c101390 0x7fcc5c196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.965+0000 7fcc62c6f700 1 -- 172.21.15.45:0/809593394 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5c19b740 con 0x7fcc5c107400 2024-06-06T04:32:24.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.965+0000 7fcc6246e700 1 --2- 172.21.15.45:0/809593394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5c101390 0x7fcc5c196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:24.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.965+0000 7fcc62c6f700 1 --2- 172.21.15.45:0/809593394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc5c107400 0x7fcc5c19b000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400c090 tx=0x7fcc54008cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:24.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.966+0000 7fcc537fe700 1 -- 172.21.15.45:0/809593394 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54003a00 con 0x7fcc5c107400 2024-06-06T04:32:24.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.966+0000 7fcc537fe700 1 -- 172.21.15.45:0/809593394 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc54025070 con 0x7fcc5c107400 2024-06-06T04:32:24.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.966+0000 7fcc537fe700 1 -- 172.21.15.45:0/809593394 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54010730 con 0x7fcc5c107400 2024-06-06T04:32:24.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.966+0000 7fcc646d2700 1 -- 172.21.15.45:0/809593394 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5c19b9d0 con 0x7fcc5c107400 2024-06-06T04:32:24.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.966+0000 7fcc646d2700 1 -- 172.21.15.45:0/809593394 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc5c19bda0 con 0x7fcc5c107400 2024-06-06T04:32:24.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.969+0000 7fcc537fe700 1 -- 172.21.15.45:0/809593394 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fcc54010ae0 con 0x7fcc5c107400 2024-06-06T04:32:24.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.970+0000 7fcc537fe700 1 --2- 172.21.15.45:0/809593394 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fcc480a2d30 0x7fcc480a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:24.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.970+0000 7fcc537fe700 1 -- 172.21.15.45:0/809593394 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fcc540d4730 con 0x7fcc5c107400 2024-06-06T04:32:24.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.970+0000 7fcc6246e700 1 --2- 172.21.15.45:0/809593394 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fcc480a2d30 0x7fcc480a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:24.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.971+0000 7fcc6ef55b80 1 Processor -- start 2024-06-06T04:32:24.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.971+0000 7fcc6ef55b80 1 -- start start 2024-06-06T04:32:24.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.971+0000 7fcc6246e700 1 --2- 172.21.15.45:0/809593394 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fcc480a2d30 0x7fcc480a51f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcc58005950 tx=0x7fcc5800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:24.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.972+0000 7fcc6ef55b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c0770 0x56258c3c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.972+0000 7fcc6ef55b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c1090 0x56258c3baa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.972+0000 7fcc6ef55b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3bafb0 0x56258c3bb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.972+0000 7fcc6246e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c0770 0x56258c3c0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.972+0000 7fcc6246e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c0770 0x56258c3c0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57502/0 (socket says 172.21.15.45:57502) 2024-06-06T04:32:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.972+0000 7fcc6246e700 1 -- 172.21.15.45:0/910337393 learned_addr learned my addr 172.21.15.45:0/910337393 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.972+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/910337393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56258c2a4f60 con 0x56258c3c0770 2024-06-06T04:32:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.972+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/910337393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56258c303850 con 0x56258c3c1090 2024-06-06T04:32:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.972+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/910337393 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56258c26b460 con 0x56258c3bafb0 2024-06-06T04:32:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.972+0000 7fcc62c6f700 1 --2- 172.21.15.45:0/910337393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3bafb0 0x56258c3bb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.973+0000 7fcc6246e700 1 -- 172.21.15.45:0/910337393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3bafb0 msgr2=0x56258c3bb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.973+0000 7fcc6246e700 1 --2- 172.21.15.45:0/910337393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3bafb0 0x56258c3bb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.973+0000 7fcc61c6d700 1 --2- 172.21.15.45:0/910337393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c1090 0x56258c3baa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.973+0000 7fcc6246e700 1 -- 172.21.15.45:0/910337393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c1090 msgr2=0x56258c3baa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.973+0000 7fcc6246e700 1 --2- 172.21.15.45:0/910337393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c1090 0x56258c3baa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.973+0000 7fcc6246e700 1 -- 172.21.15.45:0/910337393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258c2528d0 con 0x56258c3c0770 2024-06-06T04:32:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.973+0000 7fcc61c6d700 1 --2- 172.21.15.45:0/910337393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c1090 0x56258c3baa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.973+0000 7fcc6246e700 1 --2- 172.21.15.45:0/910337393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c0770 0x56258c3c0b50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcc580174f0 tx=0x7fcc58013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8443c970ebaf1f3b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.973+0000 7fcc50ff9700 1 -- 172.21.15.45:0/910337393 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5801d040 con 0x56258c3c0770 2024-06-06T04:32:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.973+0000 7fcc50ff9700 1 -- 172.21.15.45:0/910337393 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc58026070 con 0x56258c3c0770 2024-06-06T04:32:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.973+0000 7fcc50ff9700 1 -- 172.21.15.45:0/910337393 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc58015600 con 0x56258c3c0770 2024-06-06T04:32:24.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.974+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/910337393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c0770 msgr2=0x56258c3c0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:24.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.974+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/910337393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c0770 0x56258c3c0b50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcc580174f0 tx=0x7fcc58013000 comp rx=0 tx=0).stop 2024-06-06T04:32:24.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.974+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/910337393 shutdown_connections 2024-06-06T04:32:24.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.975+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/910337393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3bafb0 0x56258c3bb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.975+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/910337393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c0770 0x56258c3c0b50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.975+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/910337393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c1090 0x56258c3baa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.975+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/910337393 >> 172.21.15.45:0/910337393 conn(0x56258c2df080 msgr2=0x56258c3ad220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.975+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/910337393 shutdown_connections 2024-06-06T04:32:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.975+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/910337393 wait complete. 2024-06-06T04:32:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.975+0000 7fcc6ef55b80 1 -- start start 2024-06-06T04:32:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.976+0000 7fcc6ef55b80 1 Processor -- start 2024-06-06T04:32:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.976+0000 7fcc6ef55b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3bafb0 0x56258c3c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.976+0000 7fcc6ef55b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c0770 0x56258c3c4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.976+0000 7fcc6ef55b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c1090 0x56258c3c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.976+0000 7fcc62c6f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c1090 0x56258c3c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.976+0000 7fcc6246e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3bafb0 0x56258c3c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.977+0000 7fcc62c6f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c1090 0x56258c3c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57508/0 (socket says 172.21.15.45:57508) 2024-06-06T04:32:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.977+0000 7fcc6246e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3bafb0 0x56258c3c6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:53720/0 (socket says 172.21.15.45:53720) 2024-06-06T04:32:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.977+0000 7fcc61c6d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c0770 0x56258c3c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.977+0000 7fcc61c6d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c0770 0x56258c3c4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:42148/0 (socket says 172.21.15.45:42148) 2024-06-06T04:32:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.977+0000 7fcc62c6f700 1 -- 172.21.15.45:0/3409985773 learned_addr learned my addr 172.21.15.45:0/3409985773 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.977+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/3409985773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56258c2a3fc0 con 0x56258c3c1090 2024-06-06T04:32:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.977+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/3409985773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56258c2ff810 con 0x56258c3c0770 2024-06-06T04:32:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.977+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/3409985773 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56258c3c7a70 con 0x56258c3bafb0 2024-06-06T04:32:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.977+0000 7fcc62c6f700 1 -- 172.21.15.45:0/3409985773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3bafb0 msgr2=0x56258c3c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.977+0000 7fcc62c6f700 1 --2- 172.21.15.45:0/3409985773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3bafb0 0x56258c3c6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.977+0000 7fcc62c6f700 1 -- 172.21.15.45:0/3409985773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c0770 msgr2=0x56258c3c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.977+0000 7fcc62c6f700 1 --2- 172.21.15.45:0/3409985773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c0770 0x56258c3c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.977+0000 7fcc62c6f700 1 -- 172.21.15.45:0/3409985773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258c2c9750 con 0x56258c3c1090 2024-06-06T04:32:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.978+0000 7fcc6ef55b80 10 client.0 _ll_register_callbacks cb 0x56258c2995b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.978+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/3409985773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc58026070 con 0x56258c3c1090 2024-06-06T04:32:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.978+0000 7fcc62c6f700 1 --2- 172.21.15.45:0/3409985773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c1090 0x56258c3c4640 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcc5409d010 tx=0x7fcc54009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea225af1c27ba09b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.978+0000 7fcc45ffb700 10 client.15477 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.978+0000 7fcc45ffb700 1 -- 172.21.15.45:0/3409985773 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5400f070 con 0x56258c3c1090 2024-06-06T04:32:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.978+0000 7fcc45ffb700 1 -- 172.21.15.45:0/3409985773 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fcc54010260 con 0x56258c3c1090 2024-06-06T04:32:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.978+0000 7fcc45ffb700 1 -- 172.21.15.45:0/3409985773 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc540a12b0 con 0x56258c3c1090 2024-06-06T04:32:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.978+0000 7fcc6ef55b80 10 client.15477 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.978+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/3409985773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=27}) -- 0x56258c3d1f30 con 0x56258c3c1090 2024-06-06T04:32:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.979+0000 7fcc45ffb700 1 -- 172.21.15.45:0/3409985773 <== mon.0 v2:172.21.15.45:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcc540a0e10 con 0x56258c3c1090 2024-06-06T04:32:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.979+0000 7fcc6ef55b80 10 client.15477 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.979+0000 7fcc6ef55b80 10 client.15477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.979+0000 7fcc6ef55b80 20 client.15477 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.979+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/3409985773 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x56258c3d4450 0x56258c3d68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.979+0000 7fcc6ef55b80 4 client.15477 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:32:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.979+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/3409985773 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56258c310d50 con 0x56258c3d4450 2024-06-06T04:32:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.979+0000 7fcc6246e700 1 --2- 172.21.15.45:0/3409985773 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x56258c3d4450 0x56258c3d68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:24.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.980+0000 7fcc45ffb700 10 client.15477 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:24.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.980+0000 7fcc6246e700 1 --2- 172.21.15.45:0/3409985773 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x56258c3d4450 0x56258c3d68f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eac55ee6dcb39bd server_cookie=a4beabe0b1144692 in_seq=0 out_seq=0 2024-06-06T04:32:24.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.980+0000 7fcc45ffb700 1 -- 172.21.15.45:0/3409985773 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56258c310d50 con 0x56258c3d4450 2024-06-06T04:32:24.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.980+0000 7fcc45ffb700 10 client.15477 handle_command_reply: tid=0 2024-06-06T04:32:24.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.980+0000 7fcc6ef55b80 1 client.15477 shutdown 2024-06-06T04:32:24.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.981+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/3409985773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c1090 msgr2=0x56258c3c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:24.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.981+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/3409985773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c1090 0x56258c3c4640 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcc5409d010 tx=0x7fcc54009cd0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.981+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/3409985773 shutdown_connections 2024-06-06T04:32:24.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.981+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/3409985773 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x56258c3d4450 0x56258c3d68f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.981+0000 7fcc45ffb700 0 client.15477 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:24.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.981+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/3409985773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c1090 0x56258c3c4640 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.981+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/3409985773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3bafb0 0x56258c3c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.981+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/3409985773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c0770 0x56258c3c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:24.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.981+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/3409985773 >> 172.21.15.45:0/3409985773 conn(0x56258c2df080 msgr2=0x56258c3ac700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:24.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.981+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/3409985773 shutdown_connections 2024-06-06T04:32:24.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.981+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/3409985773 wait complete. 2024-06-06T04:32:24.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.981+0000 7fcc6ef55b80 20 client.15477 trim_cache size 0 max 16384 2024-06-06T04:32:24.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.997+0000 7fcc6ef55b80 1 Processor -- start 2024-06-06T04:32:24.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.997+0000 7fcc6ef55b80 1 -- start start 2024-06-06T04:32:24.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.997+0000 7fcc6ef55b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3bafb0 0x56258c3c4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:24.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.997+0000 7fcc6ef55b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c0770 0x56258c3c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:24.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.997+0000 7fcc6ef55b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c1090 0x56258c3ace80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:24.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.997+0000 7fcc6ef55b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c1ae560 con 0x56258c3c1090 2024-06-06T04:32:24.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.997+0000 7fcc6ef55b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c1abdf0 con 0x56258c3c0770 2024-06-06T04:32:24.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.997+0000 7fcc6ef55b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c1a9c40 con 0x56258c3bafb0 2024-06-06T04:32:24.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.997+0000 7fcc62c6f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c1090 0x56258c3ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:24.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.997+0000 7fcc62c6f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c1090 0x56258c3ace80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57516/0 (socket says 172.21.15.45:57516) 2024-06-06T04:32:25.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.997+0000 7fcc62c6f700 1 -- 172.21.15.45:0/3577113230 learned_addr learned my addr 172.21.15.45:0/3577113230 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:25.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.997+0000 7fcc6246e700 1 --2- 172.21.15.45:0/3577113230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3bafb0 0x56258c3c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:25.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.998+0000 7fcc61c6d700 1 --2- 172.21.15.45:0/3577113230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c0770 0x56258c3c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:25.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.998+0000 7fcc62c6f700 1 -- 172.21.15.45:0/3577113230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3bafb0 msgr2=0x56258c3c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:25.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.998+0000 7fcc62c6f700 1 --2- 172.21.15.45:0/3577113230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3bafb0 0x56258c3c4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:25.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.998+0000 7fcc62c6f700 1 -- 172.21.15.45:0/3577113230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c0770 msgr2=0x56258c3c1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:25.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.998+0000 7fcc62c6f700 1 --2- 172.21.15.45:0/3577113230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c0770 0x56258c3c1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:25.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.998+0000 7fcc62c6f700 1 -- 172.21.15.45:0/3577113230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258c3c2670 con 0x56258c3c1090 2024-06-06T04:32:25.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.998+0000 7fcc62c6f700 1 --2- 172.21.15.45:0/3577113230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c1090 0x56258c3ace80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcc540a0b40 tx=0x7fcc540c0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d52cbf99859a03 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:25.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.998+0000 7fcc47fff700 1 -- 172.21.15.45:0/3577113230 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54084070 con 0x56258c3c1090 2024-06-06T04:32:25.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.998+0000 7fcc47fff700 1 -- 172.21.15.45:0/3577113230 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc58026070 con 0x56258c3c1090 2024-06-06T04:32:25.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.998+0000 7fcc47fff700 1 -- 172.21.15.45:0/3577113230 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5400f050 con 0x56258c3c1090 2024-06-06T04:32:25.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.999+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/3577113230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c1090 msgr2=0x56258c3ace80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:25.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.999+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/3577113230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c1090 0x56258c3ace80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcc540a0b40 tx=0x7fcc540c0000 comp rx=0 tx=0).stop 2024-06-06T04:32:25.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.999+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/3577113230 shutdown_connections 2024-06-06T04:32:25.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.999+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/3577113230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3c1090 0x56258c3ace80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:25.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.999+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/3577113230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3bafb0 0x56258c3c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:25.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.999+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/3577113230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c0770 0x56258c3c1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:25.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:24.999+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/3577113230 >> 172.21.15.45:0/3577113230 conn(0x56258c2df080 msgr2=0x56258c3c1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:25.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.000+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/3577113230 shutdown_connections 2024-06-06T04:32:25.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.000+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/3577113230 wait complete. 2024-06-06T04:32:25.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.000+0000 7fcc6ef55b80 1 -- start start 2024-06-06T04:32:25.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.000+0000 7fcc6ef55b80 1 Processor -- start 2024-06-06T04:32:25.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.000+0000 7fcc6ef55b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3bafb0 0x56258c3cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:25.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc6ef55b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3c0770 0x56258c3dba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:25.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc6ef55b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c1090 0x56258c3dbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:25.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc6ef55b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c1aa2b0 con 0x56258c3bafb0 2024-06-06T04:32:25.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc6246e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3bafb0 0x56258c3cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:25.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc6ef55b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c1ab720 con 0x56258c3c1090 2024-06-06T04:32:25.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc6ef55b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c1aa980 con 0x56258c3c0770 2024-06-06T04:32:25.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc6246e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3bafb0 0x56258c3cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57526/0 (socket says 172.21.15.45:57526) 2024-06-06T04:32:25.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc6246e700 1 -- 172.21.15.45:0/2943357098 learned_addr learned my addr 172.21.15.45:0/2943357098 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:25.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc61c6d700 1 --2- 172.21.15.45:0/2943357098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3c0770 0x56258c3dba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:25.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc6ef55b80 10 client.0 _ll_register_callbacks cb 0x56258c2995b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:25.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc6246e700 1 -- 172.21.15.45:0/2943357098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3c0770 msgr2=0x56258c3dba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:25.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc62c6f700 1 --2- 172.21.15.45:0/2943357098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c1090 0x56258c3dbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:25.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc6246e700 1 --2- 172.21.15.45:0/2943357098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3c0770 0x56258c3dba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:25.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc6246e700 1 -- 172.21.15.45:0/2943357098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c1090 msgr2=0x56258c3dbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:25.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc6246e700 1 --2- 172.21.15.45:0/2943357098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c1090 0x56258c3dbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:25.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc6246e700 1 -- 172.21.15.45:0/2943357098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258c3dc620 con 0x56258c3bafb0 2024-06-06T04:32:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.001+0000 7fcc62c6f700 1 --2- 172.21.15.45:0/2943357098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c1090 0x56258c3dbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.002+0000 7fcc6246e700 1 --2- 172.21.15.45:0/2943357098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3bafb0 0x56258c3cd390 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcc58000c00 tx=0x7fcc5800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7308544b038c7ceb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.002+0000 7fcc45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.002+0000 7fcc45ffb700 1 -- 172.21.15.45:0/2943357098 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc58005630 con 0x56258c3bafb0 2024-06-06T04:32:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.002+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/2943357098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc58026070 con 0x56258c3bafb0 2024-06-06T04:32:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.002+0000 7fcc45ffb700 1 -- 172.21.15.45:0/2943357098 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fcc580046f0 con 0x56258c3bafb0 2024-06-06T04:32:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.002+0000 7fcc45ffb700 1 -- 172.21.15.45:0/2943357098 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc58044050 con 0x56258c3bafb0 2024-06-06T04:32:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.002+0000 7fcc6ef55b80 10 client.15483 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.002+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/2943357098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56258c43f060 con 0x56258c3bafb0 2024-06-06T04:32:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.003+0000 7fcc45ffb700 1 -- 172.21.15.45:0/2943357098 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc58026070 con 0x56258c3bafb0 2024-06-06T04:32:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.003+0000 7fcc45ffb700 1 -- 172.21.15.45:0/2943357098 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcc58020030 con 0x56258c3bafb0 2024-06-06T04:32:25.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.003+0000 7fcc6ef55b80 10 client.15483 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.003+0000 7fcc6ef55b80 10 client.15483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.003+0000 7fcc6ef55b80 20 client.15483 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.003+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/2943357098 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x56258c3d4450 0x56258c441440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.003+0000 7fcc6ef55b80 4 client.15483 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T04:32:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.003+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/2943357098 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56258c30b2d0 con 0x56258c3d4450 2024-06-06T04:32:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.003+0000 7fcc61c6d700 1 --2- 172.21.15.45:0/2943357098 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x56258c3d4450 0x56258c441440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.004+0000 7fcc61c6d700 1 --2- 172.21.15.45:0/2943357098 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x56258c3d4450 0x56258c441440 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0997b32b66bca25 server_cookie=4df022ab69291e9e in_seq=0 out_seq=0 2024-06-06T04:32:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.004+0000 7fcc45ffb700 10 client.15483 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.004+0000 7fcc45ffb700 1 -- 172.21.15.45:0/2943357098 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56258c30b2d0 con 0x56258c3d4450 2024-06-06T04:32:25.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.004+0000 7fcc45ffb700 10 client.15483 handle_command_reply: tid=0 2024-06-06T04:32:25.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.004+0000 7fcc6ef55b80 1 client.15483 shutdown 2024-06-06T04:32:25.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.004+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/2943357098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3bafb0 msgr2=0x56258c3cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:25.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.004+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/2943357098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3bafb0 0x56258c3cd390 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcc58000c00 tx=0x7fcc5800be70 comp rx=0 tx=0).stop 2024-06-06T04:32:25.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.004+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/2943357098 shutdown_connections 2024-06-06T04:32:25.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.004+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/2943357098 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x56258c3d4450 0x56258c441440 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:25.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.004+0000 7fcc45ffb700 0 client.15483 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:25.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.004+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/2943357098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56258c3c1090 0x56258c3dbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:25.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.004+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/2943357098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56258c3bafb0 0x56258c3cd390 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:25.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.004+0000 7fcc6ef55b80 1 --2- 172.21.15.45:0/2943357098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56258c3c0770 0x56258c3dba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:25.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.004+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/2943357098 >> 172.21.15.45:0/2943357098 conn(0x56258c2df080 msgr2=0x56258c3c29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:25.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.004+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/2943357098 shutdown_connections 2024-06-06T04:32:25.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.005+0000 7fcc6ef55b80 1 -- 172.21.15.45:0/2943357098 wait complete. 2024-06-06T04:32:25.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.005+0000 7fcc6ef55b80 20 client.15483 trim_cache size 0 max 16384 2024-06-06T04:32:25.011 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-06-06T04:32:25.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.005+0000 7fcc646d2700 1 -- 172.21.15.45:0/809593394 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fcc480a2d30 msgr2=0x7fcc480a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:25.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.005+0000 7fcc646d2700 1 --2- 172.21.15.45:0/809593394 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fcc480a2d30 0x7fcc480a51f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcc58005950 tx=0x7fcc5800d040 comp rx=0 tx=0).stop 2024-06-06T04:32:25.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.005+0000 7fcc646d2700 1 -- 172.21.15.45:0/809593394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc5c107400 msgr2=0x7fcc5c19b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:25.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.005+0000 7fcc646d2700 1 --2- 172.21.15.45:0/809593394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc5c107400 0x7fcc5c19b000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400c090 tx=0x7fcc54008cb0 comp rx=0 tx=0).stop 2024-06-06T04:32:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.006+0000 7fcc646d2700 1 -- 172.21.15.45:0/809593394 shutdown_connections 2024-06-06T04:32:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.006+0000 7fcc646d2700 1 --2- 172.21.15.45:0/809593394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc5c107400 0x7fcc5c19b000 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.006+0000 7fcc646d2700 1 --2- 172.21.15.45:0/809593394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5c101390 0x7fcc5c196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.006+0000 7fcc646d2700 1 --2- 172.21.15.45:0/809593394 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fcc480a2d30 0x7fcc480a51f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.006+0000 7fcc646d2700 1 --2- 172.21.15.45:0/809593394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc5c101d50 0x7fcc5c196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.006+0000 7fcc646d2700 1 -- 172.21.15.45:0/809593394 >> 172.21.15.45:0/809593394 conn(0x7fcc5c078340 msgr2=0x7fcc5c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.006+0000 7fcc646d2700 1 -- 172.21.15.45:0/809593394 shutdown_connections 2024-06-06T04:32:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:25.006+0000 7fcc646d2700 1 -- 172.21.15.45:0/809593394 wait complete. 2024-06-06T04:32:26.057 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:26 smithi045 ceph-mon[32299]: pgmap v460: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:32:26.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:26 smithi086 ceph-mon[36053]: pgmap v460: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:32:26.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:26 smithi154 ceph-mon[36428]: pgmap v460: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:32:26.767 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-06T04:32:26.768 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-06T04:32:27.076 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:32:28.362 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:28 smithi086 ceph-mon[36053]: pgmap v461: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T04:32:28.404 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:28 smithi045 ceph-mon[32299]: pgmap v461: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T04:32:28.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:28 smithi154 ceph-mon[36428]: pgmap v461: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-06T04:32:29.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.021+0000 7f163796f700 1 -- 172.21.15.45:0/3950274231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16301013c0 msgr2=0x7f1630101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:29.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.021+0000 7f163796f700 1 --2- 172.21.15.45:0/3950274231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16301013c0 0x7f1630101840 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1620009990 tx=0x7f1620009ca0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.021+0000 7f163796f700 1 -- 172.21.15.45:0/3950274231 shutdown_connections 2024-06-06T04:32:29.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.021+0000 7f163796f700 1 --2- 172.21.15.45:0/3950274231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1630101d80 0x7f163010e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.021+0000 7f163796f700 1 --2- 172.21.15.45:0/3950274231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630107430 0x7f1630107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.021+0000 7f163796f700 1 --2- 172.21.15.45:0/3950274231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16301013c0 0x7f1630101840 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.021+0000 7f163796f700 1 -- 172.21.15.45:0/3950274231 >> 172.21.15.45:0/3950274231 conn(0x7f16300783d0 msgr2=0x7f16300787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:29.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.022+0000 7f163796f700 1 -- 172.21.15.45:0/3950274231 shutdown_connections 2024-06-06T04:32:29.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.022+0000 7f163796f700 1 -- 172.21.15.45:0/3950274231 wait complete. 2024-06-06T04:32:29.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.022+0000 7f163796f700 1 Processor -- start 2024-06-06T04:32:29.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.022+0000 7f163796f700 1 -- start start 2024-06-06T04:32:29.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.023+0000 7f163796f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16301013c0 0x7f1630073c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.023+0000 7f163796f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630101d80 0x7f163006cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.023+0000 7f163796f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1630107430 0x7f163006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.023+0000 7f163796f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1630112670 con 0x7f1630107430 2024-06-06T04:32:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.023+0000 7f163796f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16301124f0 con 0x7f1630101d80 2024-06-06T04:32:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.023+0000 7f163796f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f16301127f0 con 0x7f16301013c0 2024-06-06T04:32:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.023+0000 7f1635f0c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1630107430 0x7f163006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.023+0000 7f1635f0c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1630107430 0x7f163006d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48760/0 (socket says 172.21.15.45:48760) 2024-06-06T04:32:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.023+0000 7f1635f0c700 1 -- 172.21.15.45:0/2257157411 learned_addr learned my addr 172.21.15.45:0/2257157411 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.023+0000 7f163570b700 1 --2- 172.21.15.45:0/2257157411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16301013c0 0x7f1630073c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.023+0000 7f1634f0a700 1 --2- 172.21.15.45:0/2257157411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630101d80 0x7f163006cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.024+0000 7f1634f0a700 1 -- 172.21.15.45:0/2257157411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16301013c0 msgr2=0x7f1630073c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.024+0000 7f1634f0a700 1 --2- 172.21.15.45:0/2257157411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16301013c0 0x7f1630073c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.024+0000 7f1634f0a700 1 -- 172.21.15.45:0/2257157411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1630107430 msgr2=0x7f163006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.024+0000 7f1634f0a700 1 --2- 172.21.15.45:0/2257157411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1630107430 0x7f163006d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.024+0000 7f1634f0a700 1 -- 172.21.15.45:0/2257157411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163006d8e0 con 0x7f1630101d80 2024-06-06T04:32:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.024+0000 7f1635f0c700 1 --2- 172.21.15.45:0/2257157411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1630107430 0x7f163006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:29.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.024+0000 7f1634f0a700 1 --2- 172.21.15.45:0/2257157411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630101d80 0x7f163006cd60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1620006010 tx=0x7f1620005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.024+0000 7f16267fc700 1 -- 172.21.15.45:0/2257157411 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162000b610 con 0x7f1630101d80 2024-06-06T04:32:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.024+0000 7f16267fc700 1 -- 172.21.15.45:0/2257157411 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1620020380 con 0x7f1630101d80 2024-06-06T04:32:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.024+0000 7f163796f700 1 -- 172.21.15.45:0/2257157411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163006db70 con 0x7f1630101d80 2024-06-06T04:32:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.024+0000 7f16267fc700 1 -- 172.21.15.45:0/2257157411 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620017720 con 0x7f1630101d80 2024-06-06T04:32:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.024+0000 7f163796f700 1 -- 172.21.15.45:0/2257157411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1630071980 con 0x7f1630101d80 2024-06-06T04:32:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.028+0000 7f16267fc700 1 -- 172.21.15.45:0/2257157411 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f1620039020 con 0x7f1630101d80 2024-06-06T04:32:29.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.029+0000 7f16267fc700 1 --2- 172.21.15.45:0/2257157411 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f161c0a2d30 0x7f161c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.029+0000 7f16267fc700 1 -- 172.21.15.45:0/2257157411 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f16200e18f0 con 0x7f1630101d80 2024-06-06T04:32:29.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.029+0000 7f163570b700 1 --2- 172.21.15.45:0/2257157411 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f161c0a2d30 0x7f161c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:29.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.030+0000 7f16421f2b80 1 Processor -- start 2024-06-06T04:32:29.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.030+0000 7f16421f2b80 1 -- start start 2024-06-06T04:32:29.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.030+0000 7f163570b700 1 --2- 172.21.15.45:0/2257157411 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f161c0a2d30 0x7f161c0a51f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f162c010ac0 tx=0x7f162c012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:29.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.030+0000 7f16421f2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e386c0 0x556462e38aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.030+0000 7f16421f2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e38fe0 0x556462e32a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.030+0000 7f16421f2b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e32fa0 0x556462e333c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.030+0000 7f1634f0a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e38fe0 0x556462e32a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:29.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.030+0000 7f1634f0a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e38fe0 0x556462e32a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48768/0 (socket says 172.21.15.45:48768) 2024-06-06T04:32:29.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.030+0000 7f1634f0a700 1 -- 172.21.15.45:0/2382075454 learned_addr learned my addr 172.21.15.45:0/2382075454 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:29.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.030+0000 7f16421f2b80 1 -- 172.21.15.45:0/2382075454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556462d84890 con 0x556462e38fe0 2024-06-06T04:32:29.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.030+0000 7f16421f2b80 1 -- 172.21.15.45:0/2382075454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556462d2c7a0 con 0x556462e386c0 2024-06-06T04:32:29.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.030+0000 7f16421f2b80 1 -- 172.21.15.45:0/2382075454 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556462d80850 con 0x556462e32fa0 2024-06-06T04:32:29.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.031+0000 7f1635f0c700 1 --2- 172.21.15.45:0/2382075454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e32fa0 0x556462e333c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.031+0000 7f1634f0a700 1 -- 172.21.15.45:0/2382075454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e32fa0 msgr2=0x556462e333c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.031+0000 7f1634f0a700 1 --2- 172.21.15.45:0/2382075454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e32fa0 0x556462e333c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.031+0000 7f1634f0a700 1 -- 172.21.15.45:0/2382075454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e386c0 msgr2=0x556462e38aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.031+0000 7f1634f0a700 1 --2- 172.21.15.45:0/2382075454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e386c0 0x556462e38aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.031+0000 7f1634f0a700 1 -- 172.21.15.45:0/2382075454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556462d22fd0 con 0x556462e38fe0 2024-06-06T04:32:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.031+0000 7f1634f0a700 1 --2- 172.21.15.45:0/2382075454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e38fe0 0x556462e32a60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f16200b6420 tx=0x7f16200ad8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f168d8926f255b9c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.031+0000 7f161b7fe700 1 -- 172.21.15.45:0/2382075454 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620039310 con 0x556462e38fe0 2024-06-06T04:32:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.031+0000 7f161b7fe700 1 -- 172.21.15.45:0/2382075454 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16200ae470 con 0x556462e38fe0 2024-06-06T04:32:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.032+0000 7f161b7fe700 1 -- 172.21.15.45:0/2382075454 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16200b64c0 con 0x556462e38fe0 2024-06-06T04:32:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.032+0000 7f16421f2b80 1 -- 172.21.15.45:0/2382075454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e38fe0 msgr2=0x556462e32a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.032+0000 7f16421f2b80 1 --2- 172.21.15.45:0/2382075454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e38fe0 0x556462e32a60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f16200b6420 tx=0x7f16200ad8b0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.033+0000 7f16421f2b80 1 -- 172.21.15.45:0/2382075454 shutdown_connections 2024-06-06T04:32:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.033+0000 7f16421f2b80 1 --2- 172.21.15.45:0/2382075454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e32fa0 0x556462e333c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.033+0000 7f16421f2b80 1 --2- 172.21.15.45:0/2382075454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e386c0 0x556462e38aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.033+0000 7f16421f2b80 1 --2- 172.21.15.45:0/2382075454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e38fe0 0x556462e32a60 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.033+0000 7f16421f2b80 1 -- 172.21.15.45:0/2382075454 >> 172.21.15.45:0/2382075454 conn(0x556462d5dd40 msgr2=0x556462e2d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.033+0000 7f16421f2b80 1 -- 172.21.15.45:0/2382075454 shutdown_connections 2024-06-06T04:32:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.033+0000 7f16421f2b80 1 -- 172.21.15.45:0/2382075454 wait complete. 2024-06-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.033+0000 7f16421f2b80 1 -- start start 2024-06-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.033+0000 7f16421f2b80 1 Processor -- start 2024-06-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.034+0000 7f16421f2b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e32fa0 0x556462e3eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.034+0000 7f16421f2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e386c0 0x556462e3c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.034+0000 7f1634f0a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e386c0 0x556462e3c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.034+0000 7f1634f0a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e386c0 0x556462e3c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48772/0 (socket says 172.21.15.45:48772) 2024-06-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.035+0000 7f1634f0a700 1 -- 172.21.15.45:0/717421765 learned_addr learned my addr 172.21.15.45:0/717421765 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.035+0000 7f16421f2b80 1 --2- 172.21.15.45:0/717421765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e38fe0 0x556462e3c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.035+0000 7f163570b700 1 --2- 172.21.15.45:0/717421765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e32fa0 0x556462e3eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.035+0000 7f16421f2b80 1 -- 172.21.15.45:0/717421765 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556462e3f770 con 0x556462e386c0 2024-06-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.035+0000 7f16421f2b80 1 -- 172.21.15.45:0/717421765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556462d24c70 con 0x556462e38fe0 2024-06-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.035+0000 7f16421f2b80 1 -- 172.21.15.45:0/717421765 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556462e3f8f0 con 0x556462e32fa0 2024-06-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.035+0000 7f1635f0c700 1 --2- 172.21.15.45:0/717421765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e38fe0 0x556462e3c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.035+0000 7f1635f0c700 1 -- 172.21.15.45:0/717421765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e32fa0 msgr2=0x556462e3eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.035+0000 7f1635f0c700 1 --2- 172.21.15.45:0/717421765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e32fa0 0x556462e3eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.035+0000 7f1635f0c700 1 -- 172.21.15.45:0/717421765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e386c0 msgr2=0x556462e3c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.035+0000 7f1635f0c700 1 --2- 172.21.15.45:0/717421765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e386c0 0x556462e3c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.035+0000 7f1635f0c700 1 -- 172.21.15.45:0/717421765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556462cd3950 con 0x556462e38fe0 2024-06-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.035+0000 7f1634f0a700 1 --2- 172.21.15.45:0/717421765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e386c0 0x556462e3c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.036+0000 7f16421f2b80 10 client.0 _ll_register_callbacks cb 0x556462d1e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.036+0000 7f1635f0c700 1 --2- 172.21.15.45:0/717421765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e38fe0 0x556462e3c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1628007f40 tx=0x7f1628012520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd9e9fc47b9e8bfe server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.036+0000 7f1618ff9700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-06-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.036+0000 7f16421f2b80 1 -- 172.21.15.45:0/717421765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16200ae470 con 0x556462e38fe0 2024-06-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.036+0000 7f1618ff9700 1 -- 172.21.15.45:0/717421765 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162801a070 con 0x556462e38fe0 2024-06-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.036+0000 7f1618ff9700 1 -- 172.21.15.45:0/717421765 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f1628016420 con 0x556462e38fe0 2024-06-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.036+0000 7f1618ff9700 1 -- 172.21.15.45:0/717421765 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1628026050 con 0x556462e38fe0 2024-06-06T04:32:29.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.036+0000 7f16421f2b80 10 client.25145 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:29.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.036+0000 7f16421f2b80 1 -- 172.21.15.45:0/717421765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=27}) -- 0x556462e49e60 con 0x556462e38fe0 2024-06-06T04:32:29.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.037+0000 7f1618ff9700 1 -- 172.21.15.45:0/717421765 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1628016640 con 0x556462e38fe0 2024-06-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.037+0000 7f16421f2b80 10 client.25145 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.037+0000 7f16421f2b80 10 client.25145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.037+0000 7f16421f2b80 20 client.25145 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.037+0000 7f16421f2b80 1 --2- 172.21.15.45:0/717421765 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x556462e4c880 0x556462e4ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.037+0000 7f16421f2b80 4 client.25145 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.037+0000 7f16421f2b80 1 -- 172.21.15.45:0/717421765 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556462d91d90 con 0x556462e4c880 2024-06-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.037+0000 7f163570b700 1 --2- 172.21.15.45:0/717421765 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x556462e4c880 0x556462e4ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.038+0000 7f163570b700 1 --2- 172.21.15.45:0/717421765 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x556462e4c880 0x556462e4ed40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb64a05d4964d17 server_cookie=bcc4679e91cf1fc8 in_seq=0 out_seq=0 2024-06-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.038+0000 7f1618ff9700 10 client.25145 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.038+0000 7f1618ff9700 1 -- 172.21.15.45:0/717421765 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556462d91d90 con 0x556462e4c880 2024-06-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.038+0000 7f1618ff9700 10 client.25145 handle_command_reply: tid=0 2024-06-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.039+0000 7f16421f2b80 1 client.25145 shutdown 2024-06-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.039+0000 7f16421f2b80 1 -- 172.21.15.45:0/717421765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e38fe0 msgr2=0x556462e3c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.039+0000 7f16421f2b80 1 --2- 172.21.15.45:0/717421765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e38fe0 0x556462e3c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1628007f40 tx=0x7f1628012520 comp rx=0 tx=0).stop 2024-06-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.039+0000 7f16421f2b80 1 -- 172.21.15.45:0/717421765 shutdown_connections 2024-06-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.039+0000 7f16421f2b80 1 --2- 172.21.15.45:0/717421765 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x556462e4c880 0x556462e4ed40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.039+0000 7f16421f2b80 1 --2- 172.21.15.45:0/717421765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e38fe0 0x556462e3c630 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.039+0000 7f16421f2b80 1 --2- 172.21.15.45:0/717421765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e32fa0 0x556462e3eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.039+0000 7f16421f2b80 1 --2- 172.21.15.45:0/717421765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e386c0 0x556462e3c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.039+0000 7f16421f2b80 1 -- 172.21.15.45:0/717421765 >> 172.21.15.45:0/717421765 conn(0x556462d5dd40 msgr2=0x556462e2d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.039+0000 7f1618ff9700 0 client.25145 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.039+0000 7f16421f2b80 1 -- 172.21.15.45:0/717421765 shutdown_connections 2024-06-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.039+0000 7f16421f2b80 1 -- 172.21.15.45:0/717421765 wait complete. 2024-06-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.040+0000 7f16421f2b80 20 client.25145 trim_cache size 0 max 16384 2024-06-06T04:32:29.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.082+0000 7f16421f2b80 1 Processor -- start 2024-06-06T04:32:29.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.082+0000 7f16421f2b80 1 -- start start 2024-06-06T04:32:29.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.082+0000 7f16421f2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e32fa0 0x556462e32a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.083+0000 7f16421f2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e386c0 0x556462e27710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.083+0000 7f16421f2b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e38fe0 0x556462e54650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.083+0000 7f16421f2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16301a6050 con 0x556462e386c0 2024-06-06T04:32:29.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.083+0000 7f16421f2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16301a38e0 con 0x556462e32fa0 2024-06-06T04:32:29.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.083+0000 7f16421f2b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f16301a1730 con 0x556462e38fe0 2024-06-06T04:32:29.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.083+0000 7f1634f0a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e386c0 0x556462e27710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:29.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.083+0000 7f1634f0a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e386c0 0x556462e27710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48786/0 (socket says 172.21.15.45:48786) 2024-06-06T04:32:29.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.083+0000 7f1634f0a700 1 -- 172.21.15.45:0/245946593 learned_addr learned my addr 172.21.15.45:0/245946593 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:29.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.083+0000 7f1635f0c700 1 --2- 172.21.15.45:0/245946593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e38fe0 0x556462e54650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:29.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.083+0000 7f1634f0a700 1 -- 172.21.15.45:0/245946593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e38fe0 msgr2=0x556462e54650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:29.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.083+0000 7f1634f0a700 1 --2- 172.21.15.45:0/245946593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e38fe0 0x556462e54650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.083+0000 7f1634f0a700 1 -- 172.21.15.45:0/245946593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e32fa0 msgr2=0x556462e32a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T04:32:29.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.083+0000 7f1634f0a700 1 --2- 172.21.15.45:0/245946593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e32fa0 0x556462e32a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.083+0000 7f1634f0a700 1 -- 172.21.15.45:0/245946593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556462e4f3c0 con 0x556462e386c0 2024-06-06T04:32:29.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.084+0000 7f1634f0a700 1 --2- 172.21.15.45:0/245946593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e386c0 0x556462e27710 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f16200ae620 tx=0x7f1620009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287f9afa0131671b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:29.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.084+0000 7f161affd700 1 -- 172.21.15.45:0/245946593 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16200cb070 con 0x556462e386c0 2024-06-06T04:32:29.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.084+0000 7f161affd700 1 -- 172.21.15.45:0/245946593 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16200add70 con 0x556462e386c0 2024-06-06T04:32:29.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.084+0000 7f161affd700 1 -- 172.21.15.45:0/245946593 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16200b6720 con 0x556462e386c0 2024-06-06T04:32:29.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.085+0000 7f16421f2b80 1 -- 172.21.15.45:0/245946593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e386c0 msgr2=0x556462e27710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:29.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.085+0000 7f16421f2b80 1 --2- 172.21.15.45:0/245946593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e386c0 0x556462e27710 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f16200ae620 tx=0x7f1620009310 comp rx=0 tx=0).stop 2024-06-06T04:32:29.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.085+0000 7f16421f2b80 1 -- 172.21.15.45:0/245946593 shutdown_connections 2024-06-06T04:32:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.085+0000 7f16421f2b80 1 --2- 172.21.15.45:0/245946593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e38fe0 0x556462e54650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.085+0000 7f16421f2b80 1 --2- 172.21.15.45:0/245946593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e32fa0 0x556462e32a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.085+0000 7f16421f2b80 1 --2- 172.21.15.45:0/245946593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e386c0 0x556462e27710 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.085+0000 7f16421f2b80 1 -- 172.21.15.45:0/245946593 >> 172.21.15.45:0/245946593 conn(0x556462d5dd40 msgr2=0x556462e39af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.085+0000 7f16421f2b80 1 -- 172.21.15.45:0/245946593 shutdown_connections 2024-06-06T04:32:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.085+0000 7f16421f2b80 1 -- 172.21.15.45:0/245946593 wait complete. 2024-06-06T04:32:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.086+0000 7f16421f2b80 1 -- start start 2024-06-06T04:32:29.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.086+0000 7f16421f2b80 1 Processor -- start 2024-06-06T04:32:29.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.087+0000 7f16421f2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e32fa0 0x556462e44b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.087+0000 7f16421f2b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e386c0 0x556462e45080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.087+0000 7f16421f2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e38fe0 0x556462e41b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.087+0000 7f163570b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e32fa0 0x556462e44b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:29.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.087+0000 7f1635f0c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e38fe0 0x556462e41b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:29.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.087+0000 7f163570b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e32fa0 0x556462e44b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:59324/0 (socket says 172.21.15.45:59324) 2024-06-06T04:32:29.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.087+0000 7f1635f0c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e38fe0 0x556462e41b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48796/0 (socket says 172.21.15.45:48796) 2024-06-06T04:32:29.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.087+0000 7f163570b700 1 -- 172.21.15.45:0/3916294022 learned_addr learned my addr 172.21.15.45:0/3916294022 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:29.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.087+0000 7f1634f0a700 1 --2- 172.21.15.45:0/3916294022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e386c0 0x556462e45080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:29.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.087+0000 7f16421f2b80 1 -- 172.21.15.45:0/3916294022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16301a1da0 con 0x556462e38fe0 2024-06-06T04:32:29.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.088+0000 7f16421f2b80 1 -- 172.21.15.45:0/3916294022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16301a3210 con 0x556462e32fa0 2024-06-06T04:32:29.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.088+0000 7f16421f2b80 1 -- 172.21.15.45:0/3916294022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f16301a2470 con 0x556462e386c0 2024-06-06T04:32:29.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.088+0000 7f163570b700 1 -- 172.21.15.45:0/3916294022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e386c0 msgr2=0x556462e45080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:29.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.088+0000 7f163570b700 1 --2- 172.21.15.45:0/3916294022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e386c0 0x556462e45080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.088+0000 7f163570b700 1 -- 172.21.15.45:0/3916294022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e38fe0 msgr2=0x556462e41b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:29.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.088+0000 7f163570b700 1 --2- 172.21.15.45:0/3916294022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e38fe0 0x556462e41b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.088+0000 7f16421f2b80 10 client.0 _ll_register_callbacks cb 0x556462d1e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:29.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.088+0000 7f163570b700 1 -- 172.21.15.45:0/3916294022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556462e42050 con 0x556462e32fa0 2024-06-06T04:32:29.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.088+0000 7f1634f0a700 1 --2- 172.21.15.45:0/3916294022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e386c0 0x556462e45080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:29.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.088+0000 7f1635f0c700 1 --2- 172.21.15.45:0/3916294022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e38fe0 0x556462e41b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:29.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.088+0000 7f16421f2b80 1 -- 172.21.15.45:0/3916294022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16200add70 con 0x556462e32fa0 2024-06-06T04:32:29.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.088+0000 7f163570b700 1 --2- 172.21.15.45:0/3916294022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e32fa0 0x556462e44b40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f162c010520 tx=0x7f162c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa500a7fb085033e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:29.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.088+0000 7f1618ff9700 10 client.25148 ms_handle_connect on v2:172.21.15.86:3300/0 2024-06-06T04:32:29.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.089+0000 7f1618ff9700 1 -- 172.21.15.45:0/3916294022 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c009b60 con 0x556462e32fa0 2024-06-06T04:32:29.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.089+0000 7f1618ff9700 1 -- 172.21.15.45:0/3916294022 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f162c003890 con 0x556462e32fa0 2024-06-06T04:32:29.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.089+0000 7f1618ff9700 1 -- 172.21.15.45:0/3916294022 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f162c003a90 con 0x556462e32fa0 2024-06-06T04:32:29.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.089+0000 7f16421f2b80 10 client.25148 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:29.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.089+0000 7f16421f2b80 1 -- 172.21.15.45:0/3916294022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=27}) -- 0x556462eb7d40 con 0x556462e32fa0 2024-06-06T04:32:29.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.089+0000 7f1618ff9700 1 -- 172.21.15.45:0/3916294022 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f162c030060 con 0x556462e32fa0 2024-06-06T04:32:29.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.089+0000 7f16421f2b80 10 client.25148 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:29.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.089+0000 7f16421f2b80 10 client.25148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:29.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.089+0000 7f16421f2b80 20 client.25148 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:29.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.089+0000 7f16421f2b80 1 --2- 172.21.15.45:0/3916294022 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x556462eb82e0 0x556462eba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:29.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.089+0000 7f16421f2b80 4 client.25148 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-06T04:32:29.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.089+0000 7f16421f2b80 1 -- 172.21.15.45:0/3916294022 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556462d8c310 con 0x556462eb82e0 2024-06-06T04:32:29.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.090+0000 7f1634f0a700 1 --2- 172.21.15.45:0/3916294022 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x556462eb82e0 0x556462eba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:29.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.090+0000 7f1618ff9700 10 client.25148 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:29.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.090+0000 7f1634f0a700 1 --2- 172.21.15.45:0/3916294022 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x556462eb82e0 0x556462eba6d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb13cb99b4eb759a server_cookie=bc448a94febd1c60 in_seq=0 out_seq=0 2024-06-06T04:32:29.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.091+0000 7f1618ff9700 1 -- 172.21.15.45:0/3916294022 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556462d8c310 con 0x556462eb82e0 2024-06-06T04:32:29.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.091+0000 7f1618ff9700 10 client.25148 handle_command_reply: tid=0 2024-06-06T04:32:29.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.091+0000 7f16421f2b80 1 client.25148 shutdown 2024-06-06T04:32:29.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.092+0000 7f16421f2b80 1 -- 172.21.15.45:0/3916294022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e32fa0 msgr2=0x556462e44b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:29.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.092+0000 7f16421f2b80 1 --2- 172.21.15.45:0/3916294022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e32fa0 0x556462e44b40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f162c010520 tx=0x7f162c004750 comp rx=0 tx=0).stop 2024-06-06T04:32:29.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.092+0000 7f16421f2b80 1 -- 172.21.15.45:0/3916294022 shutdown_connections 2024-06-06T04:32:29.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.092+0000 7f16421f2b80 1 --2- 172.21.15.45:0/3916294022 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x556462eb82e0 0x556462eba6d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.092+0000 7f16421f2b80 1 --2- 172.21.15.45:0/3916294022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556462e38fe0 0x556462e41b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.092+0000 7f16421f2b80 1 --2- 172.21.15.45:0/3916294022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556462e32fa0 0x556462e44b40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.094 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:32:29.094 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-06-06T04:32:29.094 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "10664e64-c8dc-4056-95e6-cc3b59fd0697", 2024-06-06T04:32:29.094 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-06-06T04:32:29.094 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:32:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.092+0000 7f16421f2b80 1 --2- 172.21.15.45:0/3916294022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556462e386c0 0x556462e45080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.092+0000 7f16421f2b80 1 -- 172.21.15.45:0/3916294022 >> 172.21.15.45:0/3916294022 conn(0x556462d5dd40 msgr2=0x556462e27710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.092+0000 7f1618ff9700 0 client.25148 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.092+0000 7f16421f2b80 1 -- 172.21.15.45:0/3916294022 shutdown_connections 2024-06-06T04:32:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.092+0000 7f16421f2b80 1 -- 172.21.15.45:0/3916294022 wait complete. 2024-06-06T04:32:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.092+0000 7f16421f2b80 20 client.25148 trim_cache size 0 max 16384 2024-06-06T04:32:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.093+0000 7f163796f700 1 -- 172.21.15.45:0/2257157411 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f161c0a2d30 msgr2=0x7f161c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.093+0000 7f163796f700 1 --2- 172.21.15.45:0/2257157411 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f161c0a2d30 0x7f161c0a51f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f162c010ac0 tx=0x7f162c012040 comp rx=0 tx=0).stop 2024-06-06T04:32:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.093+0000 7f163796f700 1 -- 172.21.15.45:0/2257157411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630101d80 msgr2=0x7f163006cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.093+0000 7f163796f700 1 --2- 172.21.15.45:0/2257157411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630101d80 0x7f163006cd60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1620006010 tx=0x7f1620005ef0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.093+0000 7f163796f700 1 -- 172.21.15.45:0/2257157411 shutdown_connections 2024-06-06T04:32:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.093+0000 7f163796f700 1 --2- 172.21.15.45:0/2257157411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1630107430 0x7f163006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.093+0000 7f163796f700 1 --2- 172.21.15.45:0/2257157411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f16301013c0 0x7f1630073c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.093+0000 7f163796f700 1 --2- 172.21.15.45:0/2257157411 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f161c0a2d30 0x7f161c0a51f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.093+0000 7f163796f700 1 --2- 172.21.15.45:0/2257157411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1630101d80 0x7f163006cd60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.093+0000 7f163796f700 1 -- 172.21.15.45:0/2257157411 >> 172.21.15.45:0/2257157411 conn(0x7f16300783d0 msgr2=0x7f1630100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.093+0000 7f163796f700 1 -- 172.21.15.45:0/2257157411 shutdown_connections 2024-06-06T04:32:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:29.093+0000 7f163796f700 1 -- 172.21.15.45:0/2257157411 wait complete. 2024-06-06T04:32:29.991 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T04:32:30.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:30 smithi045 ceph-mon[32299]: pgmap v462: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:30.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:30 smithi045 ceph-mon[32299]: scrub queued for path: / 2024-06-06T04:32:30.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:30 smithi045 ceph-mon[32299]: scrub summary: idle+waiting paths [/] 2024-06-06T04:32:30.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:30 smithi045 ceph-mon[32299]: scrub summary: active paths [/] 2024-06-06T04:32:30.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:30 smithi045 ceph-mon[32299]: scrub summary: idle+waiting paths [/] 2024-06-06T04:32:30.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:30 smithi045 ceph-mon[32299]: scrub summary: idle 2024-06-06T04:32:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:32:30.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:30 smithi086 ceph-mon[36053]: pgmap v462: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:30.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:30 smithi086 ceph-mon[36053]: scrub queued for path: / 2024-06-06T04:32:30.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:30 smithi086 ceph-mon[36053]: scrub summary: idle+waiting paths [/] 2024-06-06T04:32:30.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:30 smithi086 ceph-mon[36053]: scrub summary: active paths [/] 2024-06-06T04:32:30.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:30 smithi086 ceph-mon[36053]: scrub summary: idle+waiting paths [/] 2024-06-06T04:32:30.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:30 smithi086 ceph-mon[36053]: scrub summary: idle 2024-06-06T04:32:30.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:30 smithi154 ceph-mon[36428]: pgmap v462: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:30.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:30 smithi154 ceph-mon[36428]: scrub queued for path: / 2024-06-06T04:32:30.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:30 smithi154 ceph-mon[36428]: scrub summary: idle+waiting paths [/] 2024-06-06T04:32:30.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:30 smithi154 ceph-mon[36428]: scrub summary: active paths [/] 2024-06-06T04:32:30.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:30 smithi154 ceph-mon[36428]: scrub summary: idle+waiting paths [/] 2024-06-06T04:32:30.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:30 smithi154 ceph-mon[36428]: scrub summary: idle 2024-06-06T04:32:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.130+0000 7f9e614b8700 1 -- 172.21.15.45:0/1577509149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e5c102540 msgr2=0x7f9e5c1029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.130+0000 7f9e614b8700 1 --2- 172.21.15.45:0/1577509149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e5c102540 0x7f9e5c1029a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009a10 tx=0x7f9e50009d20 comp rx=0 tx=0).stop 2024-06-06T04:32:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.130+0000 7f9e614b8700 1 -- 172.21.15.45:0/1577509149 shutdown_connections 2024-06-06T04:32:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.130+0000 7f9e614b8700 1 --2- 172.21.15.45:0/1577509149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e5c102ee0 0x7f9e5c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.130+0000 7f9e614b8700 1 --2- 172.21.15.45:0/1577509149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e5c107450 0x7f9e5c107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.130+0000 7f9e614b8700 1 --2- 172.21.15.45:0/1577509149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e5c102540 0x7f9e5c1029a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.130+0000 7f9e614b8700 1 -- 172.21.15.45:0/1577509149 >> 172.21.15.45:0/1577509149 conn(0x7f9e5c0795f0 msgr2=0x7f9e5c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.130+0000 7f9e614b8700 1 -- 172.21.15.45:0/1577509149 shutdown_connections 2024-06-06T04:32:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.130+0000 7f9e614b8700 1 -- 172.21.15.45:0/1577509149 wait complete. 2024-06-06T04:32:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.131+0000 7f9e614b8700 1 Processor -- start 2024-06-06T04:32:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.132+0000 7f9e614b8700 1 -- start start 2024-06-06T04:32:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.132+0000 7f9e614b8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e5c102540 0x7f9e5c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.132+0000 7f9e614b8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e5c102ee0 0x7f9e5c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.132+0000 7f9e614b8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e5c107450 0x7f9e5c1a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.132+0000 7f9e614b8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c110580 con 0x7f9e5c107450 2024-06-06T04:32:32.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.132+0000 7f9e614b8700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c110400 con 0x7f9e5c102ee0 2024-06-06T04:32:32.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.132+0000 7f9e614b8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c110700 con 0x7f9e5c102540 2024-06-06T04:32:32.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.132+0000 7f9e5b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e5c107450 0x7f9e5c1a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.132+0000 7f9e5a7fc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e5c102ee0 0x7f9e5c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.132+0000 7f9e5affd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e5c102540 0x7f9e5c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.132+0000 7f9e5a7fc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e5c102ee0 0x7f9e5c19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:59330/0 (socket says 172.21.15.45:59330) 2024-06-06T04:32:32.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.132+0000 7f9e5affd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e5c102540 0x7f9e5c19f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:60070/0 (socket says 172.21.15.45:60070) 2024-06-06T04:32:32.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.132+0000 7f9e5a7fc700 1 -- 172.21.15.45:0/3138929281 learned_addr learned my addr 172.21.15.45:0/3138929281 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.133+0000 7f9e5a7fc700 1 -- 172.21.15.45:0/3138929281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e5c102540 msgr2=0x7f9e5c19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.133+0000 7f9e5a7fc700 1 --2- 172.21.15.45:0/3138929281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e5c102540 0x7f9e5c19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.133+0000 7f9e5a7fc700 1 -- 172.21.15.45:0/3138929281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e5c107450 msgr2=0x7f9e5c1a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.133+0000 7f9e5a7fc700 1 --2- 172.21.15.45:0/3138929281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e5c107450 0x7f9e5c1a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.133+0000 7f9e5a7fc700 1 -- 172.21.15.45:0/3138929281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5c1a4180 con 0x7f9e5c102ee0 2024-06-06T04:32:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.133+0000 7f9e5a7fc700 1 --2- 172.21.15.45:0/3138929281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e5c102ee0 0x7f9e5c19f700 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9e5001a040 tx=0x7f9e5000b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.133+0000 7f9e43fff700 1 -- 172.21.15.45:0/3138929281 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50010630 con 0x7f9e5c102ee0 2024-06-06T04:32:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.133+0000 7f9e43fff700 1 -- 172.21.15.45:0/3138929281 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e50025070 con 0x7f9e5c102ee0 2024-06-06T04:32:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.133+0000 7f9e43fff700 1 -- 172.21.15.45:0/3138929281 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50020660 con 0x7f9e5c102ee0 2024-06-06T04:32:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.133+0000 7f9e614b8700 1 -- 172.21.15.45:0/3138929281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5c1a4410 con 0x7f9e5c102ee0 2024-06-06T04:32:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.133+0000 7f9e614b8700 1 -- 172.21.15.45:0/3138929281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5c1a47e0 con 0x7f9e5c102ee0 2024-06-06T04:32:32.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.136+0000 7f9e43fff700 1 -- 172.21.15.45:0/3138929281 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f9e5001f070 con 0x7f9e5c102ee0 2024-06-06T04:32:32.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.137+0000 7f9e6bd3bb80 1 Processor -- start 2024-06-06T04:32:32.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.137+0000 7f9e6bd3bb80 1 -- start start 2024-06-06T04:32:32.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.137+0000 7f9e6bd3bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4acf60 0x5597bf4ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e6bd3bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4ad880 0x5597bf4a7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e6bd3bb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4a77a0 0x5597bf4a7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e6bd3bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5597bf399f60 con 0x5597bf4acf60 2024-06-06T04:32:32.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e6bd3bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5597bf3f8850 con 0x5597bf4ad880 2024-06-06T04:32:32.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e6bd3bb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5597bf360460 con 0x5597bf4a77a0 2024-06-06T04:32:32.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e5affd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4ad880 0x5597bf4a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e5affd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4ad880 0x5597bf4a7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:59334/0 (socket says 172.21.15.45:59334) 2024-06-06T04:32:32.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e5affd700 1 -- 172.21.15.45:0/2923969939 learned_addr learned my addr 172.21.15.45:0/2923969939 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:32.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e5b7fe700 1 --2- 172.21.15.45:0/2923969939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4acf60 0x5597bf4ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e5a7fc700 1 --2- 172.21.15.45:0/2923969939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4a77a0 0x5597bf4a7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e5affd700 1 -- 172.21.15.45:0/2923969939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4a77a0 msgr2=0x5597bf4a7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e5affd700 1 --2- 172.21.15.45:0/2923969939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4a77a0 0x5597bf4a7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e5affd700 1 -- 172.21.15.45:0/2923969939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4acf60 msgr2=0x5597bf4ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e43fff700 1 --2- 172.21.15.45:0/3138929281 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9e440a2d30 0x7f9e440a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e5affd700 1 --2- 172.21.15.45:0/2923969939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4acf60 0x5597bf4ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e5affd700 1 -- 172.21.15.45:0/2923969939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597bf3478d0 con 0x5597bf4ad880 2024-06-06T04:32:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.138+0000 7f9e5a7fc700 1 --2- 172.21.15.45:0/3138929281 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9e440a2d30 0x7f9e440a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.139+0000 7f9e5b7fe700 1 --2- 172.21.15.45:0/2923969939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4acf60 0x5597bf4ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.139+0000 7f9e5affd700 1 --2- 172.21.15.45:0/2923969939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4ad880 0x5597bf4a7260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00bcc0 tx=0x7f9e4c0103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=730ae3de53c27811 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.139+0000 7f9e417fa700 1 -- 172.21.15.45:0/2923969939 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c010c20 con 0x5597bf4ad880 2024-06-06T04:32:32.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.139+0000 7f9e43fff700 1 -- 172.21.15.45:0/3138929281 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f9e500d0990 con 0x7f9e5c102ee0 2024-06-06T04:32:32.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.139+0000 7f9e5a7fc700 1 --2- 172.21.15.45:0/3138929281 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9e440a2d30 0x7f9e440a51f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9e500d40d0 tx=0x7f9e500d4040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:32.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.139+0000 7f9e417fa700 1 -- 172.21.15.45:0/2923969939 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e4c0044e0 con 0x5597bf4ad880 2024-06-06T04:32:32.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.139+0000 7f9e417fa700 1 -- 172.21.15.45:0/2923969939 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c011440 con 0x5597bf4ad880 2024-06-06T04:32:32.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.140+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/2923969939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4ad880 msgr2=0x5597bf4a7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:32.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.140+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/2923969939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4ad880 0x5597bf4a7260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00bcc0 tx=0x7f9e4c0103d0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.140+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/2923969939 shutdown_connections 2024-06-06T04:32:32.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.140+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/2923969939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4a77a0 0x5597bf4a7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.140+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/2923969939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4acf60 0x5597bf4ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.141+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/2923969939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4ad880 0x5597bf4a7260 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.141+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/2923969939 >> 172.21.15.45:0/2923969939 conn(0x5597bf3d4080 msgr2=0x5597bf4a2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:32.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.141+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/2923969939 shutdown_connections 2024-06-06T04:32:32.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.141+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/2923969939 wait complete. 2024-06-06T04:32:32.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.141+0000 7f9e6bd3bb80 1 -- start start 2024-06-06T04:32:32.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.142+0000 7f9e6bd3bb80 1 Processor -- start 2024-06-06T04:32:32.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.142+0000 7f9e6bd3bb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4a77a0 0x5597bf4b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.142+0000 7f9e6bd3bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4acf60 0x5597bf4b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.142+0000 7f9e6bd3bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4ad880 0x5597bf4b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.143+0000 7f9e6bd3bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5597bf398fc0 con 0x5597bf4ad880 2024-06-06T04:32:32.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.143+0000 7f9e6bd3bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5597bf3f4810 con 0x5597bf4acf60 2024-06-06T04:32:32.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.143+0000 7f9e6bd3bb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5597bf4b4260 con 0x5597bf4a77a0 2024-06-06T04:32:32.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.143+0000 7f9e5affd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4acf60 0x5597bf4b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.143+0000 7f9e5b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4a77a0 0x5597bf4b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.143+0000 7f9e5affd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4acf60 0x5597bf4b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:59342/0 (socket says 172.21.15.45:59342) 2024-06-06T04:32:32.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.143+0000 7f9e5affd700 1 -- 172.21.15.45:0/1721947041 learned_addr learned my addr 172.21.15.45:0/1721947041 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:32.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.143+0000 7f9e5b7fe700 1 --2- 172.21.15.45:0/1721947041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4ad880 0x5597bf4b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.143+0000 7f9e6bd3bb80 10 client.0 _ll_register_callbacks cb 0x5597bf38e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:32.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.143+0000 7f9e5b7fe700 1 -- 172.21.15.45:0/1721947041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4acf60 msgr2=0x5597bf4b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:32.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.143+0000 7f9e5b7fe700 1 --2- 172.21.15.45:0/1721947041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4acf60 0x5597bf4b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.143+0000 7f9e5b7fe700 1 -- 172.21.15.45:0/1721947041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4ad880 msgr2=0x5597bf4b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:32.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.143+0000 7f9e5b7fe700 1 --2- 172.21.15.45:0/1721947041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4ad880 0x5597bf4b0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.144+0000 7f9e5b7fe700 1 -- 172.21.15.45:0/1721947041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597bf3be750 con 0x5597bf4a77a0 2024-06-06T04:32:32.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.144+0000 7f9e5b7fe700 1 --2- 172.21.15.45:0/1721947041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4a77a0 0x5597bf4b35c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009550 tx=0x7f9e5400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caff2321bcbdace5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:32.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.144+0000 7f9e3eff5700 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-06-06T04:32:32.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.144+0000 7f9e3eff5700 1 -- 172.21.15.45:0/1721947041 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54004db0 con 0x5597bf4a77a0 2024-06-06T04:32:32.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.144+0000 7f9e3eff5700 1 -- 172.21.15.45:0/1721947041 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f9e5401b450 con 0x5597bf4a77a0 2024-06-06T04:32:32.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.144+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/1721947041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e4c0044e0 con 0x5597bf4a77a0 2024-06-06T04:32:32.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.144+0000 7f9e3eff5700 1 -- 172.21.15.45:0/1721947041 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9e5401b650 con 0x5597bf4a77a0 2024-06-06T04:32:32.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.145+0000 7f9e6bd3bb80 10 client.25141 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:32.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.145+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/1721947041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5597bf4be790 con 0x5597bf4a77a0 2024-06-06T04:32:32.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.145+0000 7f9e3eff5700 1 -- 172.21.15.45:0/1721947041 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e4c0044e0 con 0x5597bf4a77a0 2024-06-06T04:32:32.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.145+0000 7f9e3eff5700 1 -- 172.21.15.45:0/1721947041 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9e540276d0 con 0x5597bf4a77a0 2024-06-06T04:32:32.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.145+0000 7f9e6bd3bb80 10 client.25141 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:32.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.145+0000 7f9e6bd3bb80 10 client.25141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:32.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.145+0000 7f9e6bd3bb80 20 client.25141 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:32.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.145+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/1721947041 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5597bf4c0dc0 0x5597bf4c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.145+0000 7f9e6bd3bb80 4 client.25141 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:32:32.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.145+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/1721947041 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597bf405d50 con 0x5597bf4c0dc0 2024-06-06T04:32:32.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.146+0000 7f9e5affd700 1 --2- 172.21.15.45:0/1721947041 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5597bf4c0dc0 0x5597bf4c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.146+0000 7f9e3eff5700 10 client.25141 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:32.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.146+0000 7f9e5affd700 1 --2- 172.21.15.45:0/1721947041 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5597bf4c0dc0 0x5597bf4c31b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73062cc2e8b27799 server_cookie=24d8d7dd1d2525fa in_seq=0 out_seq=0 2024-06-06T04:32:32.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.147+0000 7f9e3eff5700 1 -- 172.21.15.45:0/1721947041 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5597bf405d50 con 0x5597bf4c0dc0 2024-06-06T04:32:32.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.147+0000 7f9e3eff5700 10 client.25141 handle_command_reply: tid=0 2024-06-06T04:32:32.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.147+0000 7f9e6bd3bb80 1 client.25141 shutdown 2024-06-06T04:32:32.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.147+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/1721947041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4a77a0 msgr2=0x5597bf4b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:32.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.147+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/1721947041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4a77a0 0x5597bf4b35c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009550 tx=0x7f9e5400b040 comp rx=0 tx=0).stop 2024-06-06T04:32:32.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.147+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/1721947041 shutdown_connections 2024-06-06T04:32:32.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.147+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/1721947041 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5597bf4c0dc0 0x5597bf4c31b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.147+0000 7f9e3eff5700 0 client.25141 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:32.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.147+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/1721947041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4ad880 0x5597bf4b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.147+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/1721947041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4a77a0 0x5597bf4b35c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.147+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/1721947041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4acf60 0x5597bf4b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.147+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/1721947041 >> 172.21.15.45:0/1721947041 conn(0x5597bf3d4080 msgr2=0x5597bf4a1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:32.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.148+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/1721947041 shutdown_connections 2024-06-06T04:32:32.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.148+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/1721947041 wait complete. 2024-06-06T04:32:32.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.148+0000 7f9e6bd3bb80 20 client.25141 trim_cache size 0 max 16384 2024-06-06T04:32:32.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.185+0000 7f9e6bd3bb80 1 Processor -- start 2024-06-06T04:32:32.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.185+0000 7f9e6bd3bb80 1 -- start start 2024-06-06T04:32:32.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.185+0000 7f9e6bd3bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4a77a0 0x5597bf4a7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.185+0000 7f9e6bd3bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4acf60 0x5597bf4b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.185+0000 7f9e6bd3bb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4ad880 0x5597bf4b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.185+0000 7f9e6bd3bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c1b7120 con 0x5597bf4acf60 2024-06-06T04:32:32.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.185+0000 7f9e6bd3bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c1b49b0 con 0x5597bf4a77a0 2024-06-06T04:32:32.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.185+0000 7f9e6bd3bb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c1b2800 con 0x5597bf4ad880 2024-06-06T04:32:32.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.185+0000 7f9e5affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4acf60 0x5597bf4b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.185+0000 7f9e5affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4acf60 0x5597bf4b0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48832/0 (socket says 172.21.15.45:48832) 2024-06-06T04:32:32.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.185+0000 7f9e5affd700 1 -- 172.21.15.45:0/4165178718 learned_addr learned my addr 172.21.15.45:0/4165178718 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.186+0000 7f9e5b7fe700 1 --2- 172.21.15.45:0/4165178718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4a77a0 0x5597bf4a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.186+0000 7f9e5affd700 1 -- 172.21.15.45:0/4165178718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4ad880 msgr2=0x5597bf4b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.186+0000 7f9e5affd700 1 --2- 172.21.15.45:0/4165178718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4ad880 0x5597bf4b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.186+0000 7f9e5affd700 1 -- 172.21.15.45:0/4165178718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4a77a0 msgr2=0x5597bf4a7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.186+0000 7f9e5affd700 1 --2- 172.21.15.45:0/4165178718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4a77a0 0x5597bf4a7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.186+0000 7f9e5affd700 1 -- 172.21.15.45:0/4165178718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597bf4ad680 con 0x5597bf4acf60 2024-06-06T04:32:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.186+0000 7f9e5affd700 1 --2- 172.21.15.45:0/4165178718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4acf60 0x5597bf4b0e30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x5597bf4b1e90 tx=0x7f9e4c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec3abf8859188c8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.186+0000 7f9e5b7fe700 1 --2- 172.21.15.45:0/4165178718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4a77a0 0x5597bf4a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.186+0000 7f9e40ff9700 1 -- 172.21.15.45:0/4165178718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c010ca0 con 0x5597bf4acf60 2024-06-06T04:32:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.186+0000 7f9e40ff9700 1 -- 172.21.15.45:0/4165178718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e4c010e40 con 0x5597bf4acf60 2024-06-06T04:32:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.186+0000 7f9e40ff9700 1 -- 172.21.15.45:0/4165178718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c004d10 con 0x5597bf4acf60 2024-06-06T04:32:32.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.187+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/4165178718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4acf60 msgr2=0x5597bf4b0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.187+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/4165178718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4acf60 0x5597bf4b0e30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x5597bf4b1e90 tx=0x7f9e4c00b410 comp rx=0 tx=0).stop 2024-06-06T04:32:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.187+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/4165178718 shutdown_connections 2024-06-06T04:32:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.187+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/4165178718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4ad880 0x5597bf4b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.187+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/4165178718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4a77a0 0x5597bf4a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.187+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/4165178718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4acf60 0x5597bf4b0e30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.187+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/4165178718 >> 172.21.15.45:0/4165178718 conn(0x5597bf3d4080 msgr2=0x5597bf4b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.187+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/4165178718 shutdown_connections 2024-06-06T04:32:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.187+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/4165178718 wait complete. 2024-06-06T04:32:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.188+0000 7f9e6bd3bb80 1 -- start start 2024-06-06T04:32:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.188+0000 7f9e6bd3bb80 1 Processor -- start 2024-06-06T04:32:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.188+0000 7f9e6bd3bb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4a77a0 0x5597bf4b9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.188+0000 7f9e6bd3bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4acf60 0x5597bf4b9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.188+0000 7f9e6bd3bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4ad880 0x5597bf4b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.188+0000 7f9e6bd3bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c1b2e70 con 0x5597bf4ad880 2024-06-06T04:32:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.188+0000 7f9e6bd3bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c1b42e0 con 0x5597bf4acf60 2024-06-06T04:32:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.188+0000 7f9e6bd3bb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c1b3540 con 0x5597bf4a77a0 2024-06-06T04:32:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.188+0000 7f9e6bd3bb80 10 client.0 _ll_register_callbacks cb 0x5597bf4ab4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.188+0000 7f9e5b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4ad880 0x5597bf4b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.188+0000 7f9e5b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4ad880 0x5597bf4b64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48848/0 (socket says 172.21.15.45:48848) 2024-06-06T04:32:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.188+0000 7f9e5b7fe700 1 -- 172.21.15.45:0/4085860155 learned_addr learned my addr 172.21.15.45:0/4085860155 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.188+0000 7f9e5affd700 1 --2- 172.21.15.45:0/4085860155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4acf60 0x5597bf4b9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.189+0000 7f9e5b7fe700 1 --2- 172.21.15.45:0/4085860155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4a77a0 0x5597bf4b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.189+0000 7f9e5b7fe700 1 -- 172.21.15.45:0/4085860155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4a77a0 msgr2=0x5597bf4b9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.189+0000 7f9e5b7fe700 1 --2- 172.21.15.45:0/4085860155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4a77a0 0x5597bf4b9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.189+0000 7f9e5b7fe700 1 -- 172.21.15.45:0/4085860155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4acf60 msgr2=0x5597bf4b9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.189+0000 7f9e5b7fe700 1 --2- 172.21.15.45:0/4085860155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4acf60 0x5597bf4b9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.189+0000 7f9e5b7fe700 1 -- 172.21.15.45:0/4085860155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597bf4b6ae0 con 0x5597bf4ad880 2024-06-06T04:32:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.189+0000 7f9e5b7fe700 1 --2- 172.21.15.45:0/4085860155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4ad880 0x5597bf4b64f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009550 tx=0x7f9e54009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509bedaf97b4dc83 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.189+0000 7f9e3eff5700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.189+0000 7f9e3eff5700 1 -- 172.21.15.45:0/4085860155 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54019eb0 con 0x5597bf4ad880 2024-06-06T04:32:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.189+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/4085860155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e4c010e40 con 0x5597bf4ad880 2024-06-06T04:32:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.189+0000 7f9e3eff5700 1 -- 172.21.15.45:0/4085860155 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f9e540195e0 con 0x5597bf4ad880 2024-06-06T04:32:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.190+0000 7f9e3eff5700 1 -- 172.21.15.45:0/4085860155 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9e5404a050 con 0x5597bf4ad880 2024-06-06T04:32:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.190+0000 7f9e6bd3bb80 10 client.15519 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.190+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/4085860155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5597bf52c320 con 0x5597bf4ad880 2024-06-06T04:32:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.190+0000 7f9e3eff5700 1 -- 172.21.15.45:0/4085860155 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e4c010e40 con 0x5597bf4ad880 2024-06-06T04:32:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.190+0000 7f9e3eff5700 1 -- 172.21.15.45:0/4085860155 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9e54018650 con 0x5597bf4ad880 2024-06-06T04:32:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.190+0000 7f9e6bd3bb80 10 client.15519 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:32.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.190+0000 7f9e6bd3bb80 10 client.15519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:32.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.190+0000 7f9e6bd3bb80 20 client.15519 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:32.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.190+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/4085860155 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5597bf4c0dc0 0x5597bf52ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:32.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.190+0000 7f9e6bd3bb80 4 client.15519 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T04:32:32.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.190+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/4085860155 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5597bf4002d0 con 0x5597bf4c0dc0 2024-06-06T04:32:32.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.191+0000 7f9e5affd700 1 --2- 172.21.15.45:0/4085860155 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5597bf4c0dc0 0x5597bf52ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:32.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.191+0000 7f9e3eff5700 10 client.15519 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:32.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.191+0000 7f9e5affd700 1 --2- 172.21.15.45:0/4085860155 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5597bf4c0dc0 0x5597bf52ea70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39f3fada99e5530d server_cookie=d861a4d48395d15e in_seq=0 out_seq=0 2024-06-06T04:32:32.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.191+0000 7f9e3eff5700 1 -- 172.21.15.45:0/4085860155 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5597bf4002d0 con 0x5597bf4c0dc0 2024-06-06T04:32:32.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.191+0000 7f9e3eff5700 10 client.15519 handle_command_reply: tid=0 2024-06-06T04:32:32.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.191+0000 7f9e6bd3bb80 1 client.15519 shutdown 2024-06-06T04:32:32.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.191+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/4085860155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4ad880 msgr2=0x5597bf4b64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.191+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/4085860155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4ad880 0x5597bf4b64f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009550 tx=0x7f9e54009900 comp rx=0 tx=0).stop 2024-06-06T04:32:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.192+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/4085860155 shutdown_connections 2024-06-06T04:32:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.192+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/4085860155 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5597bf4c0dc0 0x5597bf52ea70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.192+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/4085860155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597bf4ad880 0x5597bf4b64f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.192+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/4085860155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5597bf4a77a0 0x5597bf4b9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.192+0000 7f9e3eff5700 0 client.15519 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.192+0000 7f9e6bd3bb80 1 --2- 172.21.15.45:0/4085860155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597bf4acf60 0x5597bf4b9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.192+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/4085860155 >> 172.21.15.45:0/4085860155 conn(0x5597bf3d4080 msgr2=0x5597bf4ac290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.192+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/4085860155 shutdown_connections 2024-06-06T04:32:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.192+0000 7f9e6bd3bb80 1 -- 172.21.15.45:0/4085860155 wait complete. 2024-06-06T04:32:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.192+0000 7f9e6bd3bb80 20 client.15519 trim_cache size 0 max 16384 2024-06-06T04:32:32.197 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:32:32.198 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-06-06T04:32:32.198 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-06-06T04:32:32.198 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:32:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.193+0000 7f9e614b8700 1 -- 172.21.15.45:0/3138929281 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9e440a2d30 msgr2=0x7f9e440a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.193+0000 7f9e614b8700 1 --2- 172.21.15.45:0/3138929281 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9e440a2d30 0x7f9e440a51f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9e500d40d0 tx=0x7f9e500d4040 comp rx=0 tx=0).stop 2024-06-06T04:32:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.193+0000 7f9e614b8700 1 -- 172.21.15.45:0/3138929281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e5c102ee0 msgr2=0x7f9e5c19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.193+0000 7f9e614b8700 1 --2- 172.21.15.45:0/3138929281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e5c102ee0 0x7f9e5c19f700 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9e5001a040 tx=0x7f9e5000b980 comp rx=0 tx=0).stop 2024-06-06T04:32:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.193+0000 7f9e614b8700 1 -- 172.21.15.45:0/3138929281 shutdown_connections 2024-06-06T04:32:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.193+0000 7f9e614b8700 1 --2- 172.21.15.45:0/3138929281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e5c107450 0x7f9e5c1a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.193+0000 7f9e614b8700 1 --2- 172.21.15.45:0/3138929281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e5c102540 0x7f9e5c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.193+0000 7f9e614b8700 1 --2- 172.21.15.45:0/3138929281 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f9e440a2d30 0x7f9e440a51f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.193+0000 7f9e614b8700 1 --2- 172.21.15.45:0/3138929281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e5c102ee0 0x7f9e5c19f700 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.193+0000 7f9e614b8700 1 -- 172.21.15.45:0/3138929281 >> 172.21.15.45:0/3138929281 conn(0x7f9e5c0795f0 msgr2=0x7f9e5c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.193+0000 7f9e614b8700 1 -- 172.21.15.45:0/3138929281 shutdown_connections 2024-06-06T04:32:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:32.193+0000 7f9e614b8700 1 -- 172.21.15.45:0/3138929281 wait complete. 2024-06-06T04:32:32.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:32 smithi086 ceph-mon[36053]: pgmap v463: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:32.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:32 smithi154 ceph-mon[36428]: pgmap v463: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:32.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:32 smithi045 ceph-mon[32299]: pgmap v463: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:33.001 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-06T04:32:33.002 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:32:33.346 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:32:34.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:34 smithi086 ceph-mon[36053]: pgmap v464: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:34.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:34 smithi154 ceph-mon[36428]: pgmap v464: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:34.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:34 smithi045 ceph-mon[32299]: pgmap v464: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.270+0000 7f0732fa4700 1 -- 172.21.15.45:0/2662336390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072c06ab30 msgr2=0x7f072c10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.270+0000 7f0732fa4700 1 --2- 172.21.15.45:0/2662336390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072c06ab30 0x7f072c10f320 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f072400b0a0 tx=0x7f072400b3b0 comp rx=0 tx=0).stop 2024-06-06T04:32:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.270+0000 7f0732fa4700 1 -- 172.21.15.45:0/2662336390 shutdown_connections 2024-06-06T04:32:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.270+0000 7f0732fa4700 1 --2- 172.21.15.45:0/2662336390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072c06ab30 0x7f072c10f320 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.270+0000 7f0732fa4700 1 --2- 172.21.15.45:0/2662336390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f072c108500 0x7f072c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.270+0000 7f0732fa4700 1 --2- 172.21.15.45:0/2662336390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f072c06a170 0x7f072c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.270+0000 7f0732fa4700 1 -- 172.21.15.45:0/2662336390 >> 172.21.15.45:0/2662336390 conn(0x7f072c078340 msgr2=0x7f072c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.270+0000 7f0732fa4700 1 -- 172.21.15.45:0/2662336390 shutdown_connections 2024-06-06T04:32:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.270+0000 7f0732fa4700 1 -- 172.21.15.45:0/2662336390 wait complete. 2024-06-06T04:32:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.271+0000 7f0732fa4700 1 Processor -- start 2024-06-06T04:32:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.271+0000 7f0732fa4700 1 -- start start 2024-06-06T04:32:35.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.271+0000 7f0732fa4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f072c06a170 0x7f072c10efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:35.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.271+0000 7f0732fa4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072c06ab30 0x7f072c10c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:35.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.271+0000 7f0732fa4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f072c108500 0x7f072c10c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:35.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.271+0000 7f0732fa4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f072c102e00 con 0x7f072c06ab30 2024-06-06T04:32:35.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.271+0000 7f0732fa4700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f072c102c80 con 0x7f072c06a170 2024-06-06T04:32:35.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.271+0000 7f0732fa4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f072c102f80 con 0x7f072c108500 2024-06-06T04:32:35.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.271+0000 7f0730d40700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f072c06a170 0x7f072c10efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:35.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.271+0000 7f0730d40700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f072c06a170 0x7f072c10efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:59402/0 (socket says 172.21.15.45:59402) 2024-06-06T04:32:35.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.271+0000 7f0730d40700 1 -- 172.21.15.45:0/2628261662 learned_addr learned my addr 172.21.15.45:0/2628261662 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:35.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.272+0000 7f0731541700 1 --2- 172.21.15.45:0/2628261662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f072c108500 0x7f072c10c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:35.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.272+0000 7f0730d40700 1 -- 172.21.15.45:0/2628261662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f072c108500 msgr2=0x7f072c10c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:35.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.272+0000 7f0730d40700 1 --2- 172.21.15.45:0/2628261662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f072c108500 0x7f072c10c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:35.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.272+0000 7f0730d40700 1 -- 172.21.15.45:0/2628261662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072c06ab30 msgr2=0x7f072c10c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:35.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.272+0000 7f0730d40700 1 --2- 172.21.15.45:0/2628261662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072c06ab30 0x7f072c10c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:35.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.272+0000 7f0730d40700 1 -- 172.21.15.45:0/2628261662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072c10cce0 con 0x7f072c06a170 2024-06-06T04:32:35.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.272+0000 7f0731541700 1 --2- 172.21.15.45:0/2628261662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f072c108500 0x7f072c10c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:35.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.272+0000 7f0730d40700 1 --2- 172.21.15.45:0/2628261662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f072c06a170 0x7f072c10efc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f072000d980 tx=0x7f072000dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:35.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.272+0000 7f0729ffb700 1 -- 172.21.15.45:0/2628261662 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720016070 con 0x7f072c06a170 2024-06-06T04:32:35.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.272+0000 7f0732fa4700 1 -- 172.21.15.45:0/2628261662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072c1ab530 con 0x7f072c06a170 2024-06-06T04:32:35.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.272+0000 7f0729ffb700 1 -- 172.21.15.45:0/2628261662 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07200044d0 con 0x7f072c06a170 2024-06-06T04:32:35.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.272+0000 7f0729ffb700 1 -- 172.21.15.45:0/2628261662 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720004dc0 con 0x7f072c06a170 2024-06-06T04:32:35.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.273+0000 7f0732fa4700 1 -- 172.21.15.45:0/2628261662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f072c1aba10 con 0x7f072c06a170 2024-06-06T04:32:35.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.274+0000 7f0732fa4700 1 -- 172.21.15.45:0/2628261662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f072c103bc0 con 0x7f072c06a170 2024-06-06T04:32:35.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.275+0000 7f0729ffb700 1 -- 172.21.15.45:0/2628261662 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f072000bcf0 con 0x7f072c06a170 2024-06-06T04:32:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.276+0000 7f0729ffb700 1 --2- 172.21.15.45:0/2628261662 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f071c0a2ca0 0x7f071c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.276+0000 7f0729ffb700 1 -- 172.21.15.45:0/2628261662 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f07200cf1c0 con 0x7f072c06a170 2024-06-06T04:32:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.276+0000 7f072bfff700 1 --2- 172.21.15.45:0/2628261662 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f071c0a2ca0 0x7f071c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:35.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.277+0000 7f072bfff700 1 --2- 172.21.15.45:0/2628261662 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f071c0a2ca0 0x7f071c0a5160 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0718005950 tx=0x7f07180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:35.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.280+0000 7f0729ffb700 1 -- 172.21.15.45:0/2628261662 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f072001f020 con 0x7f072c06a170 2024-06-06T04:32:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.779+0000 7f0732fa4700 1 -- 172.21.15.45:0/2628261662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f072c1abc70 con 0x7f072c06a170 2024-06-06T04:32:35.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.780+0000 7f0729ffb700 1 -- 172.21.15.45:0/2628261662 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11858 (secure 0 0 0) 0x7f072009bd00 con 0x7f072c06a170 2024-06-06T04:32:35.781 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:32:35.781 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T04:27:24.462897+0000","modified":"2024-06-06T04:31:17.956589+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24451,24463,24473]},"id":1}]} 2024-06-06T04:32:35.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.783+0000 7f0732fa4700 1 -- 172.21.15.45:0/2628261662 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f071c0a2ca0 msgr2=0x7f071c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:35.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.783+0000 7f0732fa4700 1 --2- 172.21.15.45:0/2628261662 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f071c0a2ca0 0x7f071c0a5160 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0718005950 tx=0x7f07180058e0 comp rx=0 tx=0).stop 2024-06-06T04:32:35.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.783+0000 7f0732fa4700 1 -- 172.21.15.45:0/2628261662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f072c06a170 msgr2=0x7f072c10efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:35.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.783+0000 7f0732fa4700 1 --2- 172.21.15.45:0/2628261662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f072c06a170 0x7f072c10efc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f072000d980 tx=0x7f072000dd40 comp rx=0 tx=0).stop 2024-06-06T04:32:35.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.784+0000 7f0732fa4700 1 -- 172.21.15.45:0/2628261662 shutdown_connections 2024-06-06T04:32:35.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.784+0000 7f0732fa4700 1 --2- 172.21.15.45:0/2628261662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f072c108500 0x7f072c10c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:35.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.784+0000 7f0732fa4700 1 --2- 172.21.15.45:0/2628261662 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f071c0a2ca0 0x7f071c0a5160 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:35.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.784+0000 7f0732fa4700 1 --2- 172.21.15.45:0/2628261662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f072c06a170 0x7f072c10efc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:35.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.784+0000 7f0732fa4700 1 --2- 172.21.15.45:0/2628261662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072c06ab30 0x7f072c10c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:35.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.784+0000 7f0732fa4700 1 -- 172.21.15.45:0/2628261662 >> 172.21.15.45:0/2628261662 conn(0x7f072c078340 msgr2=0x7f072c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:35.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.784+0000 7f0732fa4700 1 -- 172.21.15.45:0/2628261662 shutdown_connections 2024-06-06T04:32:35.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:35.784+0000 7f0732fa4700 1 -- 172.21.15.45:0/2628261662 wait complete. 2024-06-06T04:32:35.785 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-06-06T04:32:36.076 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:36 smithi045 ceph-mon[32299]: pgmap v465: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:36.076 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:36 smithi045 ceph-mon[32299]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-06-06T04:32:36.076 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:36 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2628261662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:32:36.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:36 smithi086 ceph-mon[36053]: pgmap v465: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:36.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:36 smithi086 ceph-mon[36053]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-06-06T04:32:36.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:36 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2628261662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:32:36.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:36 smithi154 ceph-mon[36428]: pgmap v465: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-06T04:32:36.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:36 smithi154 ceph-mon[36428]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-06-06T04:32:36.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:36 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2628261662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:32:36.610 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-06T04:32:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.693+0000 7fa2309af640 1 -- 172.21.15.45:0/3189538080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22c101a10 msgr2=0x7fa22c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.693+0000 7fa2309af640 1 --2- 172.21.15.45:0/3189538080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22c101a10 0x7fa22c101e30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009900 tx=0x7fa21c02a410 comp rx=0 tx=0).stop 2024-06-06T04:32:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.693+0000 7fa2309af640 1 -- 172.21.15.45:0/3189538080 shutdown_connections 2024-06-06T04:32:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.693+0000 7fa2309af640 1 --2- 172.21.15.45:0/3189538080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22c1035c0 0x7fa22c109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.693+0000 7fa2309af640 1 --2- 172.21.15.45:0/3189538080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa22c102c00 0x7fa22c103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.693+0000 7fa2309af640 1 --2- 172.21.15.45:0/3189538080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22c101a10 0x7fa22c101e30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.693+0000 7fa2309af640 1 -- 172.21.15.45:0/3189538080 >> 172.21.15.45:0/3189538080 conn(0x7fa22c0fcf60 msgr2=0x7fa22c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.693+0000 7fa2309af640 1 -- 172.21.15.45:0/3189538080 shutdown_connections 2024-06-06T04:32:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.693+0000 7fa2309af640 1 -- 172.21.15.45:0/3189538080 wait complete. 2024-06-06T04:32:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.694+0000 7fa2309af640 1 Processor -- start 2024-06-06T04:32:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.694+0000 7fa2309af640 1 -- start start 2024-06-06T04:32:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.694+0000 7fa2309af640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22c101a10 0x7fa22c197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.694+0000 7fa2309af640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa22c102c00 0x7fa22c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.694+0000 7fa2309af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22c1035c0 0x7fa22c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.694+0000 7fa2309af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c10d530 con 0x7fa22c1035c0 2024-06-06T04:32:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.694+0000 7fa2309af640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c10d3b0 con 0x7fa22c101a10 2024-06-06T04:32:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.694+0000 7fa2309af640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c10d6b0 con 0x7fa22c102c00 2024-06-06T04:32:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.695+0000 7fa229d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa22c102c00 0x7fa22c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.695+0000 7fa22a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22c101a10 0x7fa22c197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.695+0000 7fa22ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22c1035c0 0x7fa22c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.695+0000 7fa22a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22c101a10 0x7fa22c197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:59412/0 (socket says 172.21.15.45:59412) 2024-06-06T04:32:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.695+0000 7fa22a59c640 1 -- 172.21.15.45:0/601443870 learned_addr learned my addr 172.21.15.45:0/601443870 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.695+0000 7fa229d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa22c102c00 0x7fa22c198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:60162/0 (socket says 172.21.15.45:60162) 2024-06-06T04:32:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.695+0000 7fa229d9b640 1 -- 172.21.15.45:0/601443870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22c101a10 msgr2=0x7fa22c197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.695+0000 7fa229d9b640 1 --2- 172.21.15.45:0/601443870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22c101a10 0x7fa22c197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.695+0000 7fa229d9b640 1 -- 172.21.15.45:0/601443870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22c1035c0 msgr2=0x7fa22c19f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.695+0000 7fa229d9b640 1 --2- 172.21.15.45:0/601443870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22c1035c0 0x7fa22c19f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.695+0000 7fa229d9b640 1 -- 172.21.15.45:0/601443870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22c073bd0 con 0x7fa22c102c00 2024-06-06T04:32:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.695+0000 7fa22a59c640 1 --2- 172.21.15.45:0/601443870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22c101a10 0x7fa22c197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.695+0000 7fa22ad9d640 1 --2- 172.21.15.45:0/601443870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22c1035c0 0x7fa22c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.695+0000 7fa229d9b640 1 --2- 172.21.15.45:0/601443870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa22c102c00 0x7fa22c198110 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa22000e9f0 tx=0x7fa22000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.696+0000 7fa21b7fe640 1 -- 172.21.15.45:0/601443870 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22000ccc0 con 0x7fa22c102c00 2024-06-06T04:32:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.696+0000 7fa2309af640 1 -- 172.21.15.45:0/601443870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22c073ec0 con 0x7fa22c102c00 2024-06-06T04:32:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.696+0000 7fa2309af640 1 -- 172.21.15.45:0/601443870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa22c074410 con 0x7fa22c102c00 2024-06-06T04:32:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.696+0000 7fa21b7fe640 1 -- 172.21.15.45:0/601443870 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa220005020 con 0x7fa22c102c00 2024-06-06T04:32:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.696+0000 7fa21b7fe640 1 -- 172.21.15.45:0/601443870 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22000ce60 con 0x7fa22c102c00 2024-06-06T04:32:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.699+0000 7fa21b7fe640 1 -- 172.21.15.45:0/601443870 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa220004b20 con 0x7fa22c102c00 2024-06-06T04:32:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.699+0000 7fa233795740 1 Processor -- start 2024-06-06T04:32:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.700+0000 7fa233795740 1 -- start start 2024-06-06T04:32:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.700+0000 7fa233795740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57ea010 0x560ee57ea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.700+0000 7fa233795740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee57ef500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.700+0000 7fa233795740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57efb70 0x560ee57f2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.700+0000 7fa233795740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560ee574d4f0 con 0x560ee57eaa00 2024-06-06T04:32:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.700+0000 7fa233795740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ee56eb870 con 0x560ee57ea010 2024-06-06T04:32:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.700+0000 7fa233795740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560ee57494b0 con 0x560ee57efb70 2024-06-06T04:32:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.700+0000 7fa22a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee57ef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.700+0000 7fa22a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee57ef500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48910/0 (socket says 172.21.15.45:48910) 2024-06-06T04:32:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.700+0000 7fa22a59c640 1 -- 172.21.15.45:0/4179387380 learned_addr learned my addr 172.21.15.45:0/4179387380 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.700+0000 7fa22ad9d640 1 --2- 172.21.15.45:0/4179387380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57ea010 0x560ee57ea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.700+0000 7fa229d9b640 1 --2- 172.21.15.45:0/4179387380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57efb70 0x560ee57f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.701+0000 7fa22a59c640 1 -- 172.21.15.45:0/4179387380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57efb70 msgr2=0x560ee57f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.701+0000 7fa22a59c640 1 --2- 172.21.15.45:0/4179387380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57efb70 0x560ee57f2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.701+0000 7fa22a59c640 1 -- 172.21.15.45:0/4179387380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57ea010 msgr2=0x560ee57ea430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.701+0000 7fa22a59c640 1 --2- 172.21.15.45:0/4179387380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57ea010 0x560ee57ea430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.701+0000 7fa22a59c640 1 -- 172.21.15.45:0/4179387380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ee57079b0 con 0x560ee57eaa00 2024-06-06T04:32:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.701+0000 7fa229d9b640 1 --2- 172.21.15.45:0/4179387380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57efb70 0x560ee57f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.701+0000 7fa21b7fe640 1 --2- 172.21.15.45:0/601443870 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa2140a2d30 0x7fa2140a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.701+0000 7fa22a59c640 1 --2- 172.21.15.45:0/4179387380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee57ef500 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa21c006010 tx=0x7fa21c005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a815a3a5a9167d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.701+0000 7fa218ff9640 1 -- 172.21.15.45:0/4179387380 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c035070 con 0x560ee57eaa00 2024-06-06T04:32:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.701+0000 7fa22ad9d640 1 --2- 172.21.15.45:0/601443870 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa2140a2d30 0x7fa2140a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.701+0000 7fa218ff9640 1 -- 172.21.15.45:0/4179387380 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa21c0318c0 con 0x560ee57eaa00 2024-06-06T04:32:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.701+0000 7fa218ff9640 1 -- 172.21.15.45:0/4179387380 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c045470 con 0x560ee57eaa00 2024-06-06T04:32:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.702+0000 7fa233795740 1 -- 172.21.15.45:0/4179387380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 msgr2=0x560ee57ef500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.702+0000 7fa233795740 1 --2- 172.21.15.45:0/4179387380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee57ef500 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa21c006010 tx=0x7fa21c005be0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.702+0000 7fa21b7fe640 1 -- 172.21.15.45:0/601443870 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fa2200cf430 con 0x7fa22c102c00 2024-06-06T04:32:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.702+0000 7fa22ad9d640 1 --2- 172.21.15.45:0/601443870 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa2140a2d30 0x7fa2140a51f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x560ee57eb260 tx=0x7fa224009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.702+0000 7fa233795740 1 -- 172.21.15.45:0/4179387380 shutdown_connections 2024-06-06T04:32:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.703+0000 7fa233795740 1 --2- 172.21.15.45:0/4179387380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57efb70 0x560ee57f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.703+0000 7fa233795740 1 --2- 172.21.15.45:0/4179387380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee57ef500 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.703+0000 7fa233795740 1 --2- 172.21.15.45:0/4179387380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57ea010 0x560ee57ea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.703+0000 7fa233795740 1 -- 172.21.15.45:0/4179387380 >> 172.21.15.45:0/4179387380 conn(0x560ee57e5680 msgr2=0x560ee57e5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.703+0000 7fa233795740 1 -- 172.21.15.45:0/4179387380 shutdown_connections 2024-06-06T04:32:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.703+0000 7fa233795740 1 -- 172.21.15.45:0/4179387380 wait complete. 2024-06-06T04:32:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.703+0000 7fa233795740 1 -- start start 2024-06-06T04:32:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.704+0000 7fa233795740 1 Processor -- start 2024-06-06T04:32:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.704+0000 7fa233795740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57ea010 0x560ee57f4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.704+0000 7fa233795740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57eaa00 0x560ee57f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.704+0000 7fa233795740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57efb70 0x560ee57fb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.704+0000 7fa233795740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560ee57f2ac0 con 0x560ee57efb70 2024-06-06T04:32:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.705+0000 7fa233795740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ee56d6ae0 con 0x560ee57eaa00 2024-06-06T04:32:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.705+0000 7fa233795740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560ee57f2c40 con 0x560ee57ea010 2024-06-06T04:32:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.705+0000 7fa22ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57efb70 0x560ee57fb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.705+0000 7fa229d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57eaa00 0x560ee57f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.705+0000 7fa22ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57efb70 0x560ee57fb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48912/0 (socket says 172.21.15.45:48912) 2024-06-06T04:32:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.705+0000 7fa22ad9d640 1 -- 172.21.15.45:0/1665356187 learned_addr learned my addr 172.21.15.45:0/1665356187 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.705+0000 7fa22a59c640 1 --2- 172.21.15.45:0/1665356187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57ea010 0x560ee57f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.705+0000 7fa233795740 10 client.0 _ll_register_callbacks cb 0x560ee570f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.705+0000 7fa22ad9d640 1 -- 172.21.15.45:0/1665356187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57ea010 msgr2=0x560ee57f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.705+0000 7fa22ad9d640 1 --2- 172.21.15.45:0/1665356187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57ea010 0x560ee57f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.705+0000 7fa22ad9d640 1 -- 172.21.15.45:0/1665356187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57eaa00 msgr2=0x560ee57f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.705+0000 7fa22ad9d640 1 --2- 172.21.15.45:0/1665356187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57eaa00 0x560ee57f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.705+0000 7fa22ad9d640 1 -- 172.21.15.45:0/1665356187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ee5461210 con 0x560ee57efb70 2024-06-06T04:32:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.705+0000 7fa22ad9d640 1 --2- 172.21.15.45:0/1665356187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57efb70 0x560ee57fb7a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa22400fb50 tx=0x7fa224004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db2184ff7ecb9b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.705+0000 7fa211ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.706+0000 7fa211ffb640 1 -- 172.21.15.45:0/1665356187 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224011c90 con 0x560ee57efb70 2024-06-06T04:32:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.706+0000 7fa211ffb640 1 -- 172.21.15.45:0/1665356187 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fa22401f420 con 0x560ee57efb70 2024-06-06T04:32:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.706+0000 7fa233795740 1 -- 172.21.15.45:0/1665356187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa21c0318c0 con 0x560ee57efb70 2024-06-06T04:32:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.706+0000 7fa211ffb640 1 -- 172.21.15.45:0/1665356187 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa224011e30 con 0x560ee57efb70 2024-06-06T04:32:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.706+0000 7fa233795740 10 client.15534 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.706+0000 7fa233795740 1 -- 172.21.15.45:0/1665356187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ee552d620 con 0x560ee57efb70 2024-06-06T04:32:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.707+0000 7fa211ffb640 1 -- 172.21.15.45:0/1665356187 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa21c0318c0 con 0x560ee57efb70 2024-06-06T04:32:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.707+0000 7fa211ffb640 1 -- 172.21.15.45:0/1665356187 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa224027070 con 0x560ee57efb70 2024-06-06T04:32:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.707+0000 7fa233795740 10 client.15534 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.707+0000 7fa233795740 10 client.15534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.707+0000 7fa233795740 20 client.15534 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.707+0000 7fa233795740 1 --2- 172.21.15.45:0/1665356187 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560ee5800180 0x560ee5802620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.707+0000 7fa233795740 4 client.15534 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:32:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.707+0000 7fa233795740 1 -- 172.21.15.45:0/1665356187 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ee5760f10 con 0x560ee5800180 2024-06-06T04:32:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.708+0000 7fa22a59c640 1 --2- 172.21.15.45:0/1665356187 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560ee5800180 0x560ee5802620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.708+0000 7fa211ffb640 10 client.15534 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.708+0000 7fa22a59c640 1 --2- 172.21.15.45:0/1665356187 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560ee5800180 0x560ee5802620 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15e8861bd1d77038 server_cookie=a8d2faf9436cb156 in_seq=0 out_seq=0 2024-06-06T04:32:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.709+0000 7fa211ffb640 1 -- 172.21.15.45:0/1665356187 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560ee5760f10 con 0x560ee5800180 2024-06-06T04:32:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.709+0000 7fa211ffb640 10 client.15534 handle_command_reply: tid=0 2024-06-06T04:32:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.709+0000 7fa233795740 1 client.15534 shutdown 2024-06-06T04:32:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.709+0000 7fa233795740 1 -- 172.21.15.45:0/1665356187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57efb70 msgr2=0x560ee57fb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.709+0000 7fa233795740 1 --2- 172.21.15.45:0/1665356187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57efb70 0x560ee57fb7a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa22400fb50 tx=0x7fa224004ab0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.709+0000 7fa233795740 1 -- 172.21.15.45:0/1665356187 shutdown_connections 2024-06-06T04:32:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.709+0000 7fa233795740 1 --2- 172.21.15.45:0/1665356187 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560ee5800180 0x560ee5802620 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.710+0000 7fa233795740 1 --2- 172.21.15.45:0/1665356187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57eaa00 0x560ee57f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.710+0000 7fa211ffb640 0 client.15534 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.710+0000 7fa233795740 1 --2- 172.21.15.45:0/1665356187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57efb70 0x560ee57fb7a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.710+0000 7fa233795740 1 --2- 172.21.15.45:0/1665356187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57ea010 0x560ee57f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.710+0000 7fa233795740 1 -- 172.21.15.45:0/1665356187 >> 172.21.15.45:0/1665356187 conn(0x560ee57e5680 msgr2=0x560ee57e47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.710+0000 7fa233795740 1 -- 172.21.15.45:0/1665356187 shutdown_connections 2024-06-06T04:32:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.710+0000 7fa233795740 1 -- 172.21.15.45:0/1665356187 wait complete. 2024-06-06T04:32:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.710+0000 7fa233795740 20 client.15534 trim_cache size 0 max 16384 2024-06-06T04:32:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.740+0000 7fa233795740 1 Processor -- start 2024-06-06T04:32:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.740+0000 7fa233795740 1 -- start start 2024-06-06T04:32:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.741+0000 7fa233795740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57ea010 0x560ee57e8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.741+0000 7fa233795740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee57e8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.741+0000 7fa233795740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57efb70 0x560ee57ecba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.741+0000 7fa229d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee57e8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.741+0000 7fa229d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee57e8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48920/0 (socket says 172.21.15.45:48920) 2024-06-06T04:32:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.741+0000 7fa229d9b640 1 -- 172.21.15.45:0/960309826 learned_addr learned my addr 172.21.15.45:0/960309826 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.741+0000 7fa233795740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c012510 con 0x560ee57eaa00 2024-06-06T04:32:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.741+0000 7fa22a59c640 1 --2- 172.21.15.45:0/960309826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57ea010 0x560ee57e8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.741+0000 7fa22ad9d640 1 --2- 172.21.15.45:0/960309826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57efb70 0x560ee57ecba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.742+0000 7fa233795740 1 -- 172.21.15.45:0/960309826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c00fde0 con 0x560ee57ea010 2024-06-06T04:32:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.742+0000 7fa233795740 1 -- 172.21.15.45:0/960309826 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c00dc30 con 0x560ee57efb70 2024-06-06T04:32:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.742+0000 7fa229d9b640 1 -- 172.21.15.45:0/960309826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57efb70 msgr2=0x560ee57ecba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.742+0000 7fa229d9b640 1 --2- 172.21.15.45:0/960309826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57efb70 0x560ee57ecba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.742+0000 7fa229d9b640 1 -- 172.21.15.45:0/960309826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57ea010 msgr2=0x560ee57e8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.742+0000 7fa229d9b640 1 --2- 172.21.15.45:0/960309826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57ea010 0x560ee57e8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.742+0000 7fa229d9b640 1 -- 172.21.15.45:0/960309826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ee57e54b0 con 0x560ee57eaa00 2024-06-06T04:32:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.742+0000 7fa229d9b640 1 --2- 172.21.15.45:0/960309826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee57e8ad0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa22009c3a0 tx=0x7fa220087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe87906972199237 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.743+0000 7fa213fff640 1 -- 172.21.15.45:0/960309826 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22001c070 con 0x560ee57eaa00 2024-06-06T04:32:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.743+0000 7fa213fff640 1 -- 172.21.15.45:0/960309826 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa22009cda0 con 0x560ee57eaa00 2024-06-06T04:32:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.743+0000 7fa213fff640 1 -- 172.21.15.45:0/960309826 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2200cf0b0 con 0x560ee57eaa00 2024-06-06T04:32:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.744+0000 7fa233795740 1 -- 172.21.15.45:0/960309826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 msgr2=0x560ee57e8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.744+0000 7fa233795740 1 --2- 172.21.15.45:0/960309826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee57e8ad0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa22009c3a0 tx=0x7fa220087040 comp rx=0 tx=0).stop 2024-06-06T04:32:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.744+0000 7fa233795740 1 -- 172.21.15.45:0/960309826 shutdown_connections 2024-06-06T04:32:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.744+0000 7fa233795740 1 --2- 172.21.15.45:0/960309826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57efb70 0x560ee57ecba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.744+0000 7fa233795740 1 --2- 172.21.15.45:0/960309826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee57e8ad0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.744+0000 7fa233795740 1 --2- 172.21.15.45:0/960309826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57ea010 0x560ee57e8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.744+0000 7fa233795740 1 -- 172.21.15.45:0/960309826 >> 172.21.15.45:0/960309826 conn(0x560ee57e5680 msgr2=0x560ee57f4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.744+0000 7fa233795740 1 -- 172.21.15.45:0/960309826 shutdown_connections 2024-06-06T04:32:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.744+0000 7fa233795740 1 -- 172.21.15.45:0/960309826 wait complete. 2024-06-06T04:32:36.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.745+0000 7fa233795740 1 -- start start 2024-06-06T04:32:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.745+0000 7fa233795740 1 Processor -- start 2024-06-06T04:32:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.746+0000 7fa233795740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57ea010 0x560ee5807870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.746+0000 7fa233795740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee5807db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.746+0000 7fa233795740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57efb70 0x560ee5868f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.746+0000 7fa22a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57ea010 0x560ee5807870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.746+0000 7fa22a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57ea010 0x560ee5807870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:60198/0 (socket says 172.21.15.45:60198) 2024-06-06T04:32:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.746+0000 7fa22a59c640 1 -- 172.21.15.45:0/1463163080 learned_addr learned my addr 172.21.15.45:0/1463163080 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.746+0000 7fa229d9b640 1 --2- 172.21.15.45:0/1463163080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee5807db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.746+0000 7fa233795740 1 -- 172.21.15.45:0/1463163080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c00e2a0 con 0x560ee57eaa00 2024-06-06T04:32:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.746+0000 7fa233795740 1 -- 172.21.15.45:0/1463163080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c00f710 con 0x560ee57efb70 2024-06-06T04:32:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.746+0000 7fa22ad9d640 1 --2- 172.21.15.45:0/1463163080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57efb70 0x560ee5868f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.746+0000 7fa233795740 1 -- 172.21.15.45:0/1463163080 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c00e970 con 0x560ee57ea010 2024-06-06T04:32:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.747+0000 7fa233795740 10 client.0 _ll_register_callbacks cb 0x7fa224012610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.747+0000 7fa229d9b640 1 -- 172.21.15.45:0/1463163080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57ea010 msgr2=0x560ee5807870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.747+0000 7fa229d9b640 1 --2- 172.21.15.45:0/1463163080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57ea010 0x560ee5807870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.747+0000 7fa229d9b640 1 -- 172.21.15.45:0/1463163080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57efb70 msgr2=0x560ee5868f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.747+0000 7fa229d9b640 1 --2- 172.21.15.45:0/1463163080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57efb70 0x560ee5868f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.747+0000 7fa229d9b640 1 -- 172.21.15.45:0/1463163080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ee58695b0 con 0x560ee57eaa00 2024-06-06T04:32:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.747+0000 7fa229d9b640 1 --2- 172.21.15.45:0/1463163080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee5807db0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x560ee57eae60 tx=0x7fa220005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493ca5b5b45518af server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.747+0000 7fa211ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.747+0000 7fa211ffb640 1 -- 172.21.15.45:0/1463163080 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2200b4060 con 0x560ee57eaa00 2024-06-06T04:32:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.747+0000 7fa211ffb640 1 -- 172.21.15.45:0/1463163080 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fa22009c900 con 0x560ee57eaa00 2024-06-06T04:32:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.747+0000 7fa233795740 1 -- 172.21.15.45:0/1463163080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa22009cda0 con 0x560ee57eaa00 2024-06-06T04:32:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.748+0000 7fa211ffb640 1 -- 172.21.15.45:0/1463163080 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa22001c040 con 0x560ee57eaa00 2024-06-06T04:32:36.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.748+0000 7fa233795740 10 client.15540 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:36.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.748+0000 7fa233795740 1 -- 172.21.15.45:0/1463163080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ee586a820 con 0x560ee57eaa00 2024-06-06T04:32:36.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.748+0000 7fa211ffb640 1 -- 172.21.15.45:0/1463163080 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa22009cda0 con 0x560ee57eaa00 2024-06-06T04:32:36.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.748+0000 7fa211ffb640 1 -- 172.21.15.45:0/1463163080 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa22009baf0 con 0x560ee57eaa00 2024-06-06T04:32:36.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.748+0000 7fa233795740 10 client.15540 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:36.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.748+0000 7fa233795740 10 client.15540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:36.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.748+0000 7fa233795740 20 client.15540 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:36.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.748+0000 7fa233795740 1 --2- 172.21.15.45:0/1463163080 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560ee5800180 0x560ee586d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:36.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.748+0000 7fa233795740 4 client.15540 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T04:32:36.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.748+0000 7fa233795740 1 -- 172.21.15.45:0/1463163080 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ee575ac20 con 0x560ee5800180 2024-06-06T04:32:36.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.749+0000 7fa22a59c640 1 --2- 172.21.15.45:0/1463163080 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560ee5800180 0x560ee586d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:36.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.749+0000 7fa211ffb640 10 client.15540 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:36.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.749+0000 7fa22a59c640 1 --2- 172.21.15.45:0/1463163080 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560ee5800180 0x560ee586d1a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c9fe97abab62ce server_cookie=e5e43a493dcff20c in_seq=0 out_seq=0 2024-06-06T04:32:36.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.755+0000 7fa211ffb640 1 -- 172.21.15.45:0/1463163080 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560ee575ac20 con 0x560ee5800180 2024-06-06T04:32:36.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.755+0000 7fa211ffb640 10 client.15540 handle_command_reply: tid=0 2024-06-06T04:32:36.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.755+0000 7fa233795740 1 client.15540 shutdown 2024-06-06T04:32:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.755+0000 7fa233795740 1 -- 172.21.15.45:0/1463163080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 msgr2=0x560ee5807db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.755+0000 7fa233795740 1 --2- 172.21.15.45:0/1463163080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee5807db0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x560ee57eae60 tx=0x7fa220005c60 comp rx=0 tx=0).stop 2024-06-06T04:32:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.756+0000 7fa233795740 1 -- 172.21.15.45:0/1463163080 shutdown_connections 2024-06-06T04:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.756+0000 7fa233795740 1 --2- 172.21.15.45:0/1463163080 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560ee5800180 0x560ee586d1a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.756+0000 7fa233795740 1 --2- 172.21.15.45:0/1463163080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ee57efb70 0x560ee5868f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.756+0000 7fa233795740 1 --2- 172.21.15.45:0/1463163080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ee57eaa00 0x560ee5807db0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.756+0000 7fa211ffb640 0 client.15540 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.756+0000 7fa233795740 1 --2- 172.21.15.45:0/1463163080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ee57ea010 0x560ee5807870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.756+0000 7fa233795740 1 -- 172.21.15.45:0/1463163080 >> 172.21.15.45:0/1463163080 conn(0x560ee57e5680 msgr2=0x560ee57f41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.756+0000 7fa233795740 1 -- 172.21.15.45:0/1463163080 shutdown_connections 2024-06-06T04:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.756+0000 7fa233795740 1 -- 172.21.15.45:0/1463163080 wait complete. 2024-06-06T04:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.756+0000 7fa233795740 20 client.15540 trim_cache size 0 max 16384 2024-06-06T04:32:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-06-06T04:32:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-06-06T04:32:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-06-06T04:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.757+0000 7fa2309af640 1 -- 172.21.15.45:0/601443870 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa2140a2d30 msgr2=0x7fa2140a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.757+0000 7fa2309af640 1 --2- 172.21.15.45:0/601443870 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa2140a2d30 0x7fa2140a51f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x560ee57eb260 tx=0x7fa224009500 comp rx=0 tx=0).stop 2024-06-06T04:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.757+0000 7fa2309af640 1 -- 172.21.15.45:0/601443870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa22c102c00 msgr2=0x7fa22c198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.757+0000 7fa2309af640 1 --2- 172.21.15.45:0/601443870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa22c102c00 0x7fa22c198110 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa22000e9f0 tx=0x7fa22000edf0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.757+0000 7fa2309af640 1 -- 172.21.15.45:0/601443870 shutdown_connections 2024-06-06T04:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.757+0000 7fa2309af640 1 --2- 172.21.15.45:0/601443870 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa2140a2d30 0x7fa2140a51f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.757+0000 7fa2309af640 1 --2- 172.21.15.45:0/601443870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa22c1035c0 0x7fa22c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.757+0000 7fa2309af640 1 --2- 172.21.15.45:0/601443870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa22c102c00 0x7fa22c198110 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.757+0000 7fa2309af640 1 --2- 172.21.15.45:0/601443870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22c101a10 0x7fa22c197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.757+0000 7fa2309af640 1 -- 172.21.15.45:0/601443870 >> 172.21.15.45:0/601443870 conn(0x7fa22c0fcf60 msgr2=0x7fa22c105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.757+0000 7fa2309af640 1 -- 172.21.15.45:0/601443870 shutdown_connections 2024-06-06T04:32:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:36.757+0000 7fa2309af640 1 -- 172.21.15.45:0/601443870 wait complete. 2024-06-06T04:32:37.018 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:32:38.354 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:38 smithi045 ceph-mon[32299]: pgmap v466: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-06T04:32:38.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:38 smithi086 ceph-mon[36053]: pgmap v466: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-06T04:32:38.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:38 smithi154 ceph-mon[36428]: pgmap v466: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-06T04:32:38.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.801+0000 7f6e09a88700 1 -- 172.21.15.45:0/2530803478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e041085c0 msgr2=0x7f6e041089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:38.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.801+0000 7f6e09a88700 1 --2- 172.21.15.45:0/2530803478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e041085c0 0x7f6e041089a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6df40099c0 tx=0x7f6df4009cd0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.801+0000 7f6e09a88700 1 -- 172.21.15.45:0/2530803478 shutdown_connections 2024-06-06T04:32:38.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.801+0000 7f6e09a88700 1 --2- 172.21.15.45:0/2530803478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e04102ee0 0x7f6e0410f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.801+0000 7f6e09a88700 1 --2- 172.21.15.45:0/2530803478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e041085c0 0x7f6e041089a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.801+0000 7f6e09a88700 1 --2- 172.21.15.45:0/2530803478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e04102520 0x7f6e041029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.801+0000 7f6e09a88700 1 -- 172.21.15.45:0/2530803478 >> 172.21.15.45:0/2530803478 conn(0x7f6e04079680 msgr2=0x7f6e04079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:38.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.801+0000 7f6e09a88700 1 -- 172.21.15.45:0/2530803478 shutdown_connections 2024-06-06T04:32:38.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.801+0000 7f6e09a88700 1 -- 172.21.15.45:0/2530803478 wait complete. 2024-06-06T04:32:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.802+0000 7f6e09a88700 1 Processor -- start 2024-06-06T04:32:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.802+0000 7f6e09a88700 1 -- start start 2024-06-06T04:32:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.802+0000 7f6e09a88700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e04102520 0x7f6e0419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.802+0000 7f6e09a88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e04102ee0 0x7f6e0419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e09a88700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e041085c0 0x7f6e0419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e09a88700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e041126c0 con 0x7f6e04102ee0 2024-06-06T04:32:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e09a88700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e04112540 con 0x7f6e041085c0 2024-06-06T04:32:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e09a88700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6e04112840 con 0x7f6e04102520 2024-06-06T04:32:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e02ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e04102ee0 0x7f6e0419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e037fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e04102520 0x7f6e0419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e037fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e04102520 0x7f6e0419ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:43452/0 (socket says 172.21.15.45:43452) 2024-06-06T04:32:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e037fe700 1 -- 172.21.15.45:0/299970659 learned_addr learned my addr 172.21.15.45:0/299970659 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e02ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e04102ee0 0x7f6e0419b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48788/0 (socket says 172.21.15.45:48788) 2024-06-06T04:32:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e037fe700 1 -- 172.21.15.45:0/299970659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e041085c0 msgr2=0x7f6e0419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e03fff700 1 --2- 172.21.15.45:0/299970659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e041085c0 0x7f6e0419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e037fe700 1 --2- 172.21.15.45:0/299970659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e041085c0 0x7f6e0419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e037fe700 1 -- 172.21.15.45:0/299970659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e04102ee0 msgr2=0x7f6e0419b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e037fe700 1 --2- 172.21.15.45:0/299970659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e04102ee0 0x7f6e0419b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e037fe700 1 -- 172.21.15.45:0/299970659 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0419fd40 con 0x7f6e04102520 2024-06-06T04:32:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e02ffd700 1 --2- 172.21.15.45:0/299970659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e04102ee0 0x7f6e0419b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.803+0000 7f6e03fff700 1 --2- 172.21.15.45:0/299970659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e041085c0 0x7f6e0419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.804+0000 7f6e037fe700 1 --2- 172.21.15.45:0/299970659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e04102520 0x7f6e0419ad30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6df4006010 tx=0x7f6df4009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.804+0000 7f6e00ff9700 1 -- 172.21.15.45:0/299970659 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4027070 con 0x7f6e04102520 2024-06-06T04:32:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.804+0000 7f6e09a88700 1 -- 172.21.15.45:0/299970659 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0419ffd0 con 0x7f6e04102520 2024-06-06T04:32:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.804+0000 7f6e00ff9700 1 -- 172.21.15.45:0/299970659 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6df4003f50 con 0x7f6e04102520 2024-06-06T04:32:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.804+0000 7f6e09a88700 1 -- 172.21.15.45:0/299970659 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e041a78b0 con 0x7f6e04102520 2024-06-06T04:32:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.804+0000 7f6e00ff9700 1 -- 172.21.15.45:0/299970659 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4022cd0 con 0x7f6e04102520 2024-06-06T04:32:38.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.807+0000 7f6e00ff9700 1 -- 172.21.15.45:0/299970659 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6df402a070 con 0x7f6e04102520 2024-06-06T04:32:38.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.807+0000 7f6e00ff9700 1 --2- 172.21.15.45:0/299970659 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6dec0a2c20 0x7f6dec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.807+0000 7f6e00ff9700 1 -- 172.21.15.45:0/299970659 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f6df40d0140 con 0x7f6e04102520 2024-06-06T04:32:38.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.807+0000 7f6e02ffd700 1 --2- 172.21.15.45:0/299970659 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6dec0a2c20 0x7f6dec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:38.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.808+0000 7f6e02ffd700 1 --2- 172.21.15.45:0/299970659 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6dec0a2c20 0x7f6dec0a50e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6e04079f30 tx=0x7f6df800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.809+0000 7f6e1430bb80 1 Processor -- start 2024-06-06T04:32:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.809+0000 7f6e1430bb80 1 -- start start 2024-06-06T04:32:38.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.809+0000 7f6e1430bb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce924f60 0x5649ce925340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.809+0000 7f6e1430bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce925880 0x5649ce91f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.809+0000 7f6e1430bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce91f7a0 0x5649ce91fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.809+0000 7f6e1430bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5649ce811f60 con 0x5649ce91f7a0 2024-06-06T04:32:38.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.809+0000 7f6e1430bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5649ce870850 con 0x5649ce925880 2024-06-06T04:32:38.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.809+0000 7f6e1430bb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5649ce7d8460 con 0x5649ce924f60 2024-06-06T04:32:38.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.810+0000 7f6e02ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce925880 0x5649ce91f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:38.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.810+0000 7f6e02ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce925880 0x5649ce91f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:48646/0 (socket says 172.21.15.45:48646) 2024-06-06T04:32:38.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.810+0000 7f6e02ffd700 1 -- 172.21.15.45:0/3695942274 learned_addr learned my addr 172.21.15.45:0/3695942274 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:38.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.810+0000 7f6e037fe700 1 --2- 172.21.15.45:0/3695942274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce924f60 0x5649ce925340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:38.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.810+0000 7f6e02ffd700 1 -- 172.21.15.45:0/3695942274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce924f60 msgr2=0x5649ce925340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:38.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.810+0000 7f6e02ffd700 1 --2- 172.21.15.45:0/3695942274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce924f60 0x5649ce925340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.810+0000 7f6e02ffd700 1 -- 172.21.15.45:0/3695942274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce91f7a0 msgr2=0x5649ce91fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:38.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.810+0000 7f6e02ffd700 1 --2- 172.21.15.45:0/3695942274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce91f7a0 0x5649ce91fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.810+0000 7f6e02ffd700 1 -- 172.21.15.45:0/3695942274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649ce7bf8d0 con 0x5649ce925880 2024-06-06T04:32:38.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.810+0000 7f6e037fe700 1 --2- 172.21.15.45:0/3695942274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce924f60 0x5649ce925340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:38.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.810+0000 7f6e02ffd700 1 --2- 172.21.15.45:0/3695942274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce925880 0x5649ce91f260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6df8011790 tx=0x7f6df8013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=228126638ffaa9f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:38.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.810+0000 7f6de9ffb700 1 -- 172.21.15.45:0/3695942274 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df801e620 con 0x5649ce925880 2024-06-06T04:32:38.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.810+0000 7f6de9ffb700 1 -- 172.21.15.45:0/3695942274 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6df8010e70 con 0x5649ce925880 2024-06-06T04:32:38.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.811+0000 7f6de9ffb700 1 -- 172.21.15.45:0/3695942274 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8026c50 con 0x5649ce925880 2024-06-06T04:32:38.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.811+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3695942274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce925880 msgr2=0x5649ce91f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:38.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.811+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3695942274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce925880 0x5649ce91f260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6df8011790 tx=0x7f6df8013000 comp rx=0 tx=0).stop 2024-06-06T04:32:38.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.811+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3695942274 shutdown_connections 2024-06-06T04:32:38.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.811+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3695942274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce91f7a0 0x5649ce91fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.811+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3695942274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce924f60 0x5649ce925340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.811+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3695942274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce925880 0x5649ce91f260 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.811+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3695942274 >> 172.21.15.45:0/3695942274 conn(0x5649ce84c080 msgr2=0x5649ce91a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.812+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3695942274 shutdown_connections 2024-06-06T04:32:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.812+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3695942274 wait complete. 2024-06-06T04:32:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.812+0000 7f6e1430bb80 1 -- start start 2024-06-06T04:32:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.812+0000 7f6e1430bb80 1 Processor -- start 2024-06-06T04:32:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e1430bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce91f7a0 0x5649ce92b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e1430bb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce924f60 0x5649ce928960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e1430bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce925880 0x5649ce928ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e037fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce91f7a0 0x5649ce92b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e1430bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5649ce810fc0 con 0x5649ce925880 2024-06-06T04:32:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e1430bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5649ce86c810 con 0x5649ce91f7a0 2024-06-06T04:32:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e1430bb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5649ce92c260 con 0x5649ce924f60 2024-06-06T04:32:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e037fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce91f7a0 0x5649ce92b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:48648/0 (socket says 172.21.15.45:48648) 2024-06-06T04:32:38.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e037fe700 1 -- 172.21.15.45:0/3110319181 learned_addr learned my addr 172.21.15.45:0/3110319181 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:38.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e1430bb80 10 client.0 _ll_register_callbacks cb 0x5649ce8065b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:38.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e02ffd700 1 --2- 172.21.15.45:0/3110319181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce924f60 0x5649ce928960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:38.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e037fe700 1 -- 172.21.15.45:0/3110319181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce924f60 msgr2=0x5649ce928960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:38.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e037fe700 1 --2- 172.21.15.45:0/3110319181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce924f60 0x5649ce928960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e037fe700 1 -- 172.21.15.45:0/3110319181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce925880 msgr2=0x5649ce928ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:38.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e037fe700 1 --2- 172.21.15.45:0/3110319181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce925880 0x5649ce928ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e037fe700 1 -- 172.21.15.45:0/3110319181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649ce836750 con 0x5649ce91f7a0 2024-06-06T04:32:38.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.813+0000 7f6e02ffd700 1 --2- 172.21.15.45:0/3110319181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce924f60 0x5649ce928960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:38.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.814+0000 7f6e037fe700 1 --2- 172.21.15.45:0/3110319181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce91f7a0 0x5649ce92b660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6df4000c00 tx=0x7f6df409c1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec1f4172bf8a46ef server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:38.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.814+0000 7f6de2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-06-06T04:32:38.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.814+0000 7f6de2ffd700 1 -- 172.21.15.45:0/3110319181 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df409c730 con 0x5649ce91f7a0 2024-06-06T04:32:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.814+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3110319181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6df8010e70 con 0x5649ce91f7a0 2024-06-06T04:32:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.814+0000 7f6de2ffd700 1 -- 172.21.15.45:0/3110319181 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f6df40094b0 con 0x5649ce91f7a0 2024-06-06T04:32:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.814+0000 7f6de2ffd700 1 -- 172.21.15.45:0/3110319181 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6df40bb080 con 0x5649ce91f7a0 2024-06-06T04:32:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.814+0000 7f6e1430bb80 10 client.25196 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.814+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3110319181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5649ce936850 con 0x5649ce91f7a0 2024-06-06T04:32:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.814+0000 7f6de2ffd700 1 -- 172.21.15.45:0/3110319181 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6df40b5070 con 0x5649ce91f7a0 2024-06-06T04:32:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.815+0000 7f6de2ffd700 1 -- 172.21.15.45:0/3110319181 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6df40cdce0 con 0x5649ce91f7a0 2024-06-06T04:32:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.815+0000 7f6e1430bb80 10 client.25196 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.815+0000 7f6e1430bb80 10 client.25196 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-06T04:32:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.815+0000 7f6e1430bb80 20 client.25196 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.815+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3110319181 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x5649ce938f80 0x5649ce93b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.815+0000 7f6e1430bb80 4 client.25196 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:32:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.815+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3110319181 --> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6df8010e70 con 0x5649ce938f80 2024-06-06T04:32:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.815+0000 7f6e02ffd700 1 --2- 172.21.15.45:0/3110319181 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x5649ce938f80 0x5649ce93b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.815+0000 7f6e02ffd700 1 --2- 172.21.15.45:0/3110319181 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x5649ce938f80 0x5649ce93b3a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=812ef134dd01e6ba server_cookie=3fb004907b3807e5 in_seq=0 out_seq=0 2024-06-06T04:32:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.815+0000 7f6de2ffd700 10 client.25196 ms_handle_connect on v2:172.21.15.154:6836/1140071744 2024-06-06T04:32:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.816+0000 7f6de2ffd700 1 -- 172.21.15.45:0/3110319181 <== mds.2 v2:172.21.15.154:6836/1140071744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f6df8010e70 con 0x5649ce938f80 2024-06-06T04:32:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.816+0000 7f6de2ffd700 10 client.25196 handle_command_reply: tid=0 2024-06-06T04:32:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.816+0000 7f6e1430bb80 1 client.25196 shutdown 2024-06-06T04:32:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.816+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3110319181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce91f7a0 msgr2=0x5649ce92b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.816+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3110319181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce91f7a0 0x5649ce92b660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6df4000c00 tx=0x7f6df409c1c0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.816+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3110319181 shutdown_connections 2024-06-06T04:32:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.816+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3110319181 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x5649ce938f80 0x5649ce93b3a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.817+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3110319181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce925880 0x5649ce928ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.817+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3110319181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce91f7a0 0x5649ce92b660 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.817+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3110319181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce924f60 0x5649ce928960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.817+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3110319181 >> 172.21.15.45:0/3110319181 conn(0x5649ce84c080 msgr2=0x5649ce919700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:38.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.817+0000 7f6de2ffd700 0 client.25196 ms_handle_reset on v2:172.21.15.154:6836/1140071744 2024-06-06T04:32:38.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.817+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3110319181 shutdown_connections 2024-06-06T04:32:38.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.817+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3110319181 wait complete. 2024-06-06T04:32:38.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.817+0000 7f6e1430bb80 20 client.25196 trim_cache size 0 max 16384 2024-06-06T04:32:38.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.836+0000 7f6e1430bb80 1 Processor -- start 2024-06-06T04:32:38.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.836+0000 7f6e1430bb80 1 -- start start 2024-06-06T04:32:38.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.836+0000 7f6e1430bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce91f7a0 0x5649ce91f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.836+0000 7f6e1430bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce924f60 0x5649ce926430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.836+0000 7f6e1430bb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce925880 0x5649ce919e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.836+0000 7f6e1430bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e041b2c80 con 0x5649ce91f7a0 2024-06-06T04:32:38.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.836+0000 7f6e1430bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e041b0510 con 0x5649ce924f60 2024-06-06T04:32:38.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.836+0000 7f6e1430bb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6e041ae360 con 0x5649ce925880 2024-06-06T04:32:38.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.836+0000 7f6e037fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce91f7a0 0x5649ce91f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:38.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.836+0000 7f6e037fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce91f7a0 0x5649ce91f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48814/0 (socket says 172.21.15.45:48814) 2024-06-06T04:32:38.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.836+0000 7f6e037fe700 1 -- 172.21.15.45:0/3483392216 learned_addr learned my addr 172.21.15.45:0/3483392216 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:38.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.837+0000 7f6e02ffd700 1 --2- 172.21.15.45:0/3483392216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce924f60 0x5649ce926430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:38.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.837+0000 7f6e037fe700 1 -- 172.21.15.45:0/3483392216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce925880 msgr2=0x5649ce919e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:38.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.837+0000 7f6e03fff700 1 --2- 172.21.15.45:0/3483392216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce925880 0x5649ce919e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:38.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.837+0000 7f6e037fe700 1 --2- 172.21.15.45:0/3483392216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce925880 0x5649ce919e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.837+0000 7f6e037fe700 1 -- 172.21.15.45:0/3483392216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce924f60 msgr2=0x5649ce926430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:38.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.837+0000 7f6e037fe700 1 --2- 172.21.15.45:0/3483392216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce924f60 0x5649ce926430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.837+0000 7f6e037fe700 1 -- 172.21.15.45:0/3483392216 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649ce91b800 con 0x5649ce91f7a0 2024-06-06T04:32:38.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.837+0000 7f6e02ffd700 1 --2- 172.21.15.45:0/3483392216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce924f60 0x5649ce926430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:38.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.837+0000 7f6e037fe700 1 --2- 172.21.15.45:0/3483392216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce91f7a0 0x5649ce91f260 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6df4022590 tx=0x7f6df4005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f3af60b932cc46 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:38.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.837+0000 7f6de97fa700 1 -- 172.21.15.45:0/3483392216 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df409c730 con 0x5649ce91f7a0 2024-06-06T04:32:38.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.837+0000 7f6de97fa700 1 -- 172.21.15.45:0/3483392216 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6df40b5040 con 0x5649ce91f7a0 2024-06-06T04:32:38.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.837+0000 7f6de97fa700 1 -- 172.21.15.45:0/3483392216 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4098480 con 0x5649ce91f7a0 2024-06-06T04:32:38.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.838+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3483392216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce91f7a0 msgr2=0x5649ce91f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:38.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.838+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3483392216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce91f7a0 0x5649ce91f260 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6df4022590 tx=0x7f6df4005a90 comp rx=0 tx=0).stop 2024-06-06T04:32:38.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.838+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3483392216 shutdown_connections 2024-06-06T04:32:38.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.838+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3483392216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce925880 0x5649ce919e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.838+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3483392216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce91f7a0 0x5649ce91f260 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.838+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3483392216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce924f60 0x5649ce926430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.838+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3483392216 >> 172.21.15.45:0/3483392216 conn(0x5649ce84c080 msgr2=0x5649ce925d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:38.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.838+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3483392216 shutdown_connections 2024-06-06T04:32:38.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.839+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3483392216 wait complete. 2024-06-06T04:32:38.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.839+0000 7f6e1430bb80 1 -- start start 2024-06-06T04:32:38.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.839+0000 7f6e1430bb80 1 Processor -- start 2024-06-06T04:32:38.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.839+0000 7f6e1430bb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce91f7a0 0x5649ce931520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.839+0000 7f6e1430bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce924f60 0x5649ce931a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.839+0000 7f6e1430bb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce925880 0x5649ce93fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.839+0000 7f6e1430bb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e041ae9d0 con 0x5649ce925880 2024-06-06T04:32:38.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.839+0000 7f6e1430bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e041afe40 con 0x5649ce924f60 2024-06-06T04:32:38.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.839+0000 7f6e1430bb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6e041af0a0 con 0x5649ce91f7a0 2024-06-06T04:32:38.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.839+0000 7f6e1430bb80 10 client.0 _ll_register_callbacks cb 0x5649ce8065b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:38.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.840+0000 7f6e02ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce924f60 0x5649ce931a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:38.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.840+0000 7f6e02ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce924f60 0x5649ce931a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:48674/0 (socket says 172.21.15.45:48674) 2024-06-06T04:32:38.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.840+0000 7f6e02ffd700 1 -- 172.21.15.45:0/3082787543 learned_addr learned my addr 172.21.15.45:0/3082787543 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:38.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.840+0000 7f6e037fe700 1 --2- 172.21.15.45:0/3082787543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce91f7a0 0x5649ce931520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:38.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.840+0000 7f6e03fff700 1 --2- 172.21.15.45:0/3082787543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce925880 0x5649ce93fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:38.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.840+0000 7f6e02ffd700 1 -- 172.21.15.45:0/3082787543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce91f7a0 msgr2=0x5649ce931520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:38.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.840+0000 7f6e02ffd700 1 --2- 172.21.15.45:0/3082787543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce91f7a0 0x5649ce931520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.840+0000 7f6e02ffd700 1 -- 172.21.15.45:0/3082787543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce925880 msgr2=0x5649ce93fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:38.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.840+0000 7f6e02ffd700 1 --2- 172.21.15.45:0/3082787543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce925880 0x5649ce93fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.840+0000 7f6e02ffd700 1 -- 172.21.15.45:0/3082787543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649ce9401e0 con 0x5649ce924f60 2024-06-06T04:32:38.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.840+0000 7f6e03fff700 1 --2- 172.21.15.45:0/3082787543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce925880 0x5649ce93fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:38.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.840+0000 7f6e02ffd700 1 --2- 172.21.15.45:0/3082787543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce924f60 0x5649ce931a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6df8011760 tx=0x7f6df800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41d8ffb11392c64e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:38.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.840+0000 7f6de2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-06-06T04:32:38.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.841+0000 7f6de2ffd700 1 -- 172.21.15.45:0/3082787543 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8026e60 con 0x5649ce924f60 2024-06-06T04:32:38.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.841+0000 7f6de2ffd700 1 -- 172.21.15.45:0/3082787543 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f6df8004020 con 0x5649ce924f60 2024-06-06T04:32:38.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.841+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3082787543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6df40b5040 con 0x5649ce924f60 2024-06-06T04:32:38.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.841+0000 7f6de2ffd700 1 -- 172.21.15.45:0/3082787543 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6df8041050 con 0x5649ce924f60 2024-06-06T04:32:38.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.841+0000 7f6e1430bb80 10 client.25202 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:38.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.841+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3082787543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5649ce9a3300 con 0x5649ce924f60 2024-06-06T04:32:38.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.841+0000 7f6de2ffd700 1 -- 172.21.15.45:0/3082787543 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6df8005040 con 0x5649ce924f60 2024-06-06T04:32:38.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.841+0000 7f6de2ffd700 1 -- 172.21.15.45:0/3082787543 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6df8004830 con 0x5649ce924f60 2024-06-06T04:32:38.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.841+0000 7f6e1430bb80 10 client.25202 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:38.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.841+0000 7f6e1430bb80 10 client.25202 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-06T04:32:38.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.841+0000 7f6e1430bb80 20 client.25202 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:38.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.842+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3082787543 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x5649ce9a3780 0x5649ce9a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:38.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.842+0000 7f6e1430bb80 4 client.25202 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T04:32:38.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.842+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3082787543 --> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6df40b5040 con 0x5649ce9a3780 2024-06-06T04:32:38.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.842+0000 7f6e037fe700 1 --2- 172.21.15.45:0/3082787543 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x5649ce9a3780 0x5649ce9a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:38.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.842+0000 7f6de2ffd700 10 client.25202 ms_handle_connect on v2:172.21.15.154:6836/1140071744 2024-06-06T04:32:38.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.842+0000 7f6e037fe700 1 --2- 172.21.15.45:0/3082787543 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x5649ce9a3780 0x5649ce9a5ba0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77633df5cb668dd1 server_cookie=367886976bbb024 in_seq=0 out_seq=0 2024-06-06T04:32:38.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.842+0000 7f6de2ffd700 1 -- 172.21.15.45:0/3082787543 <== mds.2 v2:172.21.15.154:6836/1140071744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6df40b5040 con 0x5649ce9a3780 2024-06-06T04:32:38.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.842+0000 7f6de2ffd700 10 client.25202 handle_command_reply: tid=0 2024-06-06T04:32:38.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.843+0000 7f6e1430bb80 1 client.25202 shutdown 2024-06-06T04:32:38.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.843+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3082787543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce924f60 msgr2=0x5649ce931a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:38.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.843+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3082787543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce924f60 0x5649ce931a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6df8011760 tx=0x7f6df800bf10 comp rx=0 tx=0).stop 2024-06-06T04:32:38.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.843+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3082787543 shutdown_connections 2024-06-06T04:32:38.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.843+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3082787543 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x5649ce9a3780 0x5649ce9a5ba0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.843+0000 7f6de2ffd700 0 client.25202 ms_handle_reset on v2:172.21.15.154:6836/1140071744 2024-06-06T04:32:38.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.843+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3082787543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5649ce925880 0x5649ce93fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.843+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3082787543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649ce91f7a0 0x5649ce931520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.843+0000 7f6e1430bb80 1 --2- 172.21.15.45:0/3082787543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649ce924f60 0x5649ce931a60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.843+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3082787543 >> 172.21.15.45:0/3082787543 conn(0x5649ce84c080 msgr2=0x5649ce92a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:38.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.843+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3082787543 shutdown_connections 2024-06-06T04:32:38.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.843+0000 7f6e1430bb80 1 -- 172.21.15.45:0/3082787543 wait complete. 2024-06-06T04:32:38.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.843+0000 7f6e1430bb80 20 client.25202 trim_cache size 0 max 16384 2024-06-06T04:32:38.847 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-06-06T04:32:38.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.844+0000 7f6e09a88700 1 -- 172.21.15.45:0/299970659 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6dec0a2c20 msgr2=0x7f6dec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:38.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.844+0000 7f6e09a88700 1 --2- 172.21.15.45:0/299970659 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6dec0a2c20 0x7f6dec0a50e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6e04079f30 tx=0x7f6df800b410 comp rx=0 tx=0).stop 2024-06-06T04:32:38.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.844+0000 7f6e09a88700 1 -- 172.21.15.45:0/299970659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e04102520 msgr2=0x7f6e0419ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:38.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.844+0000 7f6e09a88700 1 --2- 172.21.15.45:0/299970659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e04102520 0x7f6e0419ad30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6df4006010 tx=0x7f6df4009f80 comp rx=0 tx=0).stop 2024-06-06T04:32:38.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.844+0000 7f6e09a88700 1 -- 172.21.15.45:0/299970659 shutdown_connections 2024-06-06T04:32:38.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.844+0000 7f6e09a88700 1 --2- 172.21.15.45:0/299970659 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f6dec0a2c20 0x7f6dec0a50e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.844+0000 7f6e09a88700 1 --2- 172.21.15.45:0/299970659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e041085c0 0x7f6e0419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.844+0000 7f6e09a88700 1 --2- 172.21.15.45:0/299970659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e04102520 0x7f6e0419ad30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.844+0000 7f6e09a88700 1 --2- 172.21.15.45:0/299970659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e04102ee0 0x7f6e0419b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:38.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.844+0000 7f6e09a88700 1 -- 172.21.15.45:0/299970659 >> 172.21.15.45:0/299970659 conn(0x7f6e04079680 msgr2=0x7f6e0410d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:38.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.844+0000 7f6e09a88700 1 -- 172.21.15.45:0/299970659 shutdown_connections 2024-06-06T04:32:38.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:38.844+0000 7f6e09a88700 1 -- 172.21.15.45:0/299970659 wait complete. 2024-06-06T04:32:39.678 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-06T04:32:40.114 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:32:40.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:40 smithi045 ceph-mon[32299]: pgmap v467: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-06-06T04:32:40.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:40 smithi086 ceph-mon[36053]: pgmap v467: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-06-06T04:32:40.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:40 smithi154 ceph-mon[36428]: pgmap v467: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-06-06T04:32:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.083+0000 7fbfbe246700 1 -- 172.21.15.45:0/3873527386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfb8101d50 msgr2=0x7fbfb810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.083+0000 7fbfbe246700 1 --2- 172.21.15.45:0/3873527386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfb8101d50 0x7fbfb810e240 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000b0a0 tx=0x7fbfb000b3b0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.083+0000 7fbfbe246700 1 -- 172.21.15.45:0/3873527386 shutdown_connections 2024-06-06T04:32:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.083+0000 7fbfbe246700 1 --2- 172.21.15.45:0/3873527386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfb8101d50 0x7fbfb810e240 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.083+0000 7fbfbe246700 1 --2- 172.21.15.45:0/3873527386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb8107400 0x7fbfb81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.083+0000 7fbfbe246700 1 --2- 172.21.15.45:0/3873527386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb8101390 0x7fbfb8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.083+0000 7fbfbe246700 1 -- 172.21.15.45:0/3873527386 >> 172.21.15.45:0/3873527386 conn(0x7fbfb8078340 msgr2=0x7fbfb8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.083+0000 7fbfbe246700 1 -- 172.21.15.45:0/3873527386 shutdown_connections 2024-06-06T04:32:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.083+0000 7fbfbe246700 1 -- 172.21.15.45:0/3873527386 wait complete. 2024-06-06T04:32:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.084+0000 7fbfbe246700 1 Processor -- start 2024-06-06T04:32:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.084+0000 7fbfbe246700 1 -- start start 2024-06-06T04:32:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.085+0000 7fbfbe246700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfb8101390 0x7fbfb8073fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.085+0000 7fbfbe246700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb8101d50 0x7fbfb806bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.085+0000 7fbfbe246700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb8107400 0x7fbfb806c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.085+0000 7fbfbe246700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81103e0 con 0x7fbfb8101390 2024-06-06T04:32:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.085+0000 7fbfbe246700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb8110260 con 0x7fbfb8101d50 2024-06-06T04:32:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.085+0000 7fbfbe246700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb8110560 con 0x7fbfb8107400 2024-06-06T04:32:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.085+0000 7fbfb6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb8101d50 0x7fbfb806bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.085+0000 7fbfb6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb8101d50 0x7fbfb806bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:48696/0 (socket says 172.21.15.45:48696) 2024-06-06T04:32:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.085+0000 7fbfb6ffd700 1 -- 172.21.15.45:0/3092654949 learned_addr learned my addr 172.21.15.45:0/3092654949 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.085+0000 7fbfb7fff700 1 --2- 172.21.15.45:0/3092654949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb8107400 0x7fbfb806c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.085+0000 7fbfb77fe700 1 --2- 172.21.15.45:0/3092654949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfb8101390 0x7fbfb8073fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.086+0000 7fbfb6ffd700 1 -- 172.21.15.45:0/3092654949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb8107400 msgr2=0x7fbfb806c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.086+0000 7fbfb6ffd700 1 --2- 172.21.15.45:0/3092654949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb8107400 0x7fbfb806c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.086+0000 7fbfb6ffd700 1 -- 172.21.15.45:0/3092654949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfb8101390 msgr2=0x7fbfb8073fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.086+0000 7fbfb6ffd700 1 --2- 172.21.15.45:0/3092654949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfb8101390 0x7fbfb8073fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.086+0000 7fbfb6ffd700 1 -- 172.21.15.45:0/3092654949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb806cd30 con 0x7fbfb8101d50 2024-06-06T04:32:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.086+0000 7fbfb77fe700 1 --2- 172.21.15.45:0/3092654949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfb8101390 0x7fbfb8073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.086+0000 7fbfb6ffd700 1 --2- 172.21.15.45:0/3092654949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb8101d50 0x7fbfb806bf80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009a20 tx=0x7fbfac009d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.086+0000 7fbfb4ff9700 1 -- 172.21.15.45:0/3092654949 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac01b070 con 0x7fbfb8101d50 2024-06-06T04:32:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.086+0000 7fbfb4ff9700 1 -- 172.21.15.45:0/3092654949 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfac009e50 con 0x7fbfb8101d50 2024-06-06T04:32:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.086+0000 7fbfb4ff9700 1 -- 172.21.15.45:0/3092654949 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac016d70 con 0x7fbfb8101d50 2024-06-06T04:32:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.086+0000 7fbfbe246700 1 -- 172.21.15.45:0/3092654949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb81ab6a0 con 0x7fbfb8101d50 2024-06-06T04:32:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.086+0000 7fbfbe246700 1 -- 172.21.15.45:0/3092654949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb81abb00 con 0x7fbfb8101d50 2024-06-06T04:32:42.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.089+0000 7fbfb4ff9700 1 -- 172.21.15.45:0/3092654949 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbfac01e070 con 0x7fbfb8101d50 2024-06-06T04:32:42.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.090+0000 7fbfb4ff9700 1 --2- 172.21.15.45:0/3092654949 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbfa00a2ca0 0x7fbfa00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.090+0000 7fbfb4ff9700 1 -- 172.21.15.45:0/3092654949 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fbfac0cf5e0 con 0x7fbfb8101d50 2024-06-06T04:32:42.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.090+0000 7fbfb77fe700 1 --2- 172.21.15.45:0/3092654949 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbfa00a2ca0 0x7fbfa00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.090+0000 7fbfb77fe700 1 --2- 172.21.15.45:0/3092654949 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbfa00a2ca0 0x7fbfa00a5160 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005e10 tx=0x7fbfa8005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:42.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.091+0000 7fbfc8ac9b80 1 Processor -- start 2024-06-06T04:32:42.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfc8ac9b80 1 -- start start 2024-06-06T04:32:42.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfc8ac9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be8f60 0x55d1b6be9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfc8ac9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be9880 0x55d1b6be3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfc8ac9b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be37a0 0x55d1b6be3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfc8ac9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d1b6ad5f60 con 0x55d1b6be9880 2024-06-06T04:32:42.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfc8ac9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d1b6b34850 con 0x55d1b6be8f60 2024-06-06T04:32:42.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfc8ac9b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d1b6a9c460 con 0x55d1b6be37a0 2024-06-06T04:32:42.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfb6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be9880 0x55d1b6be3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfb6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be9880 0x55d1b6be3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48834/0 (socket says 172.21.15.45:48834) 2024-06-06T04:32:42.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfb6ffd700 1 -- 172.21.15.45:0/207064547 learned_addr learned my addr 172.21.15.45:0/207064547 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:42.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfb77fe700 1 --2- 172.21.15.45:0/207064547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be8f60 0x55d1b6be9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfb6ffd700 1 -- 172.21.15.45:0/207064547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be37a0 msgr2=0x55d1b6be3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:42.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfb7fff700 1 --2- 172.21.15.45:0/207064547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be37a0 0x55d1b6be3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfb6ffd700 1 --2- 172.21.15.45:0/207064547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be37a0 0x55d1b6be3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfb6ffd700 1 -- 172.21.15.45:0/207064547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be8f60 msgr2=0x55d1b6be9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:42.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfb6ffd700 1 --2- 172.21.15.45:0/207064547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be8f60 0x55d1b6be9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.092+0000 7fbfb6ffd700 1 -- 172.21.15.45:0/207064547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b6a838d0 con 0x55d1b6be9880 2024-06-06T04:32:42.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.093+0000 7fbfb7fff700 1 --2- 172.21.15.45:0/207064547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be37a0 0x55d1b6be3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:42.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.093+0000 7fbfb6ffd700 1 --2- 172.21.15.45:0/207064547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be9880 0x55d1b6be3260 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00c550 tx=0x7fbfac09b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed25926546c9c595 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:42.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.093+0000 7fbf9dffb700 1 -- 172.21.15.45:0/207064547 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac0b3020 con 0x55d1b6be9880 2024-06-06T04:32:42.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.093+0000 7fbf9dffb700 1 -- 172.21.15.45:0/207064547 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfac09ba60 con 0x55d1b6be9880 2024-06-06T04:32:42.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.093+0000 7fbf9dffb700 1 -- 172.21.15.45:0/207064547 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac0a4240 con 0x55d1b6be9880 2024-06-06T04:32:42.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.094+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/207064547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be9880 msgr2=0x55d1b6be3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:42.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.094+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/207064547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be9880 0x55d1b6be3260 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00c550 tx=0x7fbfac09b490 comp rx=0 tx=0).stop 2024-06-06T04:32:42.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.094+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/207064547 shutdown_connections 2024-06-06T04:32:42.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.094+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/207064547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be37a0 0x55d1b6be3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.094+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/207064547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be8f60 0x55d1b6be9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.094+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/207064547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be9880 0x55d1b6be3260 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.094+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/207064547 >> 172.21.15.45:0/207064547 conn(0x55d1b6b10080 msgr2=0x55d1b6bde220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:42.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.095+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/207064547 shutdown_connections 2024-06-06T04:32:42.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.095+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/207064547 wait complete. 2024-06-06T04:32:42.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.095+0000 7fbfc8ac9b80 1 -- start start 2024-06-06T04:32:42.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.096+0000 7fbfc8ac9b80 1 Processor -- start 2024-06-06T04:32:42.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.096+0000 7fbfc8ac9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be37a0 0x55d1b6bef6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.096+0000 7fbfc8ac9b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be8f60 0x55d1b6beca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.096+0000 7fbfc8ac9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be9880 0x55d1b6becf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.096+0000 7fbfc8ac9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d1b6ad4fc0 con 0x55d1b6be9880 2024-06-06T04:32:42.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.096+0000 7fbfc8ac9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d1b6b30810 con 0x55d1b6be37a0 2024-06-06T04:32:42.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.096+0000 7fbfc8ac9b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d1b6bf0200 con 0x55d1b6be8f60 2024-06-06T04:32:42.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.096+0000 7fbfb77fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be37a0 0x55d1b6bef6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.096+0000 7fbfb77fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be37a0 0x55d1b6bef6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:48726/0 (socket says 172.21.15.45:48726) 2024-06-06T04:32:42.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.096+0000 7fbfb77fe700 1 -- 172.21.15.45:0/2614500508 learned_addr learned my addr 172.21.15.45:0/2614500508 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:42.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.096+0000 7fbfb7fff700 1 --2- 172.21.15.45:0/2614500508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be9880 0x55d1b6becf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.097+0000 7fbfb7fff700 1 -- 172.21.15.45:0/2614500508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be8f60 msgr2=0x55d1b6beca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:42.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.097+0000 7fbfb7fff700 1 --2- 172.21.15.45:0/2614500508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be8f60 0x55d1b6beca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.097+0000 7fbfb7fff700 1 -- 172.21.15.45:0/2614500508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be37a0 msgr2=0x55d1b6bef6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:42.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.097+0000 7fbfb7fff700 1 --2- 172.21.15.45:0/2614500508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be37a0 0x55d1b6bef6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.097+0000 7fbfb7fff700 1 -- 172.21.15.45:0/2614500508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b6afa750 con 0x55d1b6be9880 2024-06-06T04:32:42.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.097+0000 7fbfc8ac9b80 10 client.0 _ll_register_callbacks cb 0x55d1b6aca5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:42.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.097+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/2614500508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfac09ba60 con 0x55d1b6be9880 2024-06-06T04:32:42.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.097+0000 7fbfb7fff700 1 --2- 172.21.15.45:0/2614500508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be9880 0x55d1b6becf40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009fd0 tx=0x7fbfb000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6ee5150348a5e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:42.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.097+0000 7fbf96ffd700 10 client.15564 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:42.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.097+0000 7fbf96ffd700 1 -- 172.21.15.45:0/2614500508 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0010bf0 con 0x55d1b6be9880 2024-06-06T04:32:42.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.097+0000 7fbf96ffd700 1 -- 172.21.15.45:0/2614500508 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fbfb0010d90 con 0x55d1b6be9880 2024-06-06T04:32:42.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.098+0000 7fbf96ffd700 1 -- 172.21.15.45:0/2614500508 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbfb0010130 con 0x55d1b6be9880 2024-06-06T04:32:42.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.098+0000 7fbfc8ac9b80 10 client.15564 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:42.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.098+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/2614500508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55d1b6c02ea0 con 0x55d1b6be9880 2024-06-06T04:32:42.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.098+0000 7fbf96ffd700 1 -- 172.21.15.45:0/2614500508 <== mon.0 v2:172.21.15.45:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbfb00102d0 con 0x55d1b6be9880 2024-06-06T04:32:42.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.098+0000 7fbfc8ac9b80 10 client.15564 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:42.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.098+0000 7fbfc8ac9b80 10 client.15564 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T04:32:42.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.098+0000 7fbfc8ac9b80 20 client.15564 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:42.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.098+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/2614500508 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55d1b6c053d0 0x55d1b6c07860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.098+0000 7fbfc8ac9b80 4 client.15564 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:32:42.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.098+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/2614500508 --> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1b6b41d50 con 0x55d1b6c053d0 2024-06-06T04:32:42.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.099+0000 7fbfb77fe700 1 --2- 172.21.15.45:0/2614500508 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55d1b6c053d0 0x55d1b6c07860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.099+0000 7fbfb77fe700 1 --2- 172.21.15.45:0/2614500508 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55d1b6c053d0 0x55d1b6c07860 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ed586b2ada8f736 server_cookie=43fb178ca3a30071 in_seq=0 out_seq=0 2024-06-06T04:32:42.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.099+0000 7fbf96ffd700 10 client.15564 ms_handle_connect on v2:172.21.15.154:6838/982197202 2024-06-06T04:32:42.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.100+0000 7fbf96ffd700 1 -- 172.21.15.45:0/2614500508 <== mds.1 v2:172.21.15.154:6838/982197202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d1b6b41d50 con 0x55d1b6c053d0 2024-06-06T04:32:42.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.100+0000 7fbf96ffd700 10 client.15564 handle_command_reply: tid=0 2024-06-06T04:32:42.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.100+0000 7fbfc8ac9b80 1 client.15564 shutdown 2024-06-06T04:32:42.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.100+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/2614500508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be9880 msgr2=0x55d1b6becf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:42.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.100+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/2614500508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be9880 0x55d1b6becf40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009fd0 tx=0x7fbfb000cda0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.100+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/2614500508 shutdown_connections 2024-06-06T04:32:42.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.101+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/2614500508 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55d1b6c053d0 0x55d1b6c07860 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.101+0000 7fbf96ffd700 0 client.15564 ms_handle_reset on v2:172.21.15.154:6838/982197202 2024-06-06T04:32:42.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.101+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/2614500508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be9880 0x55d1b6becf40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.101+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/2614500508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be37a0 0x55d1b6bef6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.101+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/2614500508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be8f60 0x55d1b6beca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.101+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/2614500508 >> 172.21.15.45:0/2614500508 conn(0x55d1b6b10080 msgr2=0x55d1b6bdd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:42.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.101+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/2614500508 shutdown_connections 2024-06-06T04:32:42.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.101+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/2614500508 wait complete. 2024-06-06T04:32:42.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.101+0000 7fbfc8ac9b80 20 client.15564 trim_cache size 0 max 16384 2024-06-06T04:32:42.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.117+0000 7fbfc8ac9b80 1 Processor -- start 2024-06-06T04:32:42.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.117+0000 7fbfc8ac9b80 1 -- start start 2024-06-06T04:32:42.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.117+0000 7fbfc8ac9b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be37a0 0x55d1b6bef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.117+0000 7fbfc8ac9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be8f60 0x55d1b6bdde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.117+0000 7fbfc8ac9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be9880 0x55d1b6bde3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.117+0000 7fbfc8ac9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81b7150 con 0x55d1b6be8f60 2024-06-06T04:32:42.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.117+0000 7fbfc8ac9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81b49e0 con 0x55d1b6be9880 2024-06-06T04:32:42.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.117+0000 7fbfc8ac9b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81b2830 con 0x55d1b6be37a0 2024-06-06T04:32:42.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.118+0000 7fbfb7fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be9880 0x55d1b6bde3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.118+0000 7fbfb7fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be9880 0x55d1b6bde3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:48742/0 (socket says 172.21.15.45:48742) 2024-06-06T04:32:42.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.118+0000 7fbfb7fff700 1 -- 172.21.15.45:0/890689258 learned_addr learned my addr 172.21.15.45:0/890689258 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:42.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.118+0000 7fbfb77fe700 1 --2- 172.21.15.45:0/890689258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be37a0 0x55d1b6bef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.118+0000 7fbfb6ffd700 1 --2- 172.21.15.45:0/890689258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be8f60 0x55d1b6bdde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.118+0000 7fbfb7fff700 1 -- 172.21.15.45:0/890689258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be37a0 msgr2=0x55d1b6bef1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:42.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.118+0000 7fbfb7fff700 1 --2- 172.21.15.45:0/890689258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be37a0 0x55d1b6bef1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.118+0000 7fbfb7fff700 1 -- 172.21.15.45:0/890689258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be8f60 msgr2=0x55d1b6bdde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:42.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.118+0000 7fbfb7fff700 1 --2- 172.21.15.45:0/890689258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be8f60 0x55d1b6bdde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.118+0000 7fbfb7fff700 1 -- 172.21.15.45:0/890689258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b6be4120 con 0x55d1b6be9880 2024-06-06T04:32:42.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.118+0000 7fbfb6ffd700 1 --2- 172.21.15.45:0/890689258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be8f60 0x55d1b6bdde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:42.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.119+0000 7fbfb7fff700 1 --2- 172.21.15.45:0/890689258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be9880 0x55d1b6bde3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000b780 tx=0x7fbfb001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d918171a14e37f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:42.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.119+0000 7fbf9d7fa700 1 -- 172.21.15.45:0/890689258 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0010350 con 0x55d1b6be9880 2024-06-06T04:32:42.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.119+0000 7fbf9d7fa700 1 -- 172.21.15.45:0/890689258 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfac09ba60 con 0x55d1b6be9880 2024-06-06T04:32:42.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.119+0000 7fbf9d7fa700 1 -- 172.21.15.45:0/890689258 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0007d30 con 0x55d1b6be9880 2024-06-06T04:32:42.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.120+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/890689258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be9880 msgr2=0x55d1b6bde3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:42.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.120+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/890689258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be9880 0x55d1b6bde3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000b780 tx=0x7fbfb001f000 comp rx=0 tx=0).stop 2024-06-06T04:32:42.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.120+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/890689258 shutdown_connections 2024-06-06T04:32:42.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.120+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/890689258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be9880 0x55d1b6bde3c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.120+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/890689258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be37a0 0x55d1b6bef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.120+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/890689258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be8f60 0x55d1b6bdde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.120+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/890689258 >> 172.21.15.45:0/890689258 conn(0x55d1b6b10080 msgr2=0x55d1b6beba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:42.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.120+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/890689258 shutdown_connections 2024-06-06T04:32:42.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.120+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/890689258 wait complete. 2024-06-06T04:32:42.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.120+0000 7fbfc8ac9b80 1 -- start start 2024-06-06T04:32:42.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.121+0000 7fbfc8ac9b80 1 Processor -- start 2024-06-06T04:32:42.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.121+0000 7fbfc8ac9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be37a0 0x55d1b6bfdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.121+0000 7fbfc8ac9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be8f60 0x55d1b6bfe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.121+0000 7fbfc8ac9b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be9880 0x55d1b6bf8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.121+0000 7fbfc8ac9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81b2ea0 con 0x55d1b6be37a0 2024-06-06T04:32:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.121+0000 7fbfc8ac9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81b4310 con 0x55d1b6be8f60 2024-06-06T04:32:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.121+0000 7fbfc8ac9b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81b3570 con 0x55d1b6be9880 2024-06-06T04:32:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.121+0000 7fbfc8ac9b80 10 client.0 _ll_register_callbacks cb 0x55d1b6aca5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.121+0000 7fbfb77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be37a0 0x55d1b6bfdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.121+0000 7fbfb77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be37a0 0x55d1b6bfdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48860/0 (socket says 172.21.15.45:48860) 2024-06-06T04:32:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.121+0000 7fbfb77fe700 1 -- 172.21.15.45:0/1025440248 learned_addr learned my addr 172.21.15.45:0/1025440248 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.122+0000 7fbfb6ffd700 1 --2- 172.21.15.45:0/1025440248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be8f60 0x55d1b6bfe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.122+0000 7fbfb7fff700 1 --2- 172.21.15.45:0/1025440248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be9880 0x55d1b6bf8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.122+0000 7fbfb77fe700 1 -- 172.21.15.45:0/1025440248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be9880 msgr2=0x55d1b6bf8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.122+0000 7fbfb77fe700 1 --2- 172.21.15.45:0/1025440248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be9880 0x55d1b6bf8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.122+0000 7fbfb77fe700 1 -- 172.21.15.45:0/1025440248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be8f60 msgr2=0x55d1b6bfe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.122+0000 7fbfb77fe700 1 --2- 172.21.15.45:0/1025440248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be8f60 0x55d1b6bfe260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.122+0000 7fbfb77fe700 1 -- 172.21.15.45:0/1025440248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b6bfe7a0 con 0x55d1b6be37a0 2024-06-06T04:32:42.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.122+0000 7fbfb77fe700 1 --2- 172.21.15.45:0/1025440248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be37a0 0x55d1b6bfdd20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005f90 tx=0x7fbfa80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbe7dc7276edafa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:42.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.122+0000 7fbf96ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:42.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.122+0000 7fbf96ffd700 1 -- 172.21.15.45:0/1025440248 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8003a90 con 0x55d1b6be37a0 2024-06-06T04:32:42.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.122+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/1025440248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfac09ba60 con 0x55d1b6be37a0 2024-06-06T04:32:42.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.122+0000 7fbf96ffd700 1 -- 172.21.15.45:0/1025440248 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fbfa8003c30 con 0x55d1b6be37a0 2024-06-06T04:32:42.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.122+0000 7fbf96ffd700 1 -- 172.21.15.45:0/1025440248 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbfa803a050 con 0x55d1b6be37a0 2024-06-06T04:32:42.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.123+0000 7fbfc8ac9b80 10 client.15567 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:42.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.123+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/1025440248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d1b6c790d0 con 0x55d1b6be37a0 2024-06-06T04:32:42.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.123+0000 7fbf96ffd700 1 -- 172.21.15.45:0/1025440248 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfac09ba60 con 0x55d1b6be37a0 2024-06-06T04:32:42.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.123+0000 7fbf96ffd700 1 -- 172.21.15.45:0/1025440248 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbfa8032420 con 0x55d1b6be37a0 2024-06-06T04:32:42.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.123+0000 7fbfc8ac9b80 10 client.15567 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:42.129 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-06-06T04:32:42.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.123+0000 7fbfc8ac9b80 10 client.15567 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T04:32:42.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.123+0000 7fbfc8ac9b80 20 client.15567 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:42.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.123+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/1025440248 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55d1b6c053d0 0x55d1b6c7b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:42.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.123+0000 7fbfc8ac9b80 4 client.15567 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T04:32:42.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.123+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/1025440248 --> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1b6b3c2d0 con 0x55d1b6c053d0 2024-06-06T04:32:42.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.124+0000 7fbfb6ffd700 1 --2- 172.21.15.45:0/1025440248 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55d1b6c053d0 0x55d1b6c7b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:42.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.124+0000 7fbf96ffd700 10 client.15567 ms_handle_connect on v2:172.21.15.154:6838/982197202 2024-06-06T04:32:42.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.124+0000 7fbfb6ffd700 1 --2- 172.21.15.45:0/1025440248 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55d1b6c053d0 0x55d1b6c7b700 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c74ff52787a5a5f server_cookie=de6143dc0dda22f in_seq=0 out_seq=0 2024-06-06T04:32:42.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.124+0000 7fbf96ffd700 1 -- 172.21.15.45:0/1025440248 <== mds.1 v2:172.21.15.154:6838/982197202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1b6b3c2d0 con 0x55d1b6c053d0 2024-06-06T04:32:42.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.124+0000 7fbf96ffd700 10 client.15567 handle_command_reply: tid=0 2024-06-06T04:32:42.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.124+0000 7fbfc8ac9b80 1 client.15567 shutdown 2024-06-06T04:32:42.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.125+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/1025440248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be37a0 msgr2=0x55d1b6bfdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:42.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.125+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/1025440248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be37a0 0x55d1b6bfdd20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005f90 tx=0x7fbfa80047c0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.125+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/1025440248 shutdown_connections 2024-06-06T04:32:42.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.125+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/1025440248 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55d1b6c053d0 0x55d1b6c7b700 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.125+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/1025440248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b6be9880 0x55d1b6bf8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.125+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/1025440248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1b6be37a0 0x55d1b6bfdd20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.125+0000 7fbfc8ac9b80 1 --2- 172.21.15.45:0/1025440248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1b6be8f60 0x55d1b6bfe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.125+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/1025440248 >> 172.21.15.45:0/1025440248 conn(0x55d1b6b10080 msgr2=0x55d1b6be7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.125+0000 7fbf96ffd700 0 client.15567 ms_handle_reset on v2:172.21.15.154:6838/982197202 2024-06-06T04:32:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.125+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/1025440248 shutdown_connections 2024-06-06T04:32:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.125+0000 7fbfc8ac9b80 1 -- 172.21.15.45:0/1025440248 wait complete. 2024-06-06T04:32:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.125+0000 7fbfc8ac9b80 20 client.15567 trim_cache size 0 max 16384 2024-06-06T04:32:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.126+0000 7fbfbe246700 1 -- 172.21.15.45:0/3092654949 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbfa00a2ca0 msgr2=0x7fbfa00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.126+0000 7fbfbe246700 1 --2- 172.21.15.45:0/3092654949 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbfa00a2ca0 0x7fbfa00a5160 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005e10 tx=0x7fbfa8005d60 comp rx=0 tx=0).stop 2024-06-06T04:32:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.126+0000 7fbfbe246700 1 -- 172.21.15.45:0/3092654949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb8101d50 msgr2=0x7fbfb806bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.126+0000 7fbfbe246700 1 --2- 172.21.15.45:0/3092654949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb8101d50 0x7fbfb806bf80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009a20 tx=0x7fbfac009d30 comp rx=0 tx=0).stop 2024-06-06T04:32:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.126+0000 7fbfbe246700 1 -- 172.21.15.45:0/3092654949 shutdown_connections 2024-06-06T04:32:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.126+0000 7fbfbe246700 1 --2- 172.21.15.45:0/3092654949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb8107400 0x7fbfb806c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.126+0000 7fbfbe246700 1 --2- 172.21.15.45:0/3092654949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfb8101390 0x7fbfb8073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.126+0000 7fbfbe246700 1 --2- 172.21.15.45:0/3092654949 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fbfa00a2ca0 0x7fbfa00a5160 secure :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005e10 tx=0x7fbfa8005d60 comp rx=0 tx=0).stop 2024-06-06T04:32:42.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.126+0000 7fbfbe246700 1 --2- 172.21.15.45:0/3092654949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb8101d50 0x7fbfb806bf80 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009a20 tx=0x7fbfac009d30 comp rx=0 tx=0).stop 2024-06-06T04:32:42.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.126+0000 7fbfbe246700 1 -- 172.21.15.45:0/3092654949 >> 172.21.15.45:0/3092654949 conn(0x7fbfb8078340 msgr2=0x7fbfb80fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:42.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.126+0000 7fbfbe246700 1 -- 172.21.15.45:0/3092654949 shutdown_connections 2024-06-06T04:32:42.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:42.126+0000 7fbfbe246700 1 -- 172.21.15.45:0/3092654949 wait complete. 2024-06-06T04:32:42.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:42 smithi045 ceph-mon[32299]: pgmap v468: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-06T04:32:42.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:42 smithi086 ceph-mon[36053]: pgmap v468: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-06T04:32:42.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:42 smithi154 ceph-mon[36428]: pgmap v468: 129 pgs: 129 active+clean; 667 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-06T04:32:42.881 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-06T04:32:43.159 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:32:44.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:44 smithi086 ceph-mon[36053]: pgmap v469: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-06-06T04:32:44.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:44 smithi154 ceph-mon[36428]: pgmap v469: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-06-06T04:32:44.452 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:44 smithi045 ceph-mon[32299]: pgmap v469: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-06-06T04:32:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.102+0000 7f43626c6700 1 -- 172.21.15.45:0/3110568289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f435c102ed0 msgr2=0x7f435c10f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.102+0000 7f43626c6700 1 --2- 172.21.15.45:0/3110568289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f435c102ed0 0x7f435c10f360 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f435400b0a0 tx=0x7f435400b3b0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.102+0000 7f43626c6700 1 -- 172.21.15.45:0/3110568289 shutdown_connections 2024-06-06T04:32:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.102+0000 7f43626c6700 1 --2- 172.21.15.45:0/3110568289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f435c102ed0 0x7f435c10f360 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.102+0000 7f43626c6700 1 --2- 172.21.15.45:0/3110568289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c108520 0x7f435c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.102+0000 7f43626c6700 1 --2- 172.21.15.45:0/3110568289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f435c102510 0x7f435c102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.102+0000 7f43626c6700 1 -- 172.21.15.45:0/3110568289 >> 172.21.15.45:0/3110568289 conn(0x7f435c079560 msgr2=0x7f435c079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.102+0000 7f43626c6700 1 -- 172.21.15.45:0/3110568289 shutdown_connections 2024-06-06T04:32:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.102+0000 7f43626c6700 1 -- 172.21.15.45:0/3110568289 wait complete. 2024-06-06T04:32:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.103+0000 7f43626c6700 1 Processor -- start 2024-06-06T04:32:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.103+0000 7f43626c6700 1 -- start start 2024-06-06T04:32:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.103+0000 7f43626c6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f435c102510 0x7f435c19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.103+0000 7f43626c6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f435c102ed0 0x7f435c19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.103+0000 7f43626c6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c108520 0x7f435c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.103+0000 7f43626c6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1126c0 con 0x7f435c102ed0 2024-06-06T04:32:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.103+0000 7f43626c6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f435c112540 con 0x7f435c102510 2024-06-06T04:32:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.103+0000 7f43626c6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f435c112840 con 0x7f435c108520 2024-06-06T04:32:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.103+0000 7f435b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f435c102ed0 0x7f435c19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.103+0000 7f435b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f435c102ed0 0x7f435c19eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48874/0 (socket says 172.21.15.45:48874) 2024-06-06T04:32:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.103+0000 7f435b7fe700 1 -- 172.21.15.45:0/1901421539 learned_addr learned my addr 172.21.15.45:0/1901421539 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.104+0000 7f435bfff700 1 --2- 172.21.15.45:0/1901421539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f435c102510 0x7f435c19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.104+0000 7f4360c63700 1 --2- 172.21.15.45:0/1901421539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c108520 0x7f435c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.104+0000 7f435bfff700 1 -- 172.21.15.45:0/1901421539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c108520 msgr2=0x7f435c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.104+0000 7f435bfff700 1 --2- 172.21.15.45:0/1901421539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c108520 0x7f435c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.104+0000 7f435bfff700 1 -- 172.21.15.45:0/1901421539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f435c102ed0 msgr2=0x7f435c19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.104+0000 7f435bfff700 1 --2- 172.21.15.45:0/1901421539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f435c102ed0 0x7f435c19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.104+0000 7f435bfff700 1 -- 172.21.15.45:0/1901421539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c199080 con 0x7f435c102510 2024-06-06T04:32:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.104+0000 7f435b7fe700 1 --2- 172.21.15.45:0/1901421539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f435c102ed0 0x7f435c19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.104+0000 7f4360c63700 1 --2- 172.21.15.45:0/1901421539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c108520 0x7f435c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.104+0000 7f435bfff700 1 --2- 172.21.15.45:0/1901421539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f435c102510 0x7f435c19e760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f434c00d750 tx=0x7f434c00da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.104+0000 7f43597fa700 1 -- 172.21.15.45:0/1901421539 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c016070 con 0x7f435c102510 2024-06-06T04:32:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.104+0000 7f43597fa700 1 -- 172.21.15.45:0/1901421539 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f434c005020 con 0x7f435c102510 2024-06-06T04:32:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.104+0000 7f43626c6700 1 -- 172.21.15.45:0/1901421539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435c199370 con 0x7f435c102510 2024-06-06T04:32:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.104+0000 7f43597fa700 1 -- 172.21.15.45:0/1901421539 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c00f9f0 con 0x7f435c102510 2024-06-06T04:32:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.104+0000 7f43626c6700 1 -- 172.21.15.45:0/1901421539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435c1a5630 con 0x7f435c102510 2024-06-06T04:32:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.107+0000 7f43597fa700 1 -- 172.21.15.45:0/1901421539 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f434c0051d0 con 0x7f435c102510 2024-06-06T04:32:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.107+0000 7f43597fa700 1 --2- 172.21.15.45:0/1901421539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43480a2c20 0x7f43480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.108+0000 7f43597fa700 1 -- 172.21.15.45:0/1901421539 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f434c0ced60 con 0x7f435c102510 2024-06-06T04:32:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.108+0000 7f435b7fe700 1 --2- 172.21.15.45:0/1901421539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43480a2c20 0x7f43480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.108+0000 7f435b7fe700 1 --2- 172.21.15.45:0/1901421539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43480a2c20 0x7f43480a50e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f435c079af0 tx=0x7f434400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.109+0000 7f436cf49b80 1 Processor -- start 2024-06-06T04:32:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f436cf49b80 1 -- start start 2024-06-06T04:32:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f436cf49b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c99df60 0x55ef3c99e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f436cf49b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99e880 0x55ef3c998260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f436cf49b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c9987a0 0x55ef3c998bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f436cf49b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ef3c88af60 con 0x55ef3c99df60 2024-06-06T04:32:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f436cf49b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef3c8e9850 con 0x55ef3c9987a0 2024-06-06T04:32:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f436cf49b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ef3c851460 con 0x55ef3c99e880 2024-06-06T04:32:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f435bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c99df60 0x55ef3c99e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f435bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c99df60 0x55ef3c99e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48878/0 (socket says 172.21.15.45:48878) 2024-06-06T04:32:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f435bfff700 1 -- 172.21.15.45:0/2197551511 learned_addr learned my addr 172.21.15.45:0/2197551511 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f435b7fe700 1 --2- 172.21.15.45:0/2197551511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99e880 0x55ef3c998260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f4360c63700 1 --2- 172.21.15.45:0/2197551511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c9987a0 0x55ef3c998bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f435bfff700 1 -- 172.21.15.45:0/2197551511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99e880 msgr2=0x55ef3c998260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f435bfff700 1 --2- 172.21.15.45:0/2197551511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99e880 0x55ef3c998260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f435bfff700 1 -- 172.21.15.45:0/2197551511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c9987a0 msgr2=0x55ef3c998bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f435bfff700 1 --2- 172.21.15.45:0/2197551511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c9987a0 0x55ef3c998bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f435bfff700 1 -- 172.21.15.45:0/2197551511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef3c8388d0 con 0x55ef3c99df60 2024-06-06T04:32:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.110+0000 7f435bfff700 1 --2- 172.21.15.45:0/2197551511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c99df60 0x55ef3c99e340 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f434c097660 tx=0x7f434c09b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fd3c495d0f6d17 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.111+0000 7f43527fc700 1 -- 172.21.15.45:0/2197551511 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c01f040 con 0x55ef3c99df60 2024-06-06T04:32:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.111+0000 7f43527fc700 1 -- 172.21.15.45:0/2197551511 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f434c09f5b0 con 0x55ef3c99df60 2024-06-06T04:32:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.111+0000 7f43527fc700 1 -- 172.21.15.45:0/2197551511 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c09f320 con 0x55ef3c99df60 2024-06-06T04:32:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.111+0000 7f436cf49b80 1 -- 172.21.15.45:0/2197551511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c99df60 msgr2=0x55ef3c99e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.111+0000 7f436cf49b80 1 --2- 172.21.15.45:0/2197551511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c99df60 0x55ef3c99e340 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f434c097660 tx=0x7f434c09b900 comp rx=0 tx=0).stop 2024-06-06T04:32:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.111+0000 7f436cf49b80 1 -- 172.21.15.45:0/2197551511 shutdown_connections 2024-06-06T04:32:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.111+0000 7f436cf49b80 1 --2- 172.21.15.45:0/2197551511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c9987a0 0x55ef3c998bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.111+0000 7f436cf49b80 1 --2- 172.21.15.45:0/2197551511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c99df60 0x55ef3c99e340 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.111+0000 7f436cf49b80 1 --2- 172.21.15.45:0/2197551511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99e880 0x55ef3c998260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.111+0000 7f436cf49b80 1 -- 172.21.15.45:0/2197551511 >> 172.21.15.45:0/2197551511 conn(0x55ef3c8c5080 msgr2=0x55ef3c993220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:45.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.111+0000 7f436cf49b80 1 -- 172.21.15.45:0/2197551511 shutdown_connections 2024-06-06T04:32:45.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.112+0000 7f436cf49b80 1 -- 172.21.15.45:0/2197551511 wait complete. 2024-06-06T04:32:45.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.112+0000 7f436cf49b80 1 -- start start 2024-06-06T04:32:45.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.112+0000 7f436cf49b80 1 Processor -- start 2024-06-06T04:32:45.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.112+0000 7f436cf49b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c9987a0 0x55ef3c9a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.112+0000 7f436cf49b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c99df60 0x55ef3c9a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.112+0000 7f436cf49b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99e880 0x55ef3c9a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.112+0000 7f436cf49b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ef3c889fc0 con 0x55ef3c99df60 2024-06-06T04:32:45.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.112+0000 7f436cf49b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef3c8e5810 con 0x55ef3c9987a0 2024-06-06T04:32:45.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.112+0000 7f436cf49b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ef3c9a5260 con 0x55ef3c99e880 2024-06-06T04:32:45.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.112+0000 7f436cf49b80 10 client.0 _ll_register_callbacks cb 0x55ef3c87f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:45.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.112+0000 7f435b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c99df60 0x55ef3c9a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f435b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c99df60 0x55ef3c9a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48884/0 (socket says 172.21.15.45:48884) 2024-06-06T04:32:45.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f435b7fe700 1 -- 172.21.15.45:0/2143460258 learned_addr learned my addr 172.21.15.45:0/2143460258 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:45.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f435bfff700 1 --2- 172.21.15.45:0/2143460258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c9987a0 0x55ef3c9a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f4360c63700 1 --2- 172.21.15.45:0/2143460258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99e880 0x55ef3c9a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f435b7fe700 1 -- 172.21.15.45:0/2143460258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99e880 msgr2=0x55ef3c9a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:45.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f435b7fe700 1 --2- 172.21.15.45:0/2143460258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99e880 0x55ef3c9a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f435b7fe700 1 -- 172.21.15.45:0/2143460258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c9987a0 msgr2=0x55ef3c9a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:45.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f435b7fe700 1 --2- 172.21.15.45:0/2143460258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c9987a0 0x55ef3c9a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f435b7fe700 1 -- 172.21.15.45:0/2143460258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef3c8af750 con 0x55ef3c99df60 2024-06-06T04:32:45.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f435bfff700 1 --2- 172.21.15.45:0/2143460258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c9987a0 0x55ef3c9a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:45.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f4360c63700 1 --2- 172.21.15.45:0/2143460258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99e880 0x55ef3c9a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:45.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f435b7fe700 1 --2- 172.21.15.45:0/2143460258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c99df60 0x55ef3c9a18c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4344011a80 tx=0x7f43440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771942875a5a5414 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:45.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f433b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:45.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f433b7fe700 1 -- 172.21.15.45:0/2143460258 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344004220 con 0x55ef3c99df60 2024-06-06T04:32:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f436cf49b80 1 -- 172.21.15.45:0/2143460258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f434c09f5b0 con 0x55ef3c99df60 2024-06-06T04:32:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f433b7fe700 1 -- 172.21.15.45:0/2143460258 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f43440043c0 con 0x55ef3c99df60 2024-06-06T04:32:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f433b7fe700 1 -- 172.21.15.45:0/2143460258 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f434401f6a0 con 0x55ef3c99df60 2024-06-06T04:32:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f436cf49b80 10 client.15579 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.113+0000 7f436cf49b80 1 -- 172.21.15.45:0/2143460258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ef3c9afa60 con 0x55ef3c99df60 2024-06-06T04:32:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.114+0000 7f433b7fe700 1 -- 172.21.15.45:0/2143460258 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f434c09f5b0 con 0x55ef3c99df60 2024-06-06T04:32:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.114+0000 7f433b7fe700 1 -- 172.21.15.45:0/2143460258 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4344003ca0 con 0x55ef3c99df60 2024-06-06T04:32:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.114+0000 7f436cf49b80 10 client.15579 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.114+0000 7f436cf49b80 10 client.15579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.114+0000 7f436cf49b80 20 client.15579 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.114+0000 7f436cf49b80 1 --2- 172.21.15.45:0/2143460258 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55ef3c9b1ff0 0x55ef3c9b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.114+0000 7f436cf49b80 4 client.15579 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:32:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.114+0000 7f436cf49b80 1 -- 172.21.15.45:0/2143460258 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef3c8f6d50 con 0x55ef3c9b1ff0 2024-06-06T04:32:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.114+0000 7f435bfff700 1 --2- 172.21.15.45:0/2143460258 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55ef3c9b1ff0 0x55ef3c9b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.115+0000 7f435bfff700 1 --2- 172.21.15.45:0/2143460258 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55ef3c9b1ff0 0x55ef3c9b43e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=996147b79dd44c10 server_cookie=8174b8fe2669c3c1 in_seq=0 out_seq=0 2024-06-06T04:32:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.115+0000 7f433b7fe700 10 client.15579 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.115+0000 7f433b7fe700 1 -- 172.21.15.45:0/2143460258 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ef3c8f6d50 con 0x55ef3c9b1ff0 2024-06-06T04:32:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.115+0000 7f433b7fe700 10 client.15579 handle_command_reply: tid=0 2024-06-06T04:32:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.116+0000 7f436cf49b80 1 client.15579 shutdown 2024-06-06T04:32:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.116+0000 7f436cf49b80 1 -- 172.21.15.45:0/2143460258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c99df60 msgr2=0x55ef3c9a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.116+0000 7f436cf49b80 1 --2- 172.21.15.45:0/2143460258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c99df60 0x55ef3c9a18c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4344011a80 tx=0x7f43440047c0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.116+0000 7f436cf49b80 1 -- 172.21.15.45:0/2143460258 shutdown_connections 2024-06-06T04:32:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.116+0000 7f436cf49b80 1 --2- 172.21.15.45:0/2143460258 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55ef3c9b1ff0 0x55ef3c9b43e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.116+0000 7f436cf49b80 1 --2- 172.21.15.45:0/2143460258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99e880 0x55ef3c9a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.116+0000 7f436cf49b80 1 --2- 172.21.15.45:0/2143460258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c9987a0 0x55ef3c9a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.116+0000 7f433b7fe700 0 client.15579 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.116+0000 7f436cf49b80 1 --2- 172.21.15.45:0/2143460258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c99df60 0x55ef3c9a18c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.116+0000 7f436cf49b80 1 -- 172.21.15.45:0/2143460258 >> 172.21.15.45:0/2143460258 conn(0x55ef3c8c5080 msgr2=0x55ef3c992700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:45.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.116+0000 7f436cf49b80 1 -- 172.21.15.45:0/2143460258 shutdown_connections 2024-06-06T04:32:45.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.116+0000 7f436cf49b80 1 -- 172.21.15.45:0/2143460258 wait complete. 2024-06-06T04:32:45.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.116+0000 7f436cf49b80 20 client.15579 trim_cache size 0 max 16384 2024-06-06T04:32:45.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f436cf49b80 1 Processor -- start 2024-06-06T04:32:45.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f436cf49b80 1 -- start start 2024-06-06T04:32:45.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f436cf49b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c9987a0 0x55ef3c998260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f436cf49b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c99df60 0x55ef3c99f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f436cf49b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99e880 0x55ef3c992e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f436cf49b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1b0bf0 con 0x55ef3c9987a0 2024-06-06T04:32:45.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f436cf49b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1ae480 con 0x55ef3c99df60 2024-06-06T04:32:45.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f436cf49b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1ac2d0 con 0x55ef3c99e880 2024-06-06T04:32:45.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f435bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c9987a0 0x55ef3c998260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f435bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c9987a0 0x55ef3c998260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48888/0 (socket says 172.21.15.45:48888) 2024-06-06T04:32:45.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f435bfff700 1 -- 172.21.15.45:0/2844786876 learned_addr learned my addr 172.21.15.45:0/2844786876 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:45.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f435b7fe700 1 --2- 172.21.15.45:0/2844786876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c99df60 0x55ef3c99f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f435bfff700 1 -- 172.21.15.45:0/2844786876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99e880 msgr2=0x55ef3c992e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:45.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f435bfff700 1 --2- 172.21.15.45:0/2844786876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99e880 0x55ef3c992e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f435bfff700 1 -- 172.21.15.45:0/2844786876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c99df60 msgr2=0x55ef3c99f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:45.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f435bfff700 1 --2- 172.21.15.45:0/2844786876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c99df60 0x55ef3c99f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.132+0000 7f435bfff700 1 -- 172.21.15.45:0/2844786876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef3c99f8b0 con 0x55ef3c9987a0 2024-06-06T04:32:45.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.133+0000 7f435bfff700 1 --2- 172.21.15.45:0/2844786876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c9987a0 0x55ef3c998260 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f434c0cc4f0 tx=0x7f434c0975d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1033e4713daf7c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:45.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.133+0000 7f4351ffb700 1 -- 172.21.15.45:0/2844786876 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c01f040 con 0x55ef3c9987a0 2024-06-06T04:32:45.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.133+0000 7f4351ffb700 1 -- 172.21.15.45:0/2844786876 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f434c09b900 con 0x55ef3c9987a0 2024-06-06T04:32:45.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.133+0000 7f4351ffb700 1 -- 172.21.15.45:0/2844786876 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c09f380 con 0x55ef3c9987a0 2024-06-06T04:32:45.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.134+0000 7f436cf49b80 1 -- 172.21.15.45:0/2844786876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c9987a0 msgr2=0x55ef3c998260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:45.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.134+0000 7f436cf49b80 1 --2- 172.21.15.45:0/2844786876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c9987a0 0x55ef3c998260 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f434c0cc4f0 tx=0x7f434c0975d0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.134+0000 7f436cf49b80 1 -- 172.21.15.45:0/2844786876 shutdown_connections 2024-06-06T04:32:45.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.134+0000 7f436cf49b80 1 --2- 172.21.15.45:0/2844786876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99e880 0x55ef3c992e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.134+0000 7f436cf49b80 1 --2- 172.21.15.45:0/2844786876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c9987a0 0x55ef3c998260 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.134+0000 7f436cf49b80 1 --2- 172.21.15.45:0/2844786876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c99df60 0x55ef3c99f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.134+0000 7f436cf49b80 1 -- 172.21.15.45:0/2844786876 >> 172.21.15.45:0/2844786876 conn(0x55ef3c8c5080 msgr2=0x55ef3c99ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:45.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.134+0000 7f436cf49b80 1 -- 172.21.15.45:0/2844786876 shutdown_connections 2024-06-06T04:32:45.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.134+0000 7f436cf49b80 1 -- 172.21.15.45:0/2844786876 wait complete. 2024-06-06T04:32:45.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.134+0000 7f436cf49b80 1 -- start start 2024-06-06T04:32:45.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f436cf49b80 1 Processor -- start 2024-06-06T04:32:45.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f436cf49b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c9987a0 0x55ef3c9aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f436cf49b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99df60 0x55ef3c9aacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f436cf49b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c99e880 0x55ef3c9b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f436cf49b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1ac940 con 0x55ef3c9987a0 2024-06-06T04:32:45.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f436cf49b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1addb0 con 0x55ef3c99e880 2024-06-06T04:32:45.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f436cf49b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1ad010 con 0x55ef3c99df60 2024-06-06T04:32:45.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f436cf49b80 10 client.0 _ll_register_callbacks cb 0x55ef3c87f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:45.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f435bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c9987a0 0x55ef3c9aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f435bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c9987a0 0x55ef3c9aa7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48898/0 (socket says 172.21.15.45:48898) 2024-06-06T04:32:45.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f435bfff700 1 -- 172.21.15.45:0/3660436334 learned_addr learned my addr 172.21.15.45:0/3660436334 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f435b7fe700 1 --2- 172.21.15.45:0/3660436334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99df60 0x55ef3c9aacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f4360c63700 1 --2- 172.21.15.45:0/3660436334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c99e880 0x55ef3c9b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f435bfff700 1 -- 172.21.15.45:0/3660436334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99df60 msgr2=0x55ef3c9aacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f435bfff700 1 --2- 172.21.15.45:0/3660436334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99df60 0x55ef3c9aacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f435bfff700 1 -- 172.21.15.45:0/3660436334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c99e880 msgr2=0x55ef3c9b9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f435bfff700 1 --2- 172.21.15.45:0/3660436334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c99e880 0x55ef3c9b9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.135+0000 7f435bfff700 1 -- 172.21.15.45:0/3660436334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef3c9b9a80 con 0x55ef3c9987a0 2024-06-06T04:32:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.136+0000 7f435bfff700 1 --2- 172.21.15.45:0/3660436334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c9987a0 0x55ef3c9aa7b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f434c09b860 tx=0x7f434c0055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f99cac2d538d49f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.136+0000 7f433b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.136+0000 7f433b7fe700 1 -- 172.21.15.45:0/3660436334 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c01f040 con 0x55ef3c9987a0 2024-06-06T04:32:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.136+0000 7f433b7fe700 1 -- 172.21.15.45:0/3660436334 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f434c09f120 con 0x55ef3c9987a0 2024-06-06T04:32:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.136+0000 7f436cf49b80 1 -- 172.21.15.45:0/3660436334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f434c09b900 con 0x55ef3c9987a0 2024-06-06T04:32:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.136+0000 7f433b7fe700 1 -- 172.21.15.45:0/3660436334 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f434c0b41f0 con 0x55ef3c9987a0 2024-06-06T04:32:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.136+0000 7f436cf49b80 10 client.15585 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:45.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.136+0000 7f436cf49b80 1 -- 172.21.15.45:0/3660436334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ef3ca1c780 con 0x55ef3c9987a0 2024-06-06T04:32:45.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.136+0000 7f433b7fe700 1 -- 172.21.15.45:0/3660436334 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f434c09b900 con 0x55ef3c9987a0 2024-06-06T04:32:45.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.136+0000 7f433b7fe700 1 -- 172.21.15.45:0/3660436334 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f434c09a650 con 0x55ef3c9987a0 2024-06-06T04:32:45.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.137+0000 7f436cf49b80 10 client.15585 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:45.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.137+0000 7f436cf49b80 10 client.15585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:45.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.137+0000 7f436cf49b80 20 client.15585 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:45.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.137+0000 7f436cf49b80 1 --2- 172.21.15.45:0/3660436334 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55ef3ca1c950 0x55ef3ca1ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:45.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.137+0000 7f436cf49b80 4 client.15585 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T04:32:45.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.137+0000 7f436cf49b80 1 -- 172.21.15.45:0/3660436334 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef3c8f12d0 con 0x55ef3ca1c950 2024-06-06T04:32:45.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.137+0000 7f435b7fe700 1 --2- 172.21.15.45:0/3660436334 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55ef3ca1c950 0x55ef3ca1ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:45.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.137+0000 7f433b7fe700 10 client.15585 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:45.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.137+0000 7f435b7fe700 1 --2- 172.21.15.45:0/3660436334 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55ef3ca1c950 0x55ef3ca1ed40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfb56717de347a7c server_cookie=9a21a2b343fd4c08 in_seq=0 out_seq=0 2024-06-06T04:32:45.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.137+0000 7f433b7fe700 1 -- 172.21.15.45:0/3660436334 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef3c8f12d0 con 0x55ef3ca1c950 2024-06-06T04:32:45.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.137+0000 7f433b7fe700 10 client.15585 handle_command_reply: tid=0 2024-06-06T04:32:45.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.137+0000 7f436cf49b80 1 client.15585 shutdown 2024-06-06T04:32:45.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.138+0000 7f436cf49b80 1 -- 172.21.15.45:0/3660436334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c9987a0 msgr2=0x55ef3c9aa7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:45.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.138+0000 7f436cf49b80 1 --2- 172.21.15.45:0/3660436334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c9987a0 0x55ef3c9aa7b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f434c09b860 tx=0x7f434c0055f0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.138+0000 7f436cf49b80 1 -- 172.21.15.45:0/3660436334 shutdown_connections 2024-06-06T04:32:45.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.138+0000 7f436cf49b80 1 --2- 172.21.15.45:0/3660436334 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55ef3ca1c950 0x55ef3ca1ed40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.138+0000 7f436cf49b80 1 --2- 172.21.15.45:0/3660436334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef3c99e880 0x55ef3c9b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.138+0000 7f436cf49b80 1 --2- 172.21.15.45:0/3660436334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ef3c9987a0 0x55ef3c9aa7b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.138+0000 7f436cf49b80 1 --2- 172.21.15.45:0/3660436334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ef3c99df60 0x55ef3c9aacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.138+0000 7f433b7fe700 0 client.15585 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:45.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.138+0000 7f436cf49b80 1 -- 172.21.15.45:0/3660436334 >> 172.21.15.45:0/3660436334 conn(0x55ef3c8c5080 msgr2=0x55ef3c9a3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:45.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.138+0000 7f436cf49b80 1 -- 172.21.15.45:0/3660436334 shutdown_connections 2024-06-06T04:32:45.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.138+0000 7f436cf49b80 1 -- 172.21.15.45:0/3660436334 wait complete. 2024-06-06T04:32:45.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.138+0000 7f436cf49b80 20 client.15585 trim_cache size 0 max 16384 2024-06-06T04:32:45.143 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-06-06T04:32:45.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.139+0000 7f43626c6700 1 -- 172.21.15.45:0/1901421539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43480a2c20 msgr2=0x7f43480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:45.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.139+0000 7f43626c6700 1 --2- 172.21.15.45:0/1901421539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43480a2c20 0x7f43480a50e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f435c079af0 tx=0x7f434400b500 comp rx=0 tx=0).stop 2024-06-06T04:32:45.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.139+0000 7f43626c6700 1 -- 172.21.15.45:0/1901421539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f435c102510 msgr2=0x7f435c19e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:45.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.139+0000 7f43626c6700 1 --2- 172.21.15.45:0/1901421539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f435c102510 0x7f435c19e760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f434c00d750 tx=0x7f434c00da60 comp rx=0 tx=0).stop 2024-06-06T04:32:45.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.139+0000 7f43626c6700 1 -- 172.21.15.45:0/1901421539 shutdown_connections 2024-06-06T04:32:45.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.139+0000 7f43626c6700 1 --2- 172.21.15.45:0/1901421539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c108520 0x7f435c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.139+0000 7f43626c6700 1 --2- 172.21.15.45:0/1901421539 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43480a2c20 0x7f43480a50e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.139+0000 7f43626c6700 1 --2- 172.21.15.45:0/1901421539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f435c102510 0x7f435c19e760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.139+0000 7f43626c6700 1 --2- 172.21.15.45:0/1901421539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f435c102ed0 0x7f435c19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:45.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.139+0000 7f43626c6700 1 -- 172.21.15.45:0/1901421539 >> 172.21.15.45:0/1901421539 conn(0x7f435c079560 msgr2=0x7f435c101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:45.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.139+0000 7f43626c6700 1 -- 172.21.15.45:0/1901421539 shutdown_connections 2024-06-06T04:32:45.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:45.139+0000 7f43626c6700 1 -- 172.21.15.45:0/1901421539 wait complete. 2024-06-06T04:32:46.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:46 smithi086 ceph-mon[36053]: pgmap v470: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-06T04:32:46.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:46 smithi154 ceph-mon[36428]: pgmap v470: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-06T04:32:46.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:46 smithi045 ceph-mon[32299]: pgmap v470: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-06T04:32:47.215 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-06T04:32:47.215 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-06T04:32:47.480 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:32:47.789 INFO:tasks.workunit.client.0.smithi045.stderr:Updating files: 93% (11712/12568) Updating files: 94% (11814/12568) Updating files: 95% (11940/12568) Updating files: 96% (12066/12568) Updating files: 97% (12191/12568) Updating files: 98% (12317/12568) Updating files: 99% (12443/12568) Updating files: 100% (12568/12568) Updating files: 100% (12568/12568), done. 2024-06-06T04:32:48.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:48 smithi086 ceph-mon[36053]: pgmap v471: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-06T04:32:48.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:48 smithi154 ceph-mon[36428]: pgmap v471: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-06T04:32:48.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:48 smithi045 ceph-mon[32299]: pgmap v471: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-06T04:32:48.588 INFO:tasks.workunit.client.0.smithi045.stderr:Note: switching to '3d746ca5346f34eef7e826a1eb71e59e02b682ae'. 2024-06-06T04:32:48.589 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-06-06T04:32:48.589 INFO:tasks.workunit.client.0.smithi045.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-06T04:32:48.589 INFO:tasks.workunit.client.0.smithi045.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-06T04:32:48.589 INFO:tasks.workunit.client.0.smithi045.stderr:state without impacting any branches by switching back to a branch. 2024-06-06T04:32:48.589 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-06-06T04:32:48.589 INFO:tasks.workunit.client.0.smithi045.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-06T04:32:48.589 INFO:tasks.workunit.client.0.smithi045.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-06T04:32:48.589 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-06-06T04:32:48.589 INFO:tasks.workunit.client.0.smithi045.stderr: git switch -c 2024-06-06T04:32:48.589 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-06-06T04:32:48.589 INFO:tasks.workunit.client.0.smithi045.stderr:Or undo this operation with: 2024-06-06T04:32:48.589 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-06-06T04:32:48.590 INFO:tasks.workunit.client.0.smithi045.stderr: git switch - 2024-06-06T04:32:48.590 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-06-06T04:32:48.590 INFO:tasks.workunit.client.0.smithi045.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-06T04:32:48.590 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-06-06T04:32:48.590 INFO:tasks.workunit.client.0.smithi045.stderr:HEAD is now at 3d746ca5346 Merge pull request #57870 from zdover23/wip-doc-2024-06-05-backport-57867-to-squid 2024-06-06T04:32:48.593 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-06T04:32:48.646 INFO:tasks.workunit.client.0.smithi045.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-06T04:32:48.648 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-06T04:32:48.648 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-06T04:32:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.621+0000 7f7e3a276700 1 -- 172.21.15.45:0/3450603018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e34101390 msgr2=0x7f7e341017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.621+0000 7f7e3a276700 1 --2- 172.21.15.45:0/3450603018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e34101390 0x7f7e341017f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009a10 tx=0x7f7e28009d20 comp rx=0 tx=0).stop 2024-06-06T04:32:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.621+0000 7f7e3a276700 1 -- 172.21.15.45:0/3450603018 shutdown_connections 2024-06-06T04:32:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.621+0000 7f7e3a276700 1 --2- 172.21.15.45:0/3450603018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e34101d30 0x7f7e3410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.621+0000 7f7e3a276700 1 --2- 172.21.15.45:0/3450603018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e341073d0 0x7f7e341077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.621+0000 7f7e3a276700 1 --2- 172.21.15.45:0/3450603018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e34101390 0x7f7e341017f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.621+0000 7f7e3a276700 1 -- 172.21.15.45:0/3450603018 >> 172.21.15.45:0/3450603018 conn(0x7f7e340782c0 msgr2=0x7f7e340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.621+0000 7f7e3a276700 1 -- 172.21.15.45:0/3450603018 shutdown_connections 2024-06-06T04:32:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.621+0000 7f7e3a276700 1 -- 172.21.15.45:0/3450603018 wait complete. 2024-06-06T04:32:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.622+0000 7f7e3a276700 1 Processor -- start 2024-06-06T04:32:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.622+0000 7f7e3a276700 1 -- start start 2024-06-06T04:32:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.622+0000 7f7e3a276700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e34101390 0x7f7e3419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.622+0000 7f7e3a276700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e34101d30 0x7f7e3419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.622+0000 7f7e3a276700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e341073d0 0x7f7e3419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.622+0000 7f7e3a276700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e34110490 con 0x7f7e34101390 2024-06-06T04:32:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.622+0000 7f7e3a276700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e34110310 con 0x7f7e34101d30 2024-06-06T04:32:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.622+0000 7f7e3a276700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7e34110610 con 0x7f7e341073d0 2024-06-06T04:32:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.622+0000 7f7e33fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e34101390 0x7f7e3419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.622+0000 7f7e33fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e34101390 0x7f7e3419ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40168/0 (socket says 172.21.15.45:40168) 2024-06-06T04:32:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.622+0000 7f7e33fff700 1 -- 172.21.15.45:0/3590234885 learned_addr learned my addr 172.21.15.45:0/3590234885 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.622+0000 7f7e337fe700 1 --2- 172.21.15.45:0/3590234885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e34101d30 0x7f7e3419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.622+0000 7f7e38813700 1 --2- 172.21.15.45:0/3590234885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e341073d0 0x7f7e3419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.623+0000 7f7e33fff700 1 -- 172.21.15.45:0/3590234885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e341073d0 msgr2=0x7f7e3419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.623+0000 7f7e33fff700 1 --2- 172.21.15.45:0/3590234885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e341073d0 0x7f7e3419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.623+0000 7f7e33fff700 1 -- 172.21.15.45:0/3590234885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e34101d30 msgr2=0x7f7e3419b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.623+0000 7f7e33fff700 1 --2- 172.21.15.45:0/3590234885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e34101d30 0x7f7e3419b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.623+0000 7f7e33fff700 1 -- 172.21.15.45:0/3590234885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3419fd20 con 0x7f7e34101390 2024-06-06T04:32:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.623+0000 7f7e33fff700 1 --2- 172.21.15.45:0/3590234885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e34101390 0x7f7e3419ad10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c920 tx=0x7f7e2400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.623+0000 7f7e317fa700 1 -- 172.21.15.45:0/3590234885 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24004a30 con 0x7f7e34101390 2024-06-06T04:32:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.623+0000 7f7e3a276700 1 -- 172.21.15.45:0/3590234885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e341a0010 con 0x7f7e34101390 2024-06-06T04:32:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.623+0000 7f7e317fa700 1 -- 172.21.15.45:0/3590234885 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e240054d0 con 0x7f7e34101390 2024-06-06T04:32:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.623+0000 7f7e317fa700 1 -- 172.21.15.45:0/3590234885 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24007bd0 con 0x7f7e34101390 2024-06-06T04:32:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.623+0000 7f7e3a276700 1 -- 172.21.15.45:0/3590234885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e341a7890 con 0x7f7e34101390 2024-06-06T04:32:49.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.625+0000 7f7e317fa700 1 -- 172.21.15.45:0/3590234885 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7e24005000 con 0x7f7e34101390 2024-06-06T04:32:49.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.626+0000 7f7e317fa700 1 --2- 172.21.15.45:0/3590234885 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7e1c0a2c20 0x7f7e1c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.626+0000 7f7e317fa700 1 -- 172.21.15.45:0/3590234885 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f7e240ceee0 con 0x7f7e34101390 2024-06-06T04:32:49.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.626+0000 7f7e337fe700 1 --2- 172.21.15.45:0/3590234885 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7e1c0a2c20 0x7f7e1c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.627+0000 7f7e337fe700 1 --2- 172.21.15.45:0/3590234885 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7e1c0a2c20 0x7f7e1c0a50e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7e34078910 tx=0x7f7e28005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:49.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.627+0000 7f7e44af9b80 1 Processor -- start 2024-06-06T04:32:49.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.627+0000 7f7e44af9b80 1 -- start start 2024-06-06T04:32:49.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.627+0000 7f7e44af9b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2dc6c0 0x55e44d2dcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.627+0000 7f7e44af9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dcfe0 0x55e44d2d6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.627+0000 7f7e44af9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2d6fa0 0x55e44d2d73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.627+0000 7f7e44af9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e44d228890 con 0x55e44d2dcfe0 2024-06-06T04:32:49.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.627+0000 7f7e44af9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e44d1d07a0 con 0x55e44d2d6fa0 2024-06-06T04:32:49.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.627+0000 7f7e44af9b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e44d224850 con 0x55e44d2dc6c0 2024-06-06T04:32:49.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.627+0000 7f7e337fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dcfe0 0x55e44d2d6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.627+0000 7f7e337fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dcfe0 0x55e44d2d6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40172/0 (socket says 172.21.15.45:40172) 2024-06-06T04:32:49.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.627+0000 7f7e337fe700 1 -- 172.21.15.45:0/1479970358 learned_addr learned my addr 172.21.15.45:0/1479970358 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:49.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.627+0000 7f7e33fff700 1 --2- 172.21.15.45:0/1479970358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2dc6c0 0x55e44d2dcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.628+0000 7f7e38813700 1 --2- 172.21.15.45:0/1479970358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2d6fa0 0x55e44d2d73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.628+0000 7f7e337fe700 1 -- 172.21.15.45:0/1479970358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2dc6c0 msgr2=0x55e44d2dcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:49.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.628+0000 7f7e337fe700 1 --2- 172.21.15.45:0/1479970358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2dc6c0 0x55e44d2dcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.628+0000 7f7e337fe700 1 -- 172.21.15.45:0/1479970358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2d6fa0 msgr2=0x55e44d2d73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:49.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.628+0000 7f7e337fe700 1 --2- 172.21.15.45:0/1479970358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2d6fa0 0x55e44d2d73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.628+0000 7f7e337fe700 1 -- 172.21.15.45:0/1479970358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e44d1c6fd0 con 0x55e44d2dcfe0 2024-06-06T04:32:49.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.628+0000 7f7e38813700 1 --2- 172.21.15.45:0/1479970358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2d6fa0 0x55e44d2d73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:49.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.628+0000 7f7e337fe700 1 --2- 172.21.15.45:0/1479970358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dcfe0 0x55e44d2d6a60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009a10 tx=0x7f7e2801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b5d0fcbc824331 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:49.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.628+0000 7f7e1a7fc700 1 -- 172.21.15.45:0/1479970358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e28028050 con 0x55e44d2dcfe0 2024-06-06T04:32:49.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.628+0000 7f7e1a7fc700 1 -- 172.21.15.45:0/1479970358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e28032070 con 0x55e44d2dcfe0 2024-06-06T04:32:49.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.628+0000 7f7e1a7fc700 1 -- 172.21.15.45:0/1479970358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e280226d0 con 0x55e44d2dcfe0 2024-06-06T04:32:49.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.629+0000 7f7e44af9b80 1 -- 172.21.15.45:0/1479970358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dcfe0 msgr2=0x55e44d2d6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.629+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/1479970358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dcfe0 0x55e44d2d6a60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009a10 tx=0x7f7e2801f000 comp rx=0 tx=0).stop 2024-06-06T04:32:49.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.629+0000 7f7e44af9b80 1 -- 172.21.15.45:0/1479970358 shutdown_connections 2024-06-06T04:32:49.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.629+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/1479970358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2d6fa0 0x55e44d2d73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.629+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/1479970358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2dc6c0 0x55e44d2dcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.629+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/1479970358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dcfe0 0x55e44d2d6a60 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.629+0000 7f7e44af9b80 1 -- 172.21.15.45:0/1479970358 >> 172.21.15.45:0/1479970358 conn(0x55e44d201d40 msgr2=0x55e44d2d18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:49.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.629+0000 7f7e44af9b80 1 -- 172.21.15.45:0/1479970358 shutdown_connections 2024-06-06T04:32:49.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.629+0000 7f7e44af9b80 1 -- 172.21.15.45:0/1479970358 wait complete. 2024-06-06T04:32:49.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.629+0000 7f7e44af9b80 1 -- start start 2024-06-06T04:32:49.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.630+0000 7f7e44af9b80 1 Processor -- start 2024-06-06T04:32:49.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.630+0000 7f7e44af9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2d6fa0 0x55e44d2e2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.630+0000 7f7e44af9b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2dc6c0 0x55e44d2e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.630+0000 7f7e44af9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dcfe0 0x55e44d2e05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.630+0000 7f7e44af9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e44d2e39f0 con 0x55e44d2dcfe0 2024-06-06T04:32:49.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.630+0000 7f7e44af9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e44d1c8c70 con 0x55e44d2d6fa0 2024-06-06T04:32:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.630+0000 7f7e44af9b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e44d2e3b70 con 0x55e44d2dc6c0 2024-06-06T04:32:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.630+0000 7f7e44af9b80 10 client.0 _ll_register_callbacks cb 0x55e44d1c29a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.630+0000 7f7e38813700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dcfe0 0x55e44d2e05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.630+0000 7f7e38813700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dcfe0 0x55e44d2e05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40174/0 (socket says 172.21.15.45:40174) 2024-06-06T04:32:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.630+0000 7f7e38813700 1 -- 172.21.15.45:0/4045923891 learned_addr learned my addr 172.21.15.45:0/4045923891 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.630+0000 7f7e337fe700 1 --2- 172.21.15.45:0/4045923891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2dc6c0 0x55e44d2e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.630+0000 7f7e33fff700 1 --2- 172.21.15.45:0/4045923891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2d6fa0 0x55e44d2e2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.631+0000 7f7e38813700 1 -- 172.21.15.45:0/4045923891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2dc6c0 msgr2=0x55e44d2e0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.631+0000 7f7e38813700 1 --2- 172.21.15.45:0/4045923891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2dc6c0 0x55e44d2e0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.631+0000 7f7e38813700 1 -- 172.21.15.45:0/4045923891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2d6fa0 msgr2=0x55e44d2e2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.631+0000 7f7e38813700 1 --2- 172.21.15.45:0/4045923891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2d6fa0 0x55e44d2e2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.631+0000 7f7e38813700 1 -- 172.21.15.45:0/4045923891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e44d177950 con 0x55e44d2dcfe0 2024-06-06T04:32:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.631+0000 7f7e33fff700 1 --2- 172.21.15.45:0/4045923891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2d6fa0 0x55e44d2e2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.631+0000 7f7e38813700 1 --2- 172.21.15.45:0/4045923891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dcfe0 0x55e44d2e05d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c004090 tx=0x7f7e2c0043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a30db22f24b86f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:49.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.631+0000 7f7e137fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.631+0000 7f7e137fe700 1 -- 172.21.15.45:0/4045923891 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c019020 con 0x55e44d2dcfe0 2024-06-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.631+0000 7f7e44af9b80 1 -- 172.21.15.45:0/4045923891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e28032070 con 0x55e44d2dcfe0 2024-06-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.631+0000 7f7e137fe700 1 -- 172.21.15.45:0/4045923891 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f7e2c0077d0 con 0x55e44d2dcfe0 2024-06-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.631+0000 7f7e137fe700 1 -- 172.21.15.45:0/4045923891 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e2c024050 con 0x55e44d2dcfe0 2024-06-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.631+0000 7f7e44af9b80 10 client.15597 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.631+0000 7f7e44af9b80 1 -- 172.21.15.45:0/4045923891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e44d2ee300 con 0x55e44d2dcfe0 2024-06-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.631+0000 7f7e137fe700 1 -- 172.21.15.45:0/4045923891 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e28032070 con 0x55e44d2dcfe0 2024-06-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.632+0000 7f7e137fe700 1 -- 172.21.15.45:0/4045923891 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7e2c015ac0 con 0x55e44d2dcfe0 2024-06-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.632+0000 7f7e44af9b80 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.632+0000 7f7e44af9b80 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.632+0000 7f7e44af9b80 20 client.15597 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.632+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/4045923891 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55e44d2f0940 0x55e44d2f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.632+0000 7f7e44af9b80 4 client.15597 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.632+0000 7f7e44af9b80 1 -- 172.21.15.45:0/4045923891 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e44d235d90 con 0x55e44d2f0940 2024-06-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.632+0000 7f7e33fff700 1 --2- 172.21.15.45:0/4045923891 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55e44d2f0940 0x55e44d2f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.632+0000 7f7e137fe700 10 client.15597 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.632+0000 7f7e33fff700 1 --2- 172.21.15.45:0/4045923891 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55e44d2f0940 0x55e44d2f2d70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b05c713648aa5f59 server_cookie=251d476135fdee68 in_seq=0 out_seq=0 2024-06-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.633+0000 7f7e137fe700 1 -- 172.21.15.45:0/4045923891 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e44d235d90 con 0x55e44d2f0940 2024-06-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.633+0000 7f7e137fe700 10 client.15597 handle_command_reply: tid=0 2024-06-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.633+0000 7f7e44af9b80 1 client.15597 shutdown 2024-06-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.633+0000 7f7e44af9b80 1 -- 172.21.15.45:0/4045923891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dcfe0 msgr2=0x55e44d2e05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.633+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/4045923891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dcfe0 0x55e44d2e05d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c004090 tx=0x7f7e2c0043a0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.633+0000 7f7e44af9b80 1 -- 172.21.15.45:0/4045923891 shutdown_connections 2024-06-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.633+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/4045923891 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55e44d2f0940 0x55e44d2f2d70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.633+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/4045923891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dcfe0 0x55e44d2e05d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.633+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/4045923891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2d6fa0 0x55e44d2e2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.633+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/4045923891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2dc6c0 0x55e44d2e0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.633+0000 7f7e44af9b80 1 -- 172.21.15.45:0/4045923891 >> 172.21.15.45:0/4045923891 conn(0x55e44d201d40 msgr2=0x55e44d2d5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.633+0000 7f7e137fe700 0 client.15597 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.634+0000 7f7e44af9b80 1 -- 172.21.15.45:0/4045923891 shutdown_connections 2024-06-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.634+0000 7f7e44af9b80 1 -- 172.21.15.45:0/4045923891 wait complete. 2024-06-06T04:32:49.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.634+0000 7f7e44af9b80 20 client.15597 trim_cache size 0 max 16384 2024-06-06T04:32:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.654+0000 7f7e44af9b80 1 Processor -- start 2024-06-06T04:32:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.654+0000 7f7e44af9b80 1 -- start start 2024-06-06T04:32:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e44af9b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2d6fa0 0x55e44d2d6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e44af9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dc6c0 0x55e44d2e2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e44af9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2dcfe0 0x55e44d2ddd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e44af9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e341b2d50 con 0x55e44d2dc6c0 2024-06-06T04:32:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e44af9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e341b05e0 con 0x55e44d2dcfe0 2024-06-06T04:32:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e44af9b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7e341ae430 con 0x55e44d2d6fa0 2024-06-06T04:32:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e337fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dc6c0 0x55e44d2e2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e337fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dc6c0 0x55e44d2e2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40180/0 (socket says 172.21.15.45:40180) 2024-06-06T04:32:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e337fe700 1 -- 172.21.15.45:0/4228012040 learned_addr learned my addr 172.21.15.45:0/4228012040 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e38813700 1 --2- 172.21.15.45:0/4228012040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2dcfe0 0x55e44d2ddd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e33fff700 1 --2- 172.21.15.45:0/4228012040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2d6fa0 0x55e44d2d6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e337fe700 1 -- 172.21.15.45:0/4228012040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2d6fa0 msgr2=0x55e44d2d6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e337fe700 1 --2- 172.21.15.45:0/4228012040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2d6fa0 0x55e44d2d6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e337fe700 1 -- 172.21.15.45:0/4228012040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2dcfe0 msgr2=0x55e44d2ddd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e337fe700 1 --2- 172.21.15.45:0/4228012040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2dcfe0 0x55e44d2ddd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e337fe700 1 -- 172.21.15.45:0/4228012040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e44d2e06c0 con 0x55e44d2dc6c0 2024-06-06T04:32:49.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e38813700 1 --2- 172.21.15.45:0/4228012040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2dcfe0 0x55e44d2ddd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:49.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.655+0000 7f7e337fe700 1 --2- 172.21.15.45:0/4228012040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dc6c0 0x55e44d2e2600 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7e28003a10 tx=0x7f7e2800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3d119e67aefada server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:49.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.656+0000 7f7e19ffb700 1 -- 172.21.15.45:0/4228012040 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800bb10 con 0x55e44d2dc6c0 2024-06-06T04:32:49.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.656+0000 7f7e19ffb700 1 -- 172.21.15.45:0/4228012040 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e2803f070 con 0x55e44d2dc6c0 2024-06-06T04:32:49.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.656+0000 7f7e19ffb700 1 -- 172.21.15.45:0/4228012040 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e28022b10 con 0x55e44d2dc6c0 2024-06-06T04:32:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.656+0000 7f7e44af9b80 1 -- 172.21.15.45:0/4228012040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dc6c0 msgr2=0x55e44d2e2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.656+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/4228012040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dc6c0 0x55e44d2e2600 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7e28003a10 tx=0x7f7e2800be30 comp rx=0 tx=0).stop 2024-06-06T04:32:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.656+0000 7f7e44af9b80 1 -- 172.21.15.45:0/4228012040 shutdown_connections 2024-06-06T04:32:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.656+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/4228012040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2dcfe0 0x55e44d2ddd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.656+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/4228012040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2d6fa0 0x55e44d2d6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.656+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/4228012040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2dc6c0 0x55e44d2e2600 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.656+0000 7f7e44af9b80 1 -- 172.21.15.45:0/4228012040 >> 172.21.15.45:0/4228012040 conn(0x55e44d201d40 msgr2=0x55e44d2cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.656+0000 7f7e44af9b80 1 -- 172.21.15.45:0/4228012040 shutdown_connections 2024-06-06T04:32:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.656+0000 7f7e44af9b80 1 -- 172.21.15.45:0/4228012040 wait complete. 2024-06-06T04:32:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.657+0000 7f7e44af9b80 1 -- start start 2024-06-06T04:32:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.657+0000 7f7e44af9b80 1 Processor -- start 2024-06-06T04:32:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.657+0000 7f7e44af9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2d6fa0 0x55e44d2f8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.657+0000 7f7e44af9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2dc6c0 0x55e44d2f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.657+0000 7f7e44af9b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2dcfe0 0x55e44d2f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.657+0000 7f7e44af9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e341aeaa0 con 0x55e44d2d6fa0 2024-06-06T04:32:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.657+0000 7f7e44af9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e341aff10 con 0x55e44d2dc6c0 2024-06-06T04:32:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.657+0000 7f7e44af9b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7e341af170 con 0x55e44d2dcfe0 2024-06-06T04:32:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e33fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2d6fa0 0x55e44d2f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e44af9b80 10 client.0 _ll_register_callbacks cb 0x55e44d1c29a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e33fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2d6fa0 0x55e44d2f8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40182/0 (socket says 172.21.15.45:40182) 2024-06-06T04:32:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e33fff700 1 -- 172.21.15.45:0/991844844 learned_addr learned my addr 172.21.15.45:0/991844844 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e38813700 1 --2- 172.21.15.45:0/991844844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2dcfe0 0x55e44d2f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e337fe700 1 --2- 172.21.15.45:0/991844844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2dc6c0 0x55e44d2f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e33fff700 1 -- 172.21.15.45:0/991844844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2dcfe0 msgr2=0x55e44d2f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e33fff700 1 --2- 172.21.15.45:0/991844844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2dcfe0 0x55e44d2f8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e33fff700 1 -- 172.21.15.45:0/991844844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2dc6c0 msgr2=0x55e44d2f89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e33fff700 1 --2- 172.21.15.45:0/991844844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2dc6c0 0x55e44d2f89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e33fff700 1 -- 172.21.15.45:0/991844844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e44d35a820 con 0x55e44d2d6fa0 2024-06-06T04:32:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e337fe700 1 --2- 172.21.15.45:0/991844844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2dc6c0 0x55e44d2f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e33fff700 1 --2- 172.21.15.45:0/991844844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2d6fa0 0x55e44d2f8460 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7e2409b2b0 tx=0x7f7e24000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c195bf6599c1f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e137fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e137fe700 1 -- 172.21.15.45:0/991844844 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2401d040 con 0x55e44d2d6fa0 2024-06-06T04:32:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e137fe700 1 -- 172.21.15.45:0/991844844 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f7e24005680 con 0x55e44d2d6fa0 2024-06-06T04:32:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e44af9b80 1 -- 172.21.15.45:0/991844844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e2803f070 con 0x55e44d2d6fa0 2024-06-06T04:32:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e137fe700 1 -- 172.21.15.45:0/991844844 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e2409f6d0 con 0x55e44d2d6fa0 2024-06-06T04:32:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e44af9b80 10 client.15603 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.658+0000 7f7e44af9b80 1 -- 172.21.15.45:0/991844844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e44d35bba0 con 0x55e44d2d6fa0 2024-06-06T04:32:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.659+0000 7f7e137fe700 1 -- 172.21.15.45:0/991844844 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e2803f070 con 0x55e44d2d6fa0 2024-06-06T04:32:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.659+0000 7f7e137fe700 1 -- 172.21.15.45:0/991844844 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7e240662a0 con 0x55e44d2d6fa0 2024-06-06T04:32:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.659+0000 7f7e44af9b80 10 client.15603 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.659+0000 7f7e44af9b80 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.659+0000 7f7e44af9b80 20 client.15603 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.659+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/991844844 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55e44d35c1b0 0x55e44d35e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.659+0000 7f7e44af9b80 4 client.15603 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-06T04:32:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.659+0000 7f7e44af9b80 1 -- 172.21.15.45:0/991844844 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e44d230310 con 0x55e44d35c1b0 2024-06-06T04:32:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.659+0000 7f7e337fe700 1 --2- 172.21.15.45:0/991844844 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55e44d35c1b0 0x55e44d35e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.660+0000 7f7e337fe700 1 --2- 172.21.15.45:0/991844844 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55e44d35c1b0 0x55e44d35e5d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e39f3d984890518e server_cookie=84e09191d369dc66 in_seq=0 out_seq=0 2024-06-06T04:32:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.660+0000 7f7e137fe700 10 client.15603 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.661+0000 7f7e137fe700 1 -- 172.21.15.45:0/991844844 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e44d230310 con 0x55e44d35c1b0 2024-06-06T04:32:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.661+0000 7f7e137fe700 10 client.15603 handle_command_reply: tid=0 2024-06-06T04:32:49.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.661+0000 7f7e44af9b80 1 client.15603 shutdown 2024-06-06T04:32:49.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.661+0000 7f7e44af9b80 1 -- 172.21.15.45:0/991844844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2d6fa0 msgr2=0x55e44d2f8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.661+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/991844844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2d6fa0 0x55e44d2f8460 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7e2409b2b0 tx=0x7f7e24000bc0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.661+0000 7f7e44af9b80 1 -- 172.21.15.45:0/991844844 shutdown_connections 2024-06-06T04:32:49.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.661+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/991844844 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55e44d35c1b0 0x55e44d35e5d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.661+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/991844844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e44d2dcfe0 0x55e44d2f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.661+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/991844844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44d2d6fa0 0x55e44d2f8460 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.661+0000 7f7e44af9b80 1 --2- 172.21.15.45:0/991844844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e44d2dc6c0 0x55e44d2f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.661+0000 7f7e44af9b80 1 -- 172.21.15.45:0/991844844 >> 172.21.15.45:0/991844844 conn(0x55e44d201d40 msgr2=0x55e44d2ddb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:49.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.661+0000 7f7e137fe700 0 client.15603 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:49.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.661+0000 7f7e44af9b80 1 -- 172.21.15.45:0/991844844 shutdown_connections 2024-06-06T04:32:49.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.661+0000 7f7e44af9b80 1 -- 172.21.15.45:0/991844844 wait complete. 2024-06-06T04:32:49.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.661+0000 7f7e44af9b80 20 client.15603 trim_cache size 0 max 16384 2024-06-06T04:32:49.667 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:32:49.667 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-06-06T04:32:49.667 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "99af536a-9204-40db-a3a4-1c808cd4d8f0", 2024-06-06T04:32:49.667 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-06-06T04:32:49.667 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:32:49.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.662+0000 7f7e3a276700 1 -- 172.21.15.45:0/3590234885 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7e1c0a2c20 msgr2=0x7f7e1c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.662+0000 7f7e3a276700 1 --2- 172.21.15.45:0/3590234885 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7e1c0a2c20 0x7f7e1c0a50e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7e34078910 tx=0x7f7e28005fb0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.662+0000 7f7e3a276700 1 -- 172.21.15.45:0/3590234885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e34101390 msgr2=0x7f7e3419ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.662+0000 7f7e3a276700 1 --2- 172.21.15.45:0/3590234885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e34101390 0x7f7e3419ad10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c920 tx=0x7f7e2400cc30 comp rx=0 tx=0).stop 2024-06-06T04:32:49.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.662+0000 7f7e3a276700 1 -- 172.21.15.45:0/3590234885 shutdown_connections 2024-06-06T04:32:49.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.662+0000 7f7e3a276700 1 --2- 172.21.15.45:0/3590234885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e341073d0 0x7f7e3419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.662+0000 7f7e3a276700 1 --2- 172.21.15.45:0/3590234885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e34101390 0x7f7e3419ad10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.662+0000 7f7e3a276700 1 --2- 172.21.15.45:0/3590234885 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f7e1c0a2c20 0x7f7e1c0a50e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.662+0000 7f7e3a276700 1 --2- 172.21.15.45:0/3590234885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e34101d30 0x7f7e3419b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.662+0000 7f7e3a276700 1 -- 172.21.15.45:0/3590234885 >> 172.21.15.45:0/3590234885 conn(0x7f7e340782c0 msgr2=0x7f7e34100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:49.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.662+0000 7f7e3a276700 1 -- 172.21.15.45:0/3590234885 shutdown_connections 2024-06-06T04:32:49.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:49.662+0000 7f7e3a276700 1 -- 172.21.15.45:0/3590234885 wait complete. 2024-06-06T04:32:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.852+0000 7f1c33222640 1 -- 172.21.15.45:0/3039423362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c2c1009a0 msgr2=0x7f1c2c100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.852+0000 7f1c33222640 1 --2- 172.21.15.45:0/3039423362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c2c1009a0 0x7f1c2c100dc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009900 tx=0x7f1c2002a410 comp rx=0 tx=0).stop 2024-06-06T04:32:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.852+0000 7f1c33222640 1 -- 172.21.15.45:0/3039423362 shutdown_connections 2024-06-06T04:32:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.852+0000 7f1c33222640 1 --2- 172.21.15.45:0/3039423362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c102550 0x7f1c2c108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.852+0000 7f1c33222640 1 --2- 172.21.15.45:0/3039423362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c2c101b90 0x7f1c2c102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.852+0000 7f1c33222640 1 --2- 172.21.15.45:0/3039423362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c2c1009a0 0x7f1c2c100dc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.852+0000 7f1c33222640 1 -- 172.21.15.45:0/3039423362 >> 172.21.15.45:0/3039423362 conn(0x7f1c2c0fbf10 msgr2=0x7f1c2c0fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.852+0000 7f1c33222640 1 -- 172.21.15.45:0/3039423362 shutdown_connections 2024-06-06T04:32:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.852+0000 7f1c33222640 1 -- 172.21.15.45:0/3039423362 wait complete. 2024-06-06T04:32:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.853+0000 7f1c33222640 1 Processor -- start 2024-06-06T04:32:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.853+0000 7f1c33222640 1 -- start start 2024-06-06T04:32:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.853+0000 7f1c33222640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c2c1009a0 0x7f1c2c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.853+0000 7f1c33222640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c101b90 0x7f1c2c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.853+0000 7f1c33222640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c2c102550 0x7f1c2c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.853+0000 7f1c33222640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c10d580 con 0x7f1c2c102550 2024-06-06T04:32:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.853+0000 7f1c33222640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c10d400 con 0x7f1c2c101b90 2024-06-06T04:32:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.853+0000 7f1c33222640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c10d700 con 0x7f1c2c1009a0 2024-06-06T04:32:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.853+0000 7f1c317bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c2c102550 0x7f1c2c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.853+0000 7f1c317bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c2c102550 0x7f1c2c19f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40204/0 (socket says 172.21.15.45:40204) 2024-06-06T04:32:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.853+0000 7f1c317bf640 1 -- 172.21.15.45:0/1521921285 learned_addr learned my addr 172.21.15.45:0/1521921285 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.854+0000 7f1c317bf640 1 -- 172.21.15.45:0/1521921285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c2c1009a0 msgr2=0x7f1c2c197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.854+0000 7f1c2bfff640 1 --2- 172.21.15.45:0/1521921285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c101b90 0x7f1c2c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.854+0000 7f1c30fbe640 1 --2- 172.21.15.45:0/1521921285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c2c1009a0 0x7f1c2c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.854+0000 7f1c317bf640 1 --2- 172.21.15.45:0/1521921285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c2c1009a0 0x7f1c2c197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.854+0000 7f1c317bf640 1 -- 172.21.15.45:0/1521921285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c101b90 msgr2=0x7f1c2c198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.854+0000 7f1c317bf640 1 --2- 172.21.15.45:0/1521921285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c101b90 0x7f1c2c198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.854+0000 7f1c317bf640 1 -- 172.21.15.45:0/1521921285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c19f760 con 0x7f1c2c102550 2024-06-06T04:32:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.854+0000 7f1c2bfff640 1 --2- 172.21.15.45:0/1521921285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c101b90 0x7f1c2c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.854+0000 7f1c30fbe640 1 --2- 172.21.15.45:0/1521921285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c2c1009a0 0x7f1c2c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.854+0000 7f1c317bf640 1 --2- 172.21.15.45:0/1521921285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c2c102550 0x7f1c2c19f0f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400cce0 tx=0x7f1c2400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.854+0000 7f1c29ffb640 1 -- 172.21.15.45:0/1521921285 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2400f450 con 0x7f1c2c102550 2024-06-06T04:32:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.854+0000 7f1c33222640 1 -- 172.21.15.45:0/1521921285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2c19fa50 con 0x7f1c2c102550 2024-06-06T04:32:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.854+0000 7f1c29ffb640 1 -- 172.21.15.45:0/1521921285 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c2400faa0 con 0x7f1c2c102550 2024-06-06T04:32:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.854+0000 7f1c29ffb640 1 -- 172.21.15.45:0/1521921285 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24018b70 con 0x7f1c2c102550 2024-06-06T04:32:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.854+0000 7f1c33222640 1 -- 172.21.15.45:0/1521921285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2c19ff50 con 0x7f1c2c102550 2024-06-06T04:32:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.856+0000 7f1c29ffb640 1 -- 172.21.15.45:0/1521921285 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1c24018d10 con 0x7f1c2c102550 2024-06-06T04:32:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.857+0000 7f1c36005740 1 Processor -- start 2024-06-06T04:32:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.857+0000 7f1c36005740 1 -- start start 2024-06-06T04:32:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.857+0000 7f1c36005740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 0x5633c5d5a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.857+0000 7f1c36005740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5aa00 0x5633c5d5f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.857+0000 7f1c29ffb640 1 --2- 172.21.15.45:0/1521921285 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1c1c0a2c30 0x7f1c1c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.857+0000 7f1c36005740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d62000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.857+0000 7f1c36005740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5633c5cbd4f0 con 0x5633c5d5fb70 2024-06-06T04:32:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.857+0000 7f1c36005740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5633c5c5b870 con 0x5633c5d5a010 2024-06-06T04:32:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.857+0000 7f1c36005740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5633c5cb94b0 con 0x5633c5d5aa00 2024-06-06T04:32:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.857+0000 7f1c317bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d62000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.857+0000 7f1c2bfff640 1 --2- 172.21.15.45:0/1521921285 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1c1c0a2c30 0x7f1c1c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.857+0000 7f1c317bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d62000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40216/0 (socket says 172.21.15.45:40216) 2024-06-06T04:32:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.857+0000 7f1c317bf640 1 -- 172.21.15.45:0/2451590588 learned_addr learned my addr 172.21.15.45:0/2451590588 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.857+0000 7f1c29ffb640 1 -- 172.21.15.45:0/1521921285 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f1c240cf920 con 0x7f1c2c102550 2024-06-06T04:32:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.857+0000 7f1c30fbe640 1 --2- 172.21.15.45:0/2451590588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5aa00 0x5633c5d5f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.857+0000 7f1c2bfff640 1 --2- 172.21.15.45:0/2451590588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 0x5633c5d5a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.858+0000 7f1c317bf640 1 -- 172.21.15.45:0/2451590588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5aa00 msgr2=0x5633c5d5f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.858+0000 7f1c317bf640 1 --2- 172.21.15.45:0/2451590588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5aa00 0x5633c5d5f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.858+0000 7f1c317bf640 1 -- 172.21.15.45:0/2451590588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 msgr2=0x5633c5d5a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.858+0000 7f1c317bf640 1 --2- 172.21.15.45:0/2451590588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 0x5633c5d5a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.858+0000 7f1c317bf640 1 -- 172.21.15.45:0/2451590588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633c5c779b0 con 0x5633c5d5fb70 2024-06-06T04:32:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.858+0000 7f1c30fbe640 1 --2- 172.21.15.45:0/2451590588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5aa00 0x5633c5d5f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.858+0000 7f1c2bfff640 1 --2- 172.21.15.45:0/1521921285 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1c1c0a2c30 0x7f1c1c0a50f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c18005d00 tx=0x7f1c180127c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.858+0000 7f1c317bf640 1 --2- 172.21.15.45:0/2451590588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d62000 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400ccb0 tx=0x7f1c2401a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db602b8ba03e1cb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.858+0000 7f1c16ffd640 1 -- 172.21.15.45:0/2451590588 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2400fc50 con 0x5633c5d5fb70 2024-06-06T04:32:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.858+0000 7f1c16ffd640 1 -- 172.21.15.45:0/2451590588 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c2409c4c0 con 0x5633c5d5fb70 2024-06-06T04:32:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.858+0000 7f1c16ffd640 1 -- 172.21.15.45:0/2451590588 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c240a3290 con 0x5633c5d5fb70 2024-06-06T04:32:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.859+0000 7f1c36005740 1 -- 172.21.15.45:0/2451590588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 msgr2=0x5633c5d62000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.859+0000 7f1c36005740 1 --2- 172.21.15.45:0/2451590588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d62000 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400ccb0 tx=0x7f1c2401a850 comp rx=0 tx=0).stop 2024-06-06T04:32:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.859+0000 7f1c36005740 1 -- 172.21.15.45:0/2451590588 shutdown_connections 2024-06-06T04:32:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.859+0000 7f1c36005740 1 --2- 172.21.15.45:0/2451590588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d62000 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.859+0000 7f1c36005740 1 --2- 172.21.15.45:0/2451590588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5aa00 0x5633c5d5f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.859+0000 7f1c36005740 1 --2- 172.21.15.45:0/2451590588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 0x5633c5d5a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.859+0000 7f1c36005740 1 -- 172.21.15.45:0/2451590588 >> 172.21.15.45:0/2451590588 conn(0x5633c5d55680 msgr2=0x5633c5d55a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.859+0000 7f1c36005740 1 -- 172.21.15.45:0/2451590588 shutdown_connections 2024-06-06T04:32:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.859+0000 7f1c36005740 1 -- 172.21.15.45:0/2451590588 wait complete. 2024-06-06T04:32:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.859+0000 7f1c36005740 1 -- start start 2024-06-06T04:32:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.859+0000 7f1c36005740 1 Processor -- start 2024-06-06T04:32:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.859+0000 7f1c36005740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5a010 0x5633c5d64260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.859+0000 7f1c36005740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5aa00 0x5633c5d647a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.859+0000 7f1c36005740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d6b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.859+0000 7f1c36005740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5633c5d62b20 con 0x5633c5d5fb70 2024-06-06T04:32:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c36005740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5633c5c46ae0 con 0x5633c5d5aa00 2024-06-06T04:32:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c36005740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5633c5d62ca0 con 0x5633c5d5a010 2024-06-06T04:32:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c36005740 10 client.0 _ll_register_callbacks cb 0x5633c5c7f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c317bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d6b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c317bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d6b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40224/0 (socket says 172.21.15.45:40224) 2024-06-06T04:32:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c317bf640 1 -- 172.21.15.45:0/2295202808 learned_addr learned my addr 172.21.15.45:0/2295202808 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c2bfff640 1 --2- 172.21.15.45:0/2295202808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5aa00 0x5633c5d647a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c30fbe640 1 --2- 172.21.15.45:0/2295202808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5a010 0x5633c5d64260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c317bf640 1 -- 172.21.15.45:0/2295202808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5a010 msgr2=0x5633c5d64260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c317bf640 1 --2- 172.21.15.45:0/2295202808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5a010 0x5633c5d64260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c317bf640 1 -- 172.21.15.45:0/2295202808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5aa00 msgr2=0x5633c5d647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c317bf640 1 --2- 172.21.15.45:0/2295202808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5aa00 0x5633c5d647a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c317bf640 1 -- 172.21.15.45:0/2295202808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633c59d1210 con 0x5633c5d5fb70 2024-06-06T04:32:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c2bfff640 1 --2- 172.21.15.45:0/2295202808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5aa00 0x5633c5d647a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c30fbe640 1 --2- 172.21.15.45:0/2295202808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5a010 0x5633c5d64260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c317bf640 1 --2- 172.21.15.45:0/2295202808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d6b7e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1c240cd3d0 tx=0x7f1c2400fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8579eaf17879d2c4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c0bfff640 1 -- 172.21.15.45:0/2295202808 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2406b020 con 0x5633c5d5fb70 2024-06-06T04:32:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c0bfff640 1 -- 172.21.15.45:0/2295202808 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f1c240a47a0 con 0x5633c5d5fb70 2024-06-06T04:32:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c36005740 1 -- 172.21.15.45:0/2295202808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c2409c4c0 con 0x5633c5d5fb70 2024-06-06T04:32:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c0bfff640 1 -- 172.21.15.45:0/2295202808 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c2409f070 con 0x5633c5d5fb70 2024-06-06T04:32:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c36005740 10 client.15615 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.860+0000 7f1c36005740 1 -- 172.21.15.45:0/2295202808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633c5a9d620 con 0x5633c5d5fb70 2024-06-06T04:32:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.861+0000 7f1c0bfff640 1 -- 172.21.15.45:0/2295202808 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c2409c4c0 con 0x5633c5d5fb70 2024-06-06T04:32:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.861+0000 7f1c0bfff640 1 -- 172.21.15.45:0/2295202808 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1c2409b210 con 0x5633c5d5fb70 2024-06-06T04:32:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.861+0000 7f1c36005740 10 client.15615 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.861+0000 7f1c36005740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.861+0000 7f1c36005740 20 client.15615 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.861+0000 7f1c36005740 1 --2- 172.21.15.45:0/2295202808 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5633c5d6fcd0 0x5633c5d720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.861+0000 7f1c36005740 4 client.15615 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:32:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.861+0000 7f1c36005740 1 -- 172.21.15.45:0/2295202808 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633c5cd0f10 con 0x5633c5d6fcd0 2024-06-06T04:32:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.861+0000 7f1c30fbe640 1 --2- 172.21.15.45:0/2295202808 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5633c5d6fcd0 0x5633c5d720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.861+0000 7f1c30fbe640 1 --2- 172.21.15.45:0/2295202808 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5633c5d6fcd0 0x5633c5d720c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64422203d2874e98 server_cookie=1a159cfb9994b141 in_seq=0 out_seq=0 2024-06-06T04:32:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.861+0000 7f1c0bfff640 10 client.15615 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.862+0000 7f1c0bfff640 1 -- 172.21.15.45:0/2295202808 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5633c5cd0f10 con 0x5633c5d6fcd0 2024-06-06T04:32:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.862+0000 7f1c0bfff640 10 client.15615 handle_command_reply: tid=0 2024-06-06T04:32:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.862+0000 7f1c36005740 1 client.15615 shutdown 2024-06-06T04:32:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.862+0000 7f1c36005740 1 -- 172.21.15.45:0/2295202808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 msgr2=0x5633c5d6b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.862+0000 7f1c36005740 1 --2- 172.21.15.45:0/2295202808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d6b7e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1c240cd3d0 tx=0x7f1c2400fc50 comp rx=0 tx=0).stop 2024-06-06T04:32:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.862+0000 7f1c36005740 1 -- 172.21.15.45:0/2295202808 shutdown_connections 2024-06-06T04:32:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.862+0000 7f1c36005740 1 --2- 172.21.15.45:0/2295202808 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5633c5d6fcd0 0x5633c5d720c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.862+0000 7f1c36005740 1 --2- 172.21.15.45:0/2295202808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5aa00 0x5633c5d647a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.862+0000 7f1c36005740 1 --2- 172.21.15.45:0/2295202808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d6b7e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.862+0000 7f1c36005740 1 --2- 172.21.15.45:0/2295202808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5a010 0x5633c5d64260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.862+0000 7f1c0bfff640 0 client.15615 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.862+0000 7f1c36005740 1 -- 172.21.15.45:0/2295202808 >> 172.21.15.45:0/2295202808 conn(0x5633c5d55680 msgr2=0x5633c5d547e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.862+0000 7f1c36005740 1 -- 172.21.15.45:0/2295202808 shutdown_connections 2024-06-06T04:32:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.862+0000 7f1c36005740 1 -- 172.21.15.45:0/2295202808 wait complete. 2024-06-06T04:32:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.862+0000 7f1c36005740 20 client.15615 trim_cache size 0 max 16384 2024-06-06T04:32:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c36005740 1 Processor -- start 2024-06-06T04:32:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c36005740 1 -- start start 2024-06-06T04:32:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c36005740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 0x5633c5d5bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c36005740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5aa00 0x5633c5d5c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c36005740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d6b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c36005740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c1acda0 con 0x5633c5d5fb70 2024-06-06T04:32:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c36005740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c1aa670 con 0x5633c5d5a010 2024-06-06T04:32:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c36005740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c1a84c0 con 0x5633c5d5aa00 2024-06-06T04:32:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c317bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d6b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c317bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d6b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40232/0 (socket says 172.21.15.45:40232) 2024-06-06T04:32:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c317bf640 1 -- 172.21.15.45:0/1568896468 learned_addr learned my addr 172.21.15.45:0/1568896468 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c2bfff640 1 --2- 172.21.15.45:0/1568896468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5aa00 0x5633c5d5c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c30fbe640 1 --2- 172.21.15.45:0/1568896468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 0x5633c5d5bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c317bf640 1 -- 172.21.15.45:0/1568896468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5aa00 msgr2=0x5633c5d5c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c317bf640 1 --2- 172.21.15.45:0/1568896468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5aa00 0x5633c5d5c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c317bf640 1 -- 172.21.15.45:0/1568896468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 msgr2=0x5633c5d5bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c317bf640 1 --2- 172.21.15.45:0/1568896468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 0x5633c5d5bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c317bf640 1 -- 172.21.15.45:0/1568896468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633c5d554b0 con 0x5633c5d5fb70 2024-06-06T04:32:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c2bfff640 1 --2- 172.21.15.45:0/1568896468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5aa00 0x5633c5d5c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c30fbe640 1 --2- 172.21.15.45:0/1568896468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 0x5633c5d5bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.878+0000 7f1c317bf640 1 --2- 172.21.15.45:0/1568896468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d6b7a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1c24098740 tx=0x7f1c24018400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c02a8a3554444 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.879+0000 7f1c167fc640 1 -- 172.21.15.45:0/1568896468 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2406b040 con 0x5633c5d5fb70 2024-06-06T04:32:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.879+0000 7f1c167fc640 1 -- 172.21.15.45:0/1568896468 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c24097b10 con 0x5633c5d5fb70 2024-06-06T04:32:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.879+0000 7f1c167fc640 1 -- 172.21.15.45:0/1568896468 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2406b440 con 0x5633c5d5fb70 2024-06-06T04:32:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.879+0000 7f1c36005740 1 -- 172.21.15.45:0/1568896468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 msgr2=0x5633c5d6b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.879+0000 7f1c36005740 1 --2- 172.21.15.45:0/1568896468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d6b7a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1c24098740 tx=0x7f1c24018400 comp rx=0 tx=0).stop 2024-06-06T04:32:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.879+0000 7f1c36005740 1 -- 172.21.15.45:0/1568896468 shutdown_connections 2024-06-06T04:32:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.879+0000 7f1c36005740 1 --2- 172.21.15.45:0/1568896468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5fb70 0x5633c5d6b7a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.879+0000 7f1c36005740 1 --2- 172.21.15.45:0/1568896468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5aa00 0x5633c5d5c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.879+0000 7f1c36005740 1 --2- 172.21.15.45:0/1568896468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 0x5633c5d5bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.879+0000 7f1c36005740 1 -- 172.21.15.45:0/1568896468 >> 172.21.15.45:0/1568896468 conn(0x5633c5d55680 msgr2=0x5633c5d64260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.879+0000 7f1c36005740 1 -- 172.21.15.45:0/1568896468 shutdown_connections 2024-06-06T04:32:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.879+0000 7f1c36005740 1 -- 172.21.15.45:0/1568896468 wait complete. 2024-06-06T04:32:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.879+0000 7f1c36005740 1 -- start start 2024-06-06T04:32:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c36005740 1 Processor -- start 2024-06-06T04:32:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c36005740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 0x5633c5d68370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c36005740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5aa00 0x5633c5d688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c36005740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5fb70 0x5633c5dd88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c36005740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c1a8b30 con 0x5633c5d5aa00 2024-06-06T04:32:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c36005740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c1a9fa0 con 0x5633c5d5a010 2024-06-06T04:32:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c36005740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c1a9200 con 0x5633c5d5fb70 2024-06-06T04:32:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c36005740 10 client.0 _ll_register_callbacks cb 0x7f1c2409bc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c2bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5aa00 0x5633c5d688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c2bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5aa00 0x5633c5d688b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40246/0 (socket says 172.21.15.45:40246) 2024-06-06T04:32:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c2bfff640 1 -- 172.21.15.45:0/3993696397 learned_addr learned my addr 172.21.15.45:0/3993696397 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c30fbe640 1 --2- 172.21.15.45:0/3993696397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 0x5633c5d68370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c2bfff640 1 -- 172.21.15.45:0/3993696397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5fb70 msgr2=0x5633c5dd88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c317bf640 1 --2- 172.21.15.45:0/3993696397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5fb70 0x5633c5dd88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c2bfff640 1 --2- 172.21.15.45:0/3993696397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5fb70 0x5633c5dd88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c2bfff640 1 -- 172.21.15.45:0/3993696397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 msgr2=0x5633c5d68370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c2bfff640 1 --2- 172.21.15.45:0/3993696397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 0x5633c5d68370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c2bfff640 1 -- 172.21.15.45:0/3993696397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633c5d6b910 con 0x5633c5d5aa00 2024-06-06T04:32:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.880+0000 7f1c30fbe640 1 --2- 172.21.15.45:0/3993696397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 0x5633c5d68370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:32:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.881+0000 7f1c2bfff640 1 --2- 172.21.15.45:0/3993696397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5aa00 0x5633c5d688b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800be00 tx=0x7f1c18005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7541cfca925c708e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.881+0000 7f1c0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.881+0000 7f1c0bfff640 1 -- 172.21.15.45:0/3993696397 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18024070 con 0x5633c5d5aa00 2024-06-06T04:32:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.881+0000 7f1c0bfff640 1 -- 172.21.15.45:0/3993696397 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f1c18020810 con 0x5633c5d5aa00 2024-06-06T04:32:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.881+0000 7f1c36005740 1 -- 172.21.15.45:0/3993696397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c24097b10 con 0x5633c5d5aa00 2024-06-06T04:32:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.881+0000 7f1c0bfff640 1 -- 172.21.15.45:0/3993696397 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c18020a10 con 0x5633c5d5aa00 2024-06-06T04:32:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.881+0000 7f1c36005740 10 client.15621 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.881+0000 7f1c36005740 1 -- 172.21.15.45:0/3993696397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633c5dd9cb0 con 0x5633c5d5aa00 2024-06-06T04:32:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.881+0000 7f1c0bfff640 1 -- 172.21.15.45:0/3993696397 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c24097b10 con 0x5633c5d5aa00 2024-06-06T04:32:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.881+0000 7f1c0bfff640 1 -- 172.21.15.45:0/3993696397 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1c1800f040 con 0x5633c5d5aa00 2024-06-06T04:32:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.881+0000 7f1c36005740 10 client.15621 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.881+0000 7f1c36005740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.881+0000 7f1c36005740 20 client.15621 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.881+0000 7f1c36005740 1 --2- 172.21.15.45:0/3993696397 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5633c5dda710 0x5633c5ddcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:49.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.881+0000 7f1c36005740 4 client.15621 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T04:32:49.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.881+0000 7f1c36005740 1 -- 172.21.15.45:0/3993696397 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5633c5ccac20 con 0x5633c5dda710 2024-06-06T04:32:49.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.882+0000 7f1c30fbe640 1 --2- 172.21.15.45:0/3993696397 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5633c5dda710 0x5633c5ddcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:49.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.882+0000 7f1c0bfff640 10 client.15621 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:49.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.882+0000 7f1c30fbe640 1 --2- 172.21.15.45:0/3993696397 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5633c5dda710 0x5633c5ddcb30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8077a428db987c86 server_cookie=1e6c54c9bab44140 in_seq=0 out_seq=0 2024-06-06T04:32:49.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.885+0000 7f1c0bfff640 1 -- 172.21.15.45:0/3993696397 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5633c5ccac20 con 0x5633c5dda710 2024-06-06T04:32:49.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.885+0000 7f1c0bfff640 10 client.15621 handle_command_reply: tid=0 2024-06-06T04:32:49.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.885+0000 7f1c36005740 1 client.15621 shutdown 2024-06-06T04:32:49.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c36005740 1 -- 172.21.15.45:0/3993696397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5aa00 msgr2=0x5633c5d688b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c36005740 1 --2- 172.21.15.45:0/3993696397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5aa00 0x5633c5d688b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800be00 tx=0x7f1c18005c90 comp rx=0 tx=0).stop 2024-06-06T04:32:49.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c36005740 1 -- 172.21.15.45:0/3993696397 shutdown_connections 2024-06-06T04:32:49.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c36005740 1 --2- 172.21.15.45:0/3993696397 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5633c5dda710 0x5633c5ddcb30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c36005740 1 --2- 172.21.15.45:0/3993696397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633c5d5fb70 0x5633c5dd88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c36005740 1 --2- 172.21.15.45:0/3993696397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633c5d5aa00 0x5633c5d688b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c0bfff640 0 client.15621 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c36005740 1 --2- 172.21.15.45:0/3993696397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5633c5d5a010 0x5633c5d68370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c36005740 1 -- 172.21.15.45:0/3993696397 >> 172.21.15.45:0/3993696397 conn(0x5633c5d55680 msgr2=0x5633c5d64260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c36005740 1 -- 172.21.15.45:0/3993696397 shutdown_connections 2024-06-06T04:32:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c36005740 1 -- 172.21.15.45:0/3993696397 wait complete. 2024-06-06T04:32:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c36005740 20 client.15621 trim_cache size 0 max 16384 2024-06-06T04:32:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-06-06T04:32:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-06-06T04:32:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-06-06T04:32:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c33222640 1 -- 172.21.15.45:0/1521921285 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1c1c0a2c30 msgr2=0x7f1c1c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c33222640 1 --2- 172.21.15.45:0/1521921285 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1c1c0a2c30 0x7f1c1c0a50f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c18005d00 tx=0x7f1c180127c0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c33222640 1 -- 172.21.15.45:0/1521921285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c2c102550 msgr2=0x7f1c2c19f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c33222640 1 --2- 172.21.15.45:0/1521921285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c2c102550 0x7f1c2c19f0f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400cce0 tx=0x7f1c2400a620 comp rx=0 tx=0).stop 2024-06-06T04:32:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c33222640 1 -- 172.21.15.45:0/1521921285 shutdown_connections 2024-06-06T04:32:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c33222640 1 --2- 172.21.15.45:0/1521921285 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f1c1c0a2c30 0x7f1c1c0a50f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c33222640 1 --2- 172.21.15.45:0/1521921285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c2c101b90 0x7f1c2c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c33222640 1 --2- 172.21.15.45:0/1521921285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c2c102550 0x7f1c2c19f0f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c33222640 1 --2- 172.21.15.45:0/1521921285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c2c1009a0 0x7f1c2c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.886+0000 7f1c33222640 1 -- 172.21.15.45:0/1521921285 >> 172.21.15.45:0/1521921285 conn(0x7f1c2c0fbf10 msgr2=0x7f1c2c104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.887+0000 7f1c33222640 1 -- 172.21.15.45:0/1521921285 shutdown_connections 2024-06-06T04:32:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:32:49.887+0000 7f1c33222640 1 -- 172.21.15.45:0/1521921285 wait complete. 2024-06-06T04:32:50.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:50 smithi086 ceph-mon[36053]: pgmap v472: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 8 op/s 2024-06-06T04:32:50.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:50 smithi154 ceph-mon[36428]: pgmap v472: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 8 op/s 2024-06-06T04:32:50.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:50 smithi045 ceph-mon[32299]: pgmap v472: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 8 op/s 2024-06-06T04:32:50.577 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T04:32:50.714 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-06T04:32:50.771 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-06T04:32:50.819 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-06T04:32:50.821 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-06T04:32:50.821 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-06T04:32:50.835 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:32:50.868 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-06T04:32:50.872 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:32:50.872 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-06T04:32:50.889 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-06-06T04:32:50.890 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-06-06T04:32:50.891 DEBUG:teuthology.orchestra.run.smithi045:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d746ca5346f34eef7e826a1eb71e59e02b682ae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-06-06T04:32:50.955 INFO:tasks.workunit.client.0.smithi045.stderr:+ set -e 2024-06-06T04:32:50.955 INFO:tasks.workunit.client.0.smithi045.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-06-06T04:32:50.959 INFO:tasks.workunit.client.0.smithi045.stderr:Cloning into 'xfstests-dev'... 2024-06-06T04:32:51.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:51 smithi045 ceph-mon[32299]: scrub queued for path: / 2024-06-06T04:32:51.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:51 smithi045 ceph-mon[32299]: scrub summary: idle+waiting paths [/] 2024-06-06T04:32:51.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:51 smithi045 ceph-mon[32299]: scrub summary: active paths [/] 2024-06-06T04:32:51.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:51 smithi045 ceph-mon[32299]: scrub summary: idle+waiting paths [/] 2024-06-06T04:32:51.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:51 smithi086 ceph-mon[36053]: scrub queued for path: / 2024-06-06T04:32:51.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:51 smithi086 ceph-mon[36053]: scrub summary: idle+waiting paths [/] 2024-06-06T04:32:51.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:51 smithi086 ceph-mon[36053]: scrub summary: active paths [/] 2024-06-06T04:32:51.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:51 smithi086 ceph-mon[36053]: scrub summary: idle+waiting paths [/] 2024-06-06T04:32:51.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:51 smithi154 ceph-mon[36428]: scrub queued for path: / 2024-06-06T04:32:51.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:51 smithi154 ceph-mon[36428]: scrub summary: idle+waiting paths [/] 2024-06-06T04:32:51.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:51 smithi154 ceph-mon[36428]: scrub summary: active paths [/] 2024-06-06T04:32:51.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:51 smithi154 ceph-mon[36428]: scrub summary: idle+waiting paths [/] 2024-06-06T04:32:52.313 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:52 smithi045 ceph-mon[32299]: pgmap v473: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:52.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:52 smithi086 ceph-mon[36053]: pgmap v473: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:52.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:52 smithi154 ceph-mon[36428]: pgmap v473: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-06T04:32:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.594+0000 7f73f209c700 1 -- 172.21.15.45:0/1685841926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73ec1017f0 msgr2=0x7f73ec10f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.594+0000 7f73f209c700 1 --2- 172.21.15.45:0/1685841926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73ec1017f0 0x7f73ec10f380 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f73e400b0d0 tx=0x7f73e400b3e0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.595+0000 7f73f209c700 1 -- 172.21.15.45:0/1685841926 shutdown_connections 2024-06-06T04:32:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.595+0000 7f73f209c700 1 --2- 172.21.15.45:0/1685841926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73ec1017f0 0x7f73ec10f380 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.595+0000 7f73f209c700 1 --2- 172.21.15.45:0/1685841926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73ec107500 0x7f73ec1078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.595+0000 7f73f209c700 1 --2- 172.21.15.45:0/1685841926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73ec100e30 0x7f73ec1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.595+0000 7f73f209c700 1 -- 172.21.15.45:0/1685841926 >> 172.21.15.45:0/1685841926 conn(0x7f73ec076730 msgr2=0x7f73ec076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.595+0000 7f73f209c700 1 -- 172.21.15.45:0/1685841926 shutdown_connections 2024-06-06T04:32:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.595+0000 7f73f209c700 1 -- 172.21.15.45:0/1685841926 wait complete. 2024-06-06T04:32:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.595+0000 7f73f209c700 1 Processor -- start 2024-06-06T04:32:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.596+0000 7f73f209c700 1 -- start start 2024-06-06T04:32:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.596+0000 7f73f209c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73ec100e30 0x7f73ec1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.596+0000 7f73f209c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73ec1017f0 0x7f73ec1a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.596+0000 7f73f209c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73ec107500 0x7f73ec19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.596+0000 7f73f209c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec112650 con 0x7f73ec1017f0 2024-06-06T04:32:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.596+0000 7f73f209c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1124d0 con 0x7f73ec100e30 2024-06-06T04:32:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.596+0000 7f73f209c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1127d0 con 0x7f73ec107500 2024-06-06T04:32:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.596+0000 7f73eaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73ec1017f0 0x7f73ec1a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.596+0000 7f73eaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73ec1017f0 0x7f73ec1a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40260/0 (socket says 172.21.15.45:40260) 2024-06-06T04:32:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.596+0000 7f73eaffd700 1 -- 172.21.15.45:0/646302901 learned_addr learned my addr 172.21.15.45:0/646302901 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.596+0000 7f73eb7fe700 1 --2- 172.21.15.45:0/646302901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73ec100e30 0x7f73ec1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.596+0000 7f73ebfff700 1 --2- 172.21.15.45:0/646302901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73ec107500 0x7f73ec19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.596+0000 7f73eaffd700 1 -- 172.21.15.45:0/646302901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73ec107500 msgr2=0x7f73ec19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.596+0000 7f73eaffd700 1 --2- 172.21.15.45:0/646302901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73ec107500 0x7f73ec19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.596+0000 7f73eaffd700 1 -- 172.21.15.45:0/646302901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73ec100e30 msgr2=0x7f73ec1a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.596+0000 7f73eaffd700 1 --2- 172.21.15.45:0/646302901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73ec100e30 0x7f73ec1a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.597+0000 7f73eaffd700 1 -- 172.21.15.45:0/646302901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec19b1b0 con 0x7f73ec1017f0 2024-06-06T04:32:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.597+0000 7f73eaffd700 1 --2- 172.21.15.45:0/646302901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73ec1017f0 0x7f73ec1a0dd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f73e000d980 tx=0x7f73e000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.597+0000 7f73e8ff9700 1 -- 172.21.15.45:0/646302901 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0016070 con 0x7f73ec1017f0 2024-06-06T04:32:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.597+0000 7f73e8ff9700 1 -- 172.21.15.45:0/646302901 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73e00044d0 con 0x7f73ec1017f0 2024-06-06T04:32:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.597+0000 7f73f209c700 1 -- 172.21.15.45:0/646302901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec19b4a0 con 0x7f73ec1017f0 2024-06-06T04:32:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.597+0000 7f73e8ff9700 1 -- 172.21.15.45:0/646302901 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0004cf0 con 0x7f73ec1017f0 2024-06-06T04:32:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.597+0000 7f73f209c700 1 -- 172.21.15.45:0/646302901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec1a7760 con 0x7f73ec1017f0 2024-06-06T04:32:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.600+0000 7f73e8ff9700 1 -- 172.21.15.45:0/646302901 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f73e0004020 con 0x7f73ec1017f0 2024-06-06T04:32:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.600+0000 7f73e8ff9700 1 --2- 172.21.15.45:0/646302901 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73d40a2ca0 0x7f73d40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.601+0000 7f73eb7fe700 1 --2- 172.21.15.45:0/646302901 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73d40a2ca0 0x7f73d40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.601+0000 7f73e8ff9700 1 -- 172.21.15.45:0/646302901 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f73e00cf260 con 0x7f73ec1017f0 2024-06-06T04:32:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.601+0000 7f73fc91fb80 1 Processor -- start 2024-06-06T04:32:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.601+0000 7f73fc91fb80 1 -- start start 2024-06-06T04:32:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.601+0000 7f73eb7fe700 1 --2- 172.21.15.45:0/646302901 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73d40a2ca0 0x7f73d40a5160 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f73dc005fd0 tx=0x7f73dc005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.601+0000 7f73fc91fb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b535428f60 0x55b535429340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.601+0000 7f73fc91fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 0x55b535423260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.601+0000 7f73fc91fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5354237a0 0x55b535423bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.601+0000 7f73fc91fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b535315f60 con 0x55b5354237a0 2024-06-06T04:32:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.601+0000 7f73fc91fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b535374850 con 0x55b535429880 2024-06-06T04:32:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.601+0000 7f73fc91fb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b5352dc460 con 0x55b535428f60 2024-06-06T04:32:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.602+0000 7f73ebfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5354237a0 0x55b535423bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.602+0000 7f73ebfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5354237a0 0x55b535423bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40270/0 (socket says 172.21.15.45:40270) 2024-06-06T04:32:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.602+0000 7f73ebfff700 1 -- 172.21.15.45:0/3260136240 learned_addr learned my addr 172.21.15.45:0/3260136240 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.602+0000 7f73eaffd700 1 --2- 172.21.15.45:0/3260136240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 0x55b535423260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.602+0000 7f73eb7fe700 1 --2- 172.21.15.45:0/3260136240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b535428f60 0x55b535429340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.602+0000 7f73ebfff700 1 -- 172.21.15.45:0/3260136240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b535428f60 msgr2=0x55b535429340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.602+0000 7f73ebfff700 1 --2- 172.21.15.45:0/3260136240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b535428f60 0x55b535429340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.602+0000 7f73ebfff700 1 -- 172.21.15.45:0/3260136240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 msgr2=0x55b535423260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.602+0000 7f73ebfff700 1 --2- 172.21.15.45:0/3260136240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 0x55b535423260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.602+0000 7f73ebfff700 1 -- 172.21.15.45:0/3260136240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5352c38d0 con 0x55b5354237a0 2024-06-06T04:32:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.602+0000 7f73eaffd700 1 --2- 172.21.15.45:0/3260136240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 0x55b535423260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.602+0000 7f73ebfff700 1 --2- 172.21.15.45:0/3260136240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5354237a0 0x55b535423bc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009c30 tx=0x7f73e4009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccffc2eea2b1c735 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.602+0000 7f73d1ffb700 1 -- 172.21.15.45:0/3260136240 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4020030 con 0x55b5354237a0 2024-06-06T04:32:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.602+0000 7f73d1ffb700 1 -- 172.21.15.45:0/3260136240 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73e4004a50 con 0x55b5354237a0 2024-06-06T04:32:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.602+0000 7f73d1ffb700 1 -- 172.21.15.45:0/3260136240 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4007c30 con 0x55b5354237a0 2024-06-06T04:32:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.603+0000 7f73fc91fb80 1 -- 172.21.15.45:0/3260136240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5354237a0 msgr2=0x55b535423bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.603+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/3260136240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5354237a0 0x55b535423bc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009c30 tx=0x7f73e4009c60 comp rx=0 tx=0).stop 2024-06-06T04:32:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.603+0000 7f73fc91fb80 1 -- 172.21.15.45:0/3260136240 shutdown_connections 2024-06-06T04:32:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.603+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/3260136240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b5354237a0 0x55b535423bc0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.603+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/3260136240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b535428f60 0x55b535429340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.603+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/3260136240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 0x55b535423260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.603+0000 7f73fc91fb80 1 -- 172.21.15.45:0/3260136240 >> 172.21.15.45:0/3260136240 conn(0x55b535350080 msgr2=0x55b53541e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.603+0000 7f73fc91fb80 1 -- 172.21.15.45:0/3260136240 shutdown_connections 2024-06-06T04:32:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.604+0000 7f73fc91fb80 1 -- 172.21.15.45:0/3260136240 wait complete. 2024-06-06T04:32:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.604+0000 7f73fc91fb80 1 -- start start 2024-06-06T04:32:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.604+0000 7f73fc91fb80 1 Processor -- start 2024-06-06T04:32:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.604+0000 7f73fc91fb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b5354237a0 0x55b53542f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.604+0000 7f73fc91fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535428f60 0x55b53542c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.604+0000 7f73fc91fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 0x55b53542ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73fc91fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b535314fc0 con 0x55b535428f60 2024-06-06T04:32:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73fc91fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b535370810 con 0x55b535429880 2024-06-06T04:32:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73fc91fb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b535430200 con 0x55b5354237a0 2024-06-06T04:32:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73fc91fb80 10 client.0 _ll_register_callbacks cb 0x55b53530a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73eaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535428f60 0x55b53542c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73eaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535428f60 0x55b53542c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40274/0 (socket says 172.21.15.45:40274) 2024-06-06T04:32:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73eaffd700 1 -- 172.21.15.45:0/4155851203 learned_addr learned my addr 172.21.15.45:0/4155851203 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73eb7fe700 1 --2- 172.21.15.45:0/4155851203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b5354237a0 0x55b53542f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73ebfff700 1 --2- 172.21.15.45:0/4155851203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 0x55b53542ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73eaffd700 1 -- 172.21.15.45:0/4155851203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b5354237a0 msgr2=0x55b53542f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73eaffd700 1 --2- 172.21.15.45:0/4155851203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b5354237a0 0x55b53542f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73eaffd700 1 -- 172.21.15.45:0/4155851203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 msgr2=0x55b53542ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73eaffd700 1 --2- 172.21.15.45:0/4155851203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 0x55b53542ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73eaffd700 1 -- 172.21.15.45:0/4155851203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b53533a750 con 0x55b535428f60 2024-06-06T04:32:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73eb7fe700 1 --2- 172.21.15.45:0/4155851203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b5354237a0 0x55b53542f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73eaffd700 1 --2- 172.21.15.45:0/4155851203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535428f60 0x55b53542c8d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f73e009b400 tx=0x7f73e0097ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becf3ce3b41316d2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73caffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:32:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73caffd700 1 -- 172.21.15.45:0/4155851203 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e00cce60 con 0x55b535428f60 2024-06-06T04:32:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73caffd700 1 -- 172.21.15.45:0/4155851203 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f73e000bbf0 con 0x55b535428f60 2024-06-06T04:32:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.605+0000 7f73fc91fb80 1 -- 172.21.15.45:0/4155851203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e4004a50 con 0x55b535428f60 2024-06-06T04:32:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.606+0000 7f73caffd700 1 -- 172.21.15.45:0/4155851203 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f73e00b9050 con 0x55b535428f60 2024-06-06T04:32:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.606+0000 7f73fc91fb80 10 client.15633 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.606+0000 7f73fc91fb80 1 -- 172.21.15.45:0/4155851203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b53543a9c0 con 0x55b535428f60 2024-06-06T04:32:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.606+0000 7f73caffd700 1 -- 172.21.15.45:0/4155851203 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73e4004a50 con 0x55b535428f60 2024-06-06T04:32:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.606+0000 7f73caffd700 1 -- 172.21.15.45:0/4155851203 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f73e009b960 con 0x55b535428f60 2024-06-06T04:32:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.606+0000 7f73fc91fb80 10 client.15633 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.606+0000 7f73fc91fb80 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.606+0000 7f73fc91fb80 20 client.15633 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.606+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/4155851203 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55b53543cf50 0x55b53543f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.606+0000 7f73fc91fb80 4 client.15633 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:32:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.606+0000 7f73fc91fb80 1 -- 172.21.15.45:0/4155851203 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b535381d50 con 0x55b53543cf50 2024-06-06T04:32:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.607+0000 7f73eb7fe700 1 --2- 172.21.15.45:0/4155851203 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55b53543cf50 0x55b53543f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.607+0000 7f73caffd700 10 client.15633 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.607+0000 7f73eb7fe700 1 --2- 172.21.15.45:0/4155851203 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55b53543cf50 0x55b53543f340 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7d7d01013de24fc server_cookie=ead2c3fd3b3ae123 in_seq=0 out_seq=0 2024-06-06T04:32:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.608+0000 7f73caffd700 1 -- 172.21.15.45:0/4155851203 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b535381d50 con 0x55b53543cf50 2024-06-06T04:32:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.608+0000 7f73caffd700 10 client.15633 handle_command_reply: tid=0 2024-06-06T04:32:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.608+0000 7f73fc91fb80 1 client.15633 shutdown 2024-06-06T04:32:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.608+0000 7f73fc91fb80 1 -- 172.21.15.45:0/4155851203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535428f60 msgr2=0x55b53542c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.608+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/4155851203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535428f60 0x55b53542c8d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f73e009b400 tx=0x7f73e0097ab0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.608+0000 7f73fc91fb80 1 -- 172.21.15.45:0/4155851203 shutdown_connections 2024-06-06T04:32:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.608+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/4155851203 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55b53543cf50 0x55b53543f340 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.608+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/4155851203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 0x55b53542ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.608+0000 7f73caffd700 0 client.15633 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.608+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/4155851203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b5354237a0 0x55b53542f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.608+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/4155851203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535428f60 0x55b53542c8d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.608+0000 7f73fc91fb80 1 -- 172.21.15.45:0/4155851203 >> 172.21.15.45:0/4155851203 conn(0x55b535350080 msgr2=0x55b53541d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:52.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.608+0000 7f73fc91fb80 1 -- 172.21.15.45:0/4155851203 shutdown_connections 2024-06-06T04:32:52.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.608+0000 7f73fc91fb80 1 -- 172.21.15.45:0/4155851203 wait complete. 2024-06-06T04:32:52.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.608+0000 7f73fc91fb80 20 client.15633 trim_cache size 0 max 16384 2024-06-06T04:32:52.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.636+0000 7f73fc91fb80 1 Processor -- start 2024-06-06T04:32:52.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.636+0000 7f73fc91fb80 1 -- start start 2024-06-06T04:32:52.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73fc91fb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b5354237a0 0x55b535423260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73fc91fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535428f60 0x55b53542b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73fc91fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 0x55b53541de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73fc91fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1b2c70 con 0x55b535428f60 2024-06-06T04:32:52.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73fc91fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1b0500 con 0x55b535429880 2024-06-06T04:32:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73fc91fb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1ae350 con 0x55b5354237a0 2024-06-06T04:32:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73eaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535428f60 0x55b53542b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73eaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535428f60 0x55b53542b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40286/0 (socket says 172.21.15.45:40286) 2024-06-06T04:32:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73eaffd700 1 -- 172.21.15.45:0/4135565809 learned_addr learned my addr 172.21.15.45:0/4135565809 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73eb7fe700 1 --2- 172.21.15.45:0/4135565809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b5354237a0 0x55b535423260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73ebfff700 1 --2- 172.21.15.45:0/4135565809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 0x55b53541de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73eaffd700 1 -- 172.21.15.45:0/4135565809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b5354237a0 msgr2=0x55b535423260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73eaffd700 1 --2- 172.21.15.45:0/4135565809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b5354237a0 0x55b535423260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73eaffd700 1 -- 172.21.15.45:0/4135565809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 msgr2=0x55b53541de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73eaffd700 1 --2- 172.21.15.45:0/4135565809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 0x55b53541de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73eaffd700 1 -- 172.21.15.45:0/4135565809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b53542a8b0 con 0x55b535428f60 2024-06-06T04:32:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73ebfff700 1 --2- 172.21.15.45:0/4135565809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 0x55b53541de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:32:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.637+0000 7f73eb7fe700 1 --2- 172.21.15.45:0/4135565809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b5354237a0 0x55b535423260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.638+0000 7f73eaffd700 1 --2- 172.21.15.45:0/4135565809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535428f60 0x55b53542b8d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f73e00cc9f0 tx=0x7f73e000b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d340c0dd359002 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.638+0000 7f73d17fa700 1 -- 172.21.15.45:0/4135565809 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e009be00 con 0x55b535428f60 2024-06-06T04:32:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.638+0000 7f73d17fa700 1 -- 172.21.15.45:0/4135565809 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73e007f020 con 0x55b535428f60 2024-06-06T04:32:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.638+0000 7f73d17fa700 1 -- 172.21.15.45:0/4135565809 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e009fab0 con 0x55b535428f60 2024-06-06T04:32:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.639+0000 7f73fc91fb80 1 -- 172.21.15.45:0/4135565809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535428f60 msgr2=0x55b53542b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.639+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/4135565809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535428f60 0x55b53542b8d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f73e00cc9f0 tx=0x7f73e000b670 comp rx=0 tx=0).stop 2024-06-06T04:32:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.639+0000 7f73fc91fb80 1 -- 172.21.15.45:0/4135565809 shutdown_connections 2024-06-06T04:32:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.639+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/4135565809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b535429880 0x55b53541de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.639+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/4135565809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b5354237a0 0x55b535423260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.639+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/4135565809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535428f60 0x55b53542b8d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.639+0000 7f73fc91fb80 1 -- 172.21.15.45:0/4135565809 >> 172.21.15.45:0/4135565809 conn(0x55b535350080 msgr2=0x55b53542b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.639+0000 7f73fc91fb80 1 -- 172.21.15.45:0/4135565809 shutdown_connections 2024-06-06T04:32:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.639+0000 7f73fc91fb80 1 -- 172.21.15.45:0/4135565809 wait complete. 2024-06-06T04:32:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.639+0000 7f73fc91fb80 1 -- start start 2024-06-06T04:32:52.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.640+0000 7f73fc91fb80 1 Processor -- start 2024-06-06T04:32:52.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.640+0000 7f73fc91fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5354237a0 0x55b535435d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.640+0000 7f73fc91fb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b535428f60 0x55b535444680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.640+0000 7f73fc91fb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535429880 0x55b535444cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.640+0000 7f73fc91fb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1ae9c0 con 0x55b535429880 2024-06-06T04:32:52.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.640+0000 7f73eb7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5354237a0 0x55b535435d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.640+0000 7f73eaffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b535428f60 0x55b535444680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.640+0000 7f73eb7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5354237a0 0x55b535435d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:51834/0 (socket says 172.21.15.45:51834) 2024-06-06T04:32:52.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.640+0000 7f73eb7fe700 1 -- 172.21.15.45:0/1075769874 learned_addr learned my addr 172.21.15.45:0/1075769874 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:32:52.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.640+0000 7f73fc91fb80 1 -- 172.21.15.45:0/1075769874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1afe30 con 0x55b5354237a0 2024-06-06T04:32:52.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.640+0000 7f73fc91fb80 1 -- 172.21.15.45:0/1075769874 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1af090 con 0x55b535428f60 2024-06-06T04:32:52.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.640+0000 7f73ebfff700 1 --2- 172.21.15.45:0/1075769874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535429880 0x55b535444cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.641+0000 7f73fc91fb80 10 client.0 _ll_register_callbacks cb 0x55b53530a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:32:52.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.641+0000 7f73eaffd700 1 -- 172.21.15.45:0/1075769874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5354237a0 msgr2=0x55b535435d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:52.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.641+0000 7f73eaffd700 1 --2- 172.21.15.45:0/1075769874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5354237a0 0x55b535435d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.641+0000 7f73eaffd700 1 -- 172.21.15.45:0/1075769874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535429880 msgr2=0x55b535444cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:32:52.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.641+0000 7f73eaffd700 1 --2- 172.21.15.45:0/1075769874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535429880 0x55b535444cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.641+0000 7f73eaffd700 1 -- 172.21.15.45:0/1075769874 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b53542c6a0 con 0x55b535428f60 2024-06-06T04:32:52.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.641+0000 7f73fc91fb80 1 -- 172.21.15.45:0/1075769874 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e007f020 con 0x55b535428f60 2024-06-06T04:32:52.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.641+0000 7f73eb7fe700 1 --2- 172.21.15.45:0/1075769874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5354237a0 0x55b535435d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:32:52.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.641+0000 7f73eaffd700 1 --2- 172.21.15.45:0/1075769874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b535428f60 0x55b535444680 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55b53541efe0 tx=0x7f73e000f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d46a2081b55355f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:32:52.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.641+0000 7f73caffd700 10 client.25246 ms_handle_connect on v2:172.21.15.154:3300/0 2024-06-06T04:32:52.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.641+0000 7f73caffd700 1 -- 172.21.15.45:0/1075769874 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e009be00 con 0x55b535428f60 2024-06-06T04:32:52.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.641+0000 7f73caffd700 1 -- 172.21.15.45:0/1075769874 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f73e00a3fb0 con 0x55b535428f60 2024-06-06T04:32:52.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.641+0000 7f73caffd700 1 -- 172.21.15.45:0/1075769874 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f73e00a43c0 con 0x55b535428f60 2024-06-06T04:32:52.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.641+0000 7f73fc91fb80 10 client.25246 fetch_fsmap learned FSMap version 27 2024-06-06T04:32:52.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.641+0000 7f73fc91fb80 1 -- 172.21.15.45:0/1075769874 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55b5354a7a40 con 0x55b535428f60 2024-06-06T04:32:52.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.642+0000 7f73caffd700 1 -- 172.21.15.45:0/1075769874 <== mon.1 v2:172.21.15.154:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f73e009ab50 con 0x55b535428f60 2024-06-06T04:32:52.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.642+0000 7f73fc91fb80 10 client.25246 fetch_fsmap finished waiting for FSMap version 27 2024-06-06T04:32:52.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.642+0000 7f73fc91fb80 10 client.25246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:32:52.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.642+0000 7f73fc91fb80 20 client.25246 populate_metadata read hostname 'smithi045' 2024-06-06T04:32:52.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.642+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/1075769874 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55b53543cf50 0x55b5354a9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:32:52.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.642+0000 7f73fc91fb80 4 client.25246 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T04:32:52.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.642+0000 7f73fc91fb80 1 -- 172.21.15.45:0/1075769874 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b53537c2d0 con 0x55b53543cf50 2024-06-06T04:32:52.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.642+0000 7f73eb7fe700 1 --2- 172.21.15.45:0/1075769874 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55b53543cf50 0x55b5354a9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:32:52.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.642+0000 7f73eb7fe700 1 --2- 172.21.15.45:0/1075769874 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55b53543cf50 0x55b5354a9dd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2273c72734612ce9 server_cookie=e7925d88562f60fb in_seq=0 out_seq=0 2024-06-06T04:32:52.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.642+0000 7f73caffd700 10 client.25246 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:52.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.643+0000 7f73caffd700 1 -- 172.21.15.45:0/1075769874 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b53537c2d0 con 0x55b53543cf50 2024-06-06T04:32:52.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.643+0000 7f73caffd700 10 client.25246 handle_command_reply: tid=0 2024-06-06T04:32:52.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.643+0000 7f73fc91fb80 1 client.25246 shutdown 2024-06-06T04:32:52.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.643+0000 7f73fc91fb80 1 -- 172.21.15.45:0/1075769874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b535428f60 msgr2=0x55b535444680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:52.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.643+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/1075769874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b535428f60 0x55b535444680 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55b53541efe0 tx=0x7f73e000f6a0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.643+0000 7f73fc91fb80 1 -- 172.21.15.45:0/1075769874 shutdown_connections 2024-06-06T04:32:52.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.643+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/1075769874 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55b53543cf50 0x55b5354a9dd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.643+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/1075769874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b535429880 0x55b535444cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.643+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/1075769874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5354237a0 0x55b535435d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.643+0000 7f73fc91fb80 1 --2- 172.21.15.45:0/1075769874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b535428f60 0x55b535444680 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.643+0000 7f73fc91fb80 1 -- 172.21.15.45:0/1075769874 >> 172.21.15.45:0/1075769874 conn(0x55b535350080 msgr2=0x55b53542ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:52.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.643+0000 7f73caffd700 0 client.25246 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:32:52.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.643+0000 7f73fc91fb80 1 -- 172.21.15.45:0/1075769874 shutdown_connections 2024-06-06T04:32:52.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.643+0000 7f73fc91fb80 1 -- 172.21.15.45:0/1075769874 wait complete. 2024-06-06T04:32:52.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.643+0000 7f73fc91fb80 20 client.25246 trim_cache size 0 max 16384 2024-06-06T04:32:52.648 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:32:52.648 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-06T04:32:52.648 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-06-06T04:32:52.648 INFO:teuthology.orchestra.run.smithi045.stdout: "99af536a-9204-40db-a3a4-1c808cd4d8f0": { 2024-06-06T04:32:52.648 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-06-06T04:32:52.648 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "99af536a-9204-40db-a3a4-1c808cd4d8f0", 2024-06-06T04:32:52.648 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-06-06T04:32:52.648 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:32:52.648 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:32:52.648 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:32:52.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.644+0000 7f73f209c700 1 -- 172.21.15.45:0/646302901 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73d40a2ca0 msgr2=0x7f73d40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:52.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.644+0000 7f73f209c700 1 --2- 172.21.15.45:0/646302901 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73d40a2ca0 0x7f73d40a5160 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f73dc005fd0 tx=0x7f73dc005db0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.644+0000 7f73f209c700 1 -- 172.21.15.45:0/646302901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73ec1017f0 msgr2=0x7f73ec1a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:32:52.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.644+0000 7f73f209c700 1 --2- 172.21.15.45:0/646302901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73ec1017f0 0x7f73ec1a0dd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f73e000d980 tx=0x7f73e000dd40 comp rx=0 tx=0).stop 2024-06-06T04:32:52.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.644+0000 7f73f209c700 1 -- 172.21.15.45:0/646302901 shutdown_connections 2024-06-06T04:32:52.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.644+0000 7f73f209c700 1 --2- 172.21.15.45:0/646302901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73ec107500 0x7f73ec19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.644+0000 7f73f209c700 1 --2- 172.21.15.45:0/646302901 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f73d40a2ca0 0x7f73d40a5160 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.644+0000 7f73f209c700 1 --2- 172.21.15.45:0/646302901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73ec100e30 0x7f73ec1a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.644+0000 7f73f209c700 1 --2- 172.21.15.45:0/646302901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73ec1017f0 0x7f73ec1a0dd0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:32:52.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.644+0000 7f73f209c700 1 -- 172.21.15.45:0/646302901 >> 172.21.15.45:0/646302901 conn(0x7f73ec076730 msgr2=0x7f73ec0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:32:52.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.644+0000 7f73f209c700 1 -- 172.21.15.45:0/646302901 shutdown_connections 2024-06-06T04:32:52.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:32:52.644+0000 7f73f209c700 1 -- 172.21.15.45:0/646302901 wait complete. 2024-06-06T04:32:53.426 INFO:tasks.cephfs.filesystem:scrub status for tag:99af536a-9204-40db-a3a4-1c808cd4d8f0 - {'path': '/', 'tag': '99af536a-9204-40db-a3a4-1c808cd4d8f0', 'options': 'recursive,force'} 2024-06-06T04:32:54.358 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:54 smithi086 ceph-mon[36053]: pgmap v474: 129 pgs: 129 active+clean; 820 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-06-06T04:32:54.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:54 smithi154 ceph-mon[36428]: pgmap v474: 129 pgs: 129 active+clean; 820 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-06-06T04:32:54.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:54 smithi045 ceph-mon[32299]: pgmap v474: 129 pgs: 129 active+clean; 820 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-06-06T04:32:55.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:55 smithi086 ceph-mon[36053]: scrub summary: idle 2024-06-06T04:32:55.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:55 smithi154 ceph-mon[36428]: scrub summary: idle 2024-06-06T04:32:55.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:55 smithi045 ceph-mon[32299]: scrub summary: idle 2024-06-06T04:32:56.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:56 smithi086 ceph-mon[36053]: pgmap v475: 129 pgs: 129 active+clean; 837 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-06-06T04:32:56.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:56 smithi086 ceph-mon[36053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:32:56.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:56 smithi154 ceph-mon[36428]: pgmap v475: 129 pgs: 129 active+clean; 837 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-06-06T04:32:56.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:56 smithi154 ceph-mon[36428]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:32:56.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:56 smithi045 ceph-mon[32299]: pgmap v475: 129 pgs: 129 active+clean; 837 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-06-06T04:32:56.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:56 smithi045 ceph-mon[32299]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:32:58.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:32:58 smithi154 ceph-mon[36428]: pgmap v476: 129 pgs: 129 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 63 KiB/s wr, 14 op/s 2024-06-06T04:32:58.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:32:58 smithi045 ceph-mon[32299]: pgmap v476: 129 pgs: 129 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 63 KiB/s wr, 14 op/s 2024-06-06T04:32:58.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:32:58 smithi086 ceph-mon[36053]: pgmap v476: 129 pgs: 129 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 63 KiB/s wr, 14 op/s 2024-06-06T04:33:00.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:00 smithi154 ceph-mon[36428]: pgmap v477: 129 pgs: 129 active+clean; 14 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-06-06T04:33:00.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:00 smithi045 ceph-mon[32299]: pgmap v477: 129 pgs: 129 active+clean; 14 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-06-06T04:33:00.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:00 smithi086 ceph-mon[36053]: pgmap v477: 129 pgs: 129 active+clean; 14 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-06-06T04:33:01.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:01 smithi154 ceph-mon[36428]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:33:01.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:01 smithi045 ceph-mon[32299]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:33:01.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:01 smithi086 ceph-mon[36053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:33:02.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:02 smithi154 ceph-mon[36428]: pgmap v478: 129 pgs: 129 active+clean; 18 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.5 MiB/s wr, 59 op/s 2024-06-06T04:33:02.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:02 smithi045 ceph-mon[32299]: pgmap v478: 129 pgs: 129 active+clean; 18 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.5 MiB/s wr, 59 op/s 2024-06-06T04:33:02.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:02 smithi086 ceph-mon[36053]: pgmap v478: 129 pgs: 129 active+clean; 18 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.5 MiB/s wr, 59 op/s 2024-06-06T04:33:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.988+0000 7f97ae6ca640 1 -- 172.21.15.45:0/4075755949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a8101a30 msgr2=0x7f97a8101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.988+0000 7f97ae6ca640 1 --2- 172.21.15.45:0/4075755949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a8101a30 0x7f97a8101e50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9798009900 tx=0x7f979802a410 comp rx=0 tx=0).stop 2024-06-06T04:33:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.988+0000 7f97ae6ca640 1 -- 172.21.15.45:0/4075755949 shutdown_connections 2024-06-06T04:33:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.988+0000 7f97ae6ca640 1 --2- 172.21.15.45:0/4075755949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97a81035e0 0x7f97a8109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.988+0000 7f97ae6ca640 1 --2- 172.21.15.45:0/4075755949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97a8102c20 0x7f97a81030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.988+0000 7f97ae6ca640 1 --2- 172.21.15.45:0/4075755949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a8101a30 0x7f97a8101e50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.988+0000 7f97ae6ca640 1 -- 172.21.15.45:0/4075755949 >> 172.21.15.45:0/4075755949 conn(0x7f97a80fcfc0 msgr2=0x7f97a80ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.988+0000 7f97ae6ca640 1 -- 172.21.15.45:0/4075755949 shutdown_connections 2024-06-06T04:33:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.988+0000 7f97ae6ca640 1 -- 172.21.15.45:0/4075755949 wait complete. 2024-06-06T04:33:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.989+0000 7f97ae6ca640 1 Processor -- start 2024-06-06T04:33:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.989+0000 7f97ae6ca640 1 -- start start 2024-06-06T04:33:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.989+0000 7f97ae6ca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97a8101a30 0x7f97a8197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.989+0000 7f97ae6ca640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97a8102c20 0x7f97a81980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.989+0000 7f97ae6ca640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a81035e0 0x7f97a819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.989+0000 7f97ae6ca640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97a810d580 con 0x7f97a81035e0 2024-06-06T04:33:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.989+0000 7f97ae6ca640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97a810d400 con 0x7f97a8102c20 2024-06-06T04:33:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.989+0000 7f97ae6ca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f97a810d700 con 0x7f97a8101a30 2024-06-06T04:33:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.989+0000 7f97acc67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a81035e0 0x7f97a819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.989+0000 7f97acc67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a81035e0 0x7f97a819f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35224/0 (socket says 172.21.15.45:35224) 2024-06-06T04:33:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.990+0000 7f97acc67640 1 -- 172.21.15.45:0/3695411071 learned_addr learned my addr 172.21.15.45:0/3695411071 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.990+0000 7f97a7fff640 1 --2- 172.21.15.45:0/3695411071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97a8101a30 0x7f97a8197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.990+0000 7f979ffff640 1 --2- 172.21.15.45:0/3695411071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97a8102c20 0x7f97a81980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.990+0000 7f97acc67640 1 -- 172.21.15.45:0/3695411071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97a8101a30 msgr2=0x7f97a8197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.990+0000 7f97acc67640 1 --2- 172.21.15.45:0/3695411071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97a8101a30 0x7f97a8197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.990+0000 7f97acc67640 1 -- 172.21.15.45:0/3695411071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97a8102c20 msgr2=0x7f97a81980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.990+0000 7f97acc67640 1 --2- 172.21.15.45:0/3695411071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97a8102c20 0x7f97a81980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.990+0000 7f97acc67640 1 -- 172.21.15.45:0/3695411071 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a819f7b0 con 0x7f97a81035e0 2024-06-06T04:33:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.990+0000 7f97acc67640 1 --2- 172.21.15.45:0/3695411071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a81035e0 0x7f97a819f140 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f97a000eaa0 tx=0x7f97a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.990+0000 7f97a5ffb640 1 -- 172.21.15.45:0/3695411071 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a000cdb0 con 0x7f97a81035e0 2024-06-06T04:33:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.990+0000 7f97a5ffb640 1 -- 172.21.15.45:0/3695411071 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97a0005020 con 0x7f97a81035e0 2024-06-06T04:33:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.990+0000 7f97a5ffb640 1 -- 172.21.15.45:0/3695411071 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a00116d0 con 0x7f97a81035e0 2024-06-06T04:33:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.990+0000 7f97ae6ca640 1 -- 172.21.15.45:0/3695411071 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a819faa0 con 0x7f97a81035e0 2024-06-06T04:33:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.990+0000 7f97ae6ca640 1 -- 172.21.15.45:0/3695411071 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a819fff0 con 0x7f97a81035e0 2024-06-06T04:33:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.993+0000 7f97a5ffb640 1 -- 172.21.15.45:0/3695411071 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f97a00051d0 con 0x7f97a81035e0 2024-06-06T04:33:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.993+0000 7f97b13d7740 1 Processor -- start 2024-06-06T04:33:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.993+0000 7f97b13d7740 1 -- start start 2024-06-06T04:33:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.993+0000 7f97b13d7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7010 0x5645586a7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.993+0000 7f97b13d7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7a00 0x5645586ac500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.993+0000 7f97b13d7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 0x5645586af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.993+0000 7f97b13d7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56455860a4f0 con 0x5645586a7010 2024-06-06T04:33:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.993+0000 7f97b13d7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5645585a8870 con 0x5645586a7a00 2024-06-06T04:33:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.993+0000 7f97b13d7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5645586064b0 con 0x5645586acb70 2024-06-06T04:33:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.993+0000 7f979ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7010 0x5645586a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.993+0000 7f979ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7010 0x5645586a7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35226/0 (socket says 172.21.15.45:35226) 2024-06-06T04:33:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.993+0000 7f979ffff640 1 -- 172.21.15.45:0/149633446 learned_addr learned my addr 172.21.15.45:0/149633446 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.993+0000 7f97a5ffb640 1 --2- 172.21.15.45:0/3695411071 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f97940a2cb0 0x7f97940a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.994+0000 7f979ffff640 1 --2- 172.21.15.45:0/149633446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 0x5645586af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.994+0000 7f97a7fff640 1 --2- 172.21.15.45:0/149633446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7a00 0x5645586ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.994+0000 7f97acc67640 1 --2- 172.21.15.45:0/3695411071 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f97940a2cb0 0x7f97940a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.994+0000 7f979ffff640 1 -- 172.21.15.45:0/149633446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 msgr2=0x5645586af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.994+0000 7f979ffff640 1 --2- 172.21.15.45:0/149633446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 0x5645586af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.994+0000 7f979ffff640 1 -- 172.21.15.45:0/149633446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7a00 msgr2=0x5645586ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.994+0000 7f97a5ffb640 1 -- 172.21.15.45:0/3695411071 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f97a00cf490 con 0x7f97a81035e0 2024-06-06T04:33:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.994+0000 7f979ffff640 1 --2- 172.21.15.45:0/149633446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7a00 0x5645586ac500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.994+0000 7f979ffff640 1 -- 172.21.15.45:0/149633446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645585c49b0 con 0x5645586a7010 2024-06-06T04:33:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.994+0000 7f97a7fff640 1 --2- 172.21.15.45:0/149633446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7a00 0x5645586ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:33:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.994+0000 7f979ffff640 1 --2- 172.21.15.45:0/149633446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7010 0x5645586a7430 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f97900125b0 tx=0x7f979000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c5d738dc542494 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.994+0000 7f97acc67640 1 --2- 172.21.15.45:0/3695411071 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f97940a2cb0 0x7f97940a5170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f97a00ccbd0 tx=0x7f97a0090ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.994+0000 7f979e7fc640 1 -- 172.21.15.45:0/149633446 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790005020 con 0x5645586a7010 2024-06-06T04:33:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.994+0000 7f979e7fc640 1 -- 172.21.15.45:0/149633446 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9790005670 con 0x5645586a7010 2024-06-06T04:33:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.994+0000 7f979e7fc640 1 -- 172.21.15.45:0/149633446 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979000e770 con 0x5645586a7010 2024-06-06T04:33:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.995+0000 7f97b13d7740 1 -- 172.21.15.45:0/149633446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7010 msgr2=0x5645586a7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.995+0000 7f97b13d7740 1 --2- 172.21.15.45:0/149633446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7010 0x5645586a7430 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f97900125b0 tx=0x7f979000b8c0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.995+0000 7f97b13d7740 1 -- 172.21.15.45:0/149633446 shutdown_connections 2024-06-06T04:33:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.995+0000 7f97b13d7740 1 --2- 172.21.15.45:0/149633446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7a00 0x5645586ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.995+0000 7f97b13d7740 1 --2- 172.21.15.45:0/149633446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 0x5645586af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.995+0000 7f97b13d7740 1 --2- 172.21.15.45:0/149633446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7010 0x5645586a7430 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.995+0000 7f97b13d7740 1 -- 172.21.15.45:0/149633446 >> 172.21.15.45:0/149633446 conn(0x5645586a2680 msgr2=0x5645586a2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.995+0000 7f97b13d7740 1 -- 172.21.15.45:0/149633446 shutdown_connections 2024-06-06T04:33:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.995+0000 7f97b13d7740 1 -- 172.21.15.45:0/149633446 wait complete. 2024-06-06T04:33:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.995+0000 7f97b13d7740 1 -- start start 2024-06-06T04:33:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f97b13d7740 1 Processor -- start 2024-06-06T04:33:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f97b13d7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7010 0x5645586b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f97b13d7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7a00 0x5645586b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f97b13d7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 0x5645586b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f97b13d7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5645586afb20 con 0x5645586a7a00 2024-06-06T04:33:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f97b13d7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564558593ae0 con 0x5645586a7010 2024-06-06T04:33:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f97b13d7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5645586afca0 con 0x5645586acb70 2024-06-06T04:33:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f97b13d7740 10 client.0 _ll_register_callbacks cb 0x5645585cc9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7a00 0x5645586b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7a00 0x5645586b17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35228/0 (socket says 172.21.15.45:35228) 2024-06-06T04:33:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f97a7fff640 1 -- 172.21.15.45:0/1379438652 learned_addr learned my addr 172.21.15.45:0/1379438652 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f979ffff640 1 --2- 172.21.15.45:0/1379438652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 0x5645586b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f97a7fff640 1 -- 172.21.15.45:0/1379438652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 msgr2=0x5645586b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f97a7fff640 1 --2- 172.21.15.45:0/1379438652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 0x5645586b87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f979ffff640 1 --2- 172.21.15.45:0/1379438652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7010 0x5645586b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f97a7fff640 1 -- 172.21.15.45:0/1379438652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7010 msgr2=0x5645586b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f97a7fff640 1 --2- 172.21.15.45:0/1379438652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7010 0x5645586b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f97a7fff640 1 -- 172.21.15.45:0/1379438652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56455831e210 con 0x5645586a7a00 2024-06-06T04:33:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.996+0000 7f979ffff640 1 --2- 172.21.15.45:0/1379438652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7010 0x5645586b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:33:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.997+0000 7f97a7fff640 1 --2- 172.21.15.45:0/1379438652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7a00 0x5645586b17a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9798009900 tx=0x7f9798004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d201f01bb2810a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.997+0000 7f97877fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:33:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.997+0000 7f97877fe640 1 -- 172.21.15.45:0/1379438652 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798035070 con 0x5645586a7a00 2024-06-06T04:33:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.997+0000 7f97877fe640 1 -- 172.21.15.45:0/1379438652 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f9798003ac0 con 0x5645586a7a00 2024-06-06T04:33:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.997+0000 7f97b13d7740 1 -- 172.21.15.45:0/1379438652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9790005670 con 0x5645586a7a00 2024-06-06T04:33:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.997+0000 7f97877fe640 1 -- 172.21.15.45:0/1379438652 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f979804d050 con 0x5645586a7a00 2024-06-06T04:33:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.997+0000 7f97b13d7740 10 client.15651 fetch_fsmap learned FSMap version 29 2024-06-06T04:33:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.997+0000 7f97b13d7740 1 -- 172.21.15.45:0/1379438652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5645583ea620 con 0x5645586a7a00 2024-06-06T04:33:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.997+0000 7f97877fe640 1 -- 172.21.15.45:0/1379438652 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9790005670 con 0x5645586a7a00 2024-06-06T04:33:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.997+0000 7f97877fe640 1 -- 172.21.15.45:0/1379438652 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f979803b430 con 0x5645586a7a00 2024-06-06T04:33:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.997+0000 7f97b13d7740 10 client.15651 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T04:33:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.997+0000 7f97b13d7740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:33:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.997+0000 7f97b13d7740 20 client.15651 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.997+0000 7f97b13d7740 1 --2- 172.21.15.45:0/1379438652 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5645586bd130 0x5645586bf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.997+0000 7f97b13d7740 4 client.15651 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:33:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.997+0000 7f97b13d7740 1 -- 172.21.15.45:0/1379438652 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56455861df10 con 0x5645586bd130 2024-06-06T04:33:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.998+0000 7f979ffff640 1 --2- 172.21.15.45:0/1379438652 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5645586bd130 0x5645586bf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.998+0000 7f979ffff640 1 --2- 172.21.15.45:0/1379438652 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5645586bd130 0x5645586bf5b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39582b71b1b8a313 server_cookie=410cf111ed1aece1 in_seq=0 out_seq=0 2024-06-06T04:33:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.998+0000 7f97877fe640 10 client.15651 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.999+0000 7f97877fe640 1 -- 172.21.15.45:0/1379438652 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56455861df10 con 0x5645586bd130 2024-06-06T04:33:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.999+0000 7f97877fe640 10 client.15651 handle_command_reply: tid=0 2024-06-06T04:33:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.999+0000 7f97b13d7740 1 client.15651 shutdown 2024-06-06T04:33:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.999+0000 7f97b13d7740 1 -- 172.21.15.45:0/1379438652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7a00 msgr2=0x5645586b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.999+0000 7f97b13d7740 1 --2- 172.21.15.45:0/1379438652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7a00 0x5645586b17a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9798009900 tx=0x7f9798004430 comp rx=0 tx=0).stop 2024-06-06T04:33:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.999+0000 7f97b13d7740 1 -- 172.21.15.45:0/1379438652 shutdown_connections 2024-06-06T04:33:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.999+0000 7f97b13d7740 1 --2- 172.21.15.45:0/1379438652 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5645586bd130 0x5645586bf5b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.999+0000 7f97b13d7740 1 --2- 172.21.15.45:0/1379438652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 0x5645586b87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.999+0000 7f97b13d7740 1 --2- 172.21.15.45:0/1379438652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7a00 0x5645586b17a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.999+0000 7f97877fe640 0 client.15651 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.999+0000 7f97b13d7740 1 --2- 172.21.15.45:0/1379438652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7010 0x5645586b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.999+0000 7f97b13d7740 1 -- 172.21.15.45:0/1379438652 >> 172.21.15.45:0/1379438652 conn(0x5645586a2680 msgr2=0x5645586a17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.999+0000 7f97b13d7740 1 -- 172.21.15.45:0/1379438652 shutdown_connections 2024-06-06T04:33:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.999+0000 7f97b13d7740 1 -- 172.21.15.45:0/1379438652 wait complete. 2024-06-06T04:33:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:02.999+0000 7f97b13d7740 20 client.15651 trim_cache size 0 max 16384 2024-06-06T04:33:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f97b13d7740 1 Processor -- start 2024-06-06T04:33:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f97b13d7740 1 -- start start 2024-06-06T04:33:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f97b13d7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7010 0x5645586aef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f97b13d7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7a00 0x5645586b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f97b13d7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 0x5645586b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f97b13d7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81ace30 con 0x5645586a7a00 2024-06-06T04:33:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f97b13d7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81aa700 con 0x5645586a7010 2024-06-06T04:33:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f97b13d7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81a8550 con 0x5645586acb70 2024-06-06T04:33:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7a00 0x5645586b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7a00 0x5645586b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35232/0 (socket says 172.21.15.45:35232) 2024-06-06T04:33:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f97a7fff640 1 -- 172.21.15.45:0/2727031786 learned_addr learned my addr 172.21.15.45:0/2727031786 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f979ffff640 1 --2- 172.21.15.45:0/2727031786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7010 0x5645586aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f979ffff640 1 --2- 172.21.15.45:0/2727031786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 0x5645586b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f97a7fff640 1 -- 172.21.15.45:0/2727031786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 msgr2=0x5645586b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f97a7fff640 1 --2- 172.21.15.45:0/2727031786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 0x5645586b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f97a7fff640 1 -- 172.21.15.45:0/2727031786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7010 msgr2=0x5645586aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f97a7fff640 1 --2- 172.21.15.45:0/2727031786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7010 0x5645586aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.015+0000 7f97a7fff640 1 -- 172.21.15.45:0/2727031786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645586bba10 con 0x5645586a7a00 2024-06-06T04:33:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.016+0000 7f97a7fff640 1 --2- 172.21.15.45:0/2727031786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7a00 0x5645586b1260 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9798005c40 tx=0x7f97980314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8795dd30e1b761e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.016+0000 7f979dffb640 1 -- 172.21.15.45:0/2727031786 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798056070 con 0x5645586a7a00 2024-06-06T04:33:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.016+0000 7f979dffb640 1 -- 172.21.15.45:0/2727031786 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9798003d70 con 0x5645586a7a00 2024-06-06T04:33:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.016+0000 7f979dffb640 1 -- 172.21.15.45:0/2727031786 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798035070 con 0x5645586a7a00 2024-06-06T04:33:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.016+0000 7f97b13d7740 1 -- 172.21.15.45:0/2727031786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7a00 msgr2=0x5645586b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.016+0000 7f97b13d7740 1 --2- 172.21.15.45:0/2727031786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7a00 0x5645586b1260 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9798005c40 tx=0x7f97980314a0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.016+0000 7f97b13d7740 1 -- 172.21.15.45:0/2727031786 shutdown_connections 2024-06-06T04:33:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.016+0000 7f97b13d7740 1 --2- 172.21.15.45:0/2727031786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 0x5645586b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.016+0000 7f97b13d7740 1 --2- 172.21.15.45:0/2727031786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7a00 0x5645586b1260 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.016+0000 7f97b13d7740 1 --2- 172.21.15.45:0/2727031786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7010 0x5645586aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.016+0000 7f97b13d7740 1 -- 172.21.15.45:0/2727031786 >> 172.21.15.45:0/2727031786 conn(0x5645586a2680 msgr2=0x5645586ac290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.016+0000 7f97b13d7740 1 -- 172.21.15.45:0/2727031786 shutdown_connections 2024-06-06T04:33:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.016+0000 7f97b13d7740 1 -- 172.21.15.45:0/2727031786 wait complete. 2024-06-06T04:33:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.017+0000 7f97b13d7740 1 -- start start 2024-06-06T04:33:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.017+0000 7f97b13d7740 1 Processor -- start 2024-06-06T04:33:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.017+0000 7f97b13d7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7010 0x564558722c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.017+0000 7f97b13d7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7a00 0x5645587231a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.017+0000 7f97b13d7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 0x56455872a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.017+0000 7f97b13d7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81a8bc0 con 0x5645586a7010 2024-06-06T04:33:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.017+0000 7f97b13d7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81aa030 con 0x5645586a7a00 2024-06-06T04:33:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.017+0000 7f97b13d7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81a9290 con 0x5645586acb70 2024-06-06T04:33:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.017+0000 7f97b13d7740 10 client.0 _ll_register_callbacks cb 0x7f9798005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.017+0000 7f979ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7010 0x564558722c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.017+0000 7f979ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7010 0x564558722c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35242/0 (socket says 172.21.15.45:35242) 2024-06-06T04:33:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.017+0000 7f979ffff640 1 -- 172.21.15.45:0/2366428705 learned_addr learned my addr 172.21.15.45:0/2366428705 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.017+0000 7f97a7fff640 1 --2- 172.21.15.45:0/2366428705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7a00 0x5645587231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f979ffff640 1 -- 172.21.15.45:0/2366428705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 msgr2=0x56455872a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f979ffff640 1 --2- 172.21.15.45:0/2366428705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 0x56455872a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f979ffff640 1 -- 172.21.15.45:0/2366428705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7a00 msgr2=0x5645587231a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f979ffff640 1 --2- 172.21.15.45:0/2366428705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7a00 0x5645587231a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f979ffff640 1 -- 172.21.15.45:0/2366428705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56455872a7d0 con 0x5645586a7010 2024-06-06T04:33:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f979ffff640 1 --2- 172.21.15.45:0/2366428705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7010 0x564558722c60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x5645586a8260 tx=0x7f9790005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb05f4689bcd2e62 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f97877fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:33:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f97877fe640 1 -- 172.21.15.45:0/2366428705 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790004220 con 0x5645586a7010 2024-06-06T04:33:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f97877fe640 1 -- 172.21.15.45:0/2366428705 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f97900043c0 con 0x5645586a7010 2024-06-06T04:33:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f97b13d7740 1 -- 172.21.15.45:0/2366428705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9798003d70 con 0x5645586a7010 2024-06-06T04:33:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f97877fe640 1 -- 172.21.15.45:0/2366428705 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97900035f0 con 0x5645586a7010 2024-06-06T04:33:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f97b13d7740 10 client.15657 fetch_fsmap learned FSMap version 29 2024-06-06T04:33:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f97b13d7740 1 -- 172.21.15.45:0/2366428705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56455872bae0 con 0x5645586a7010 2024-06-06T04:33:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f97877fe640 1 -- 172.21.15.45:0/2366428705 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9798003d70 con 0x5645586a7010 2024-06-06T04:33:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f97877fe640 1 -- 172.21.15.45:0/2366428705 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9790030020 con 0x5645586a7010 2024-06-06T04:33:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f97b13d7740 10 client.15657 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T04:33:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f97b13d7740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:33:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.018+0000 7f97b13d7740 20 client.15657 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.019+0000 7f97b13d7740 1 --2- 172.21.15.45:0/2366428705 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x56455872c340 0x56455872e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.019+0000 7f97b13d7740 4 client.15657 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T04:33:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.019+0000 7f97b13d7740 1 -- 172.21.15.45:0/2366428705 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564558617c20 con 0x56455872c340 2024-06-06T04:33:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.019+0000 7f97a7fff640 1 --2- 172.21.15.45:0/2366428705 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x56455872c340 0x56455872e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.019+0000 7f97877fe640 10 client.15657 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.019+0000 7f97a7fff640 1 --2- 172.21.15.45:0/2366428705 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x56455872c340 0x56455872e760 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b0899c6f749fc1 server_cookie=ec0900a755f3ab2b in_seq=0 out_seq=0 2024-06-06T04:33:03.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.772+0000 7f97877fe640 1 -- 172.21.15.45:0/2366428705 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564558617c20 con 0x56455872c340 2024-06-06T04:33:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.772+0000 7f97877fe640 10 client.15657 handle_command_reply: tid=0 2024-06-06T04:33:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.772+0000 7f97b13d7740 1 client.15657 shutdown 2024-06-06T04:33:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.772+0000 7f97b13d7740 1 -- 172.21.15.45:0/2366428705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7010 msgr2=0x564558722c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.772+0000 7f97b13d7740 1 --2- 172.21.15.45:0/2366428705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7010 0x564558722c60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x5645586a8260 tx=0x7f9790005c50 comp rx=0 tx=0).stop 2024-06-06T04:33:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.772+0000 7f97b13d7740 1 -- 172.21.15.45:0/2366428705 shutdown_connections 2024-06-06T04:33:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.772+0000 7f97b13d7740 1 --2- 172.21.15.45:0/2366428705 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x56455872c340 0x56455872e760 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.772+0000 7f97b13d7740 1 --2- 172.21.15.45:0/2366428705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645586a7a00 0x5645587231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.772+0000 7f97b13d7740 1 --2- 172.21.15.45:0/2366428705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645586acb70 0x56455872a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.772+0000 7f97b13d7740 1 --2- 172.21.15.45:0/2366428705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5645586a7010 0x564558722c60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.772+0000 7f97877fe640 0 client.15657 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:03.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.772+0000 7f97b13d7740 1 -- 172.21.15.45:0/2366428705 >> 172.21.15.45:0/2366428705 conn(0x5645586a2680 msgr2=0x5645586b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.772+0000 7f97b13d7740 1 -- 172.21.15.45:0/2366428705 shutdown_connections 2024-06-06T04:33:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.772+0000 7f97b13d7740 1 -- 172.21.15.45:0/2366428705 wait complete. 2024-06-06T04:33:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.772+0000 7f97b13d7740 20 client.15657 trim_cache size 0 max 16384 2024-06-06T04:33:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-06-06T04:33:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-06-06T04:33:03.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-06-06T04:33:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.773+0000 7f97ae6ca640 1 -- 172.21.15.45:0/3695411071 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f97940a2cb0 msgr2=0x7f97940a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.773+0000 7f97ae6ca640 1 --2- 172.21.15.45:0/3695411071 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f97940a2cb0 0x7f97940a5170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f97a00ccbd0 tx=0x7f97a0090ec0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.773+0000 7f97ae6ca640 1 -- 172.21.15.45:0/3695411071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a81035e0 msgr2=0x7f97a819f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.773+0000 7f97ae6ca640 1 --2- 172.21.15.45:0/3695411071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a81035e0 0x7f97a819f140 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f97a000eaa0 tx=0x7f97a000ef60 comp rx=0 tx=0).stop 2024-06-06T04:33:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.773+0000 7f97ae6ca640 1 -- 172.21.15.45:0/3695411071 shutdown_connections 2024-06-06T04:33:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.773+0000 7f97ae6ca640 1 --2- 172.21.15.45:0/3695411071 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f97940a2cb0 0x7f97940a5170 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.773+0000 7f97ae6ca640 1 --2- 172.21.15.45:0/3695411071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97a8102c20 0x7f97a81980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.773+0000 7f97ae6ca640 1 --2- 172.21.15.45:0/3695411071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a81035e0 0x7f97a819f140 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.773+0000 7f97ae6ca640 1 --2- 172.21.15.45:0/3695411071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97a8101a30 0x7f97a8197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.773+0000 7f97ae6ca640 1 -- 172.21.15.45:0/3695411071 >> 172.21.15.45:0/3695411071 conn(0x7f97a80fcfc0 msgr2=0x7f97a8105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.773+0000 7f97ae6ca640 1 -- 172.21.15.45:0/3695411071 shutdown_connections 2024-06-06T04:33:03.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:03.773+0000 7f97ae6ca640 1 -- 172.21.15.45:0/3695411071 wait complete. 2024-06-06T04:33:04.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:04 smithi154 ceph-mon[36428]: pgmap v479: 129 pgs: 129 active+clean; 19 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-06-06T04:33:04.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:04 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:33:04.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:04 smithi045 ceph-mon[32299]: pgmap v479: 129 pgs: 129 active+clean; 19 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-06-06T04:33:04.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:04 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:33:04.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:04 smithi086 ceph-mon[36053]: pgmap v479: 129 pgs: 129 active+clean; 19 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-06-06T04:33:04.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:04 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:33:05.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:33:05.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:33:05.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:33:05.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:33:05.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:33:05.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:33:05.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:33:05.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:33:05.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:33:06.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:06 smithi154 ceph-mon[36428]: pgmap v480: 129 pgs: 129 active+clean; 20 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 1.7 MiB/s wr, 162 op/s 2024-06-06T04:33:06.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:06 smithi045 ceph-mon[32299]: pgmap v480: 129 pgs: 129 active+clean; 20 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 1.7 MiB/s wr, 162 op/s 2024-06-06T04:33:06.608 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:06 smithi086 ceph-mon[36053]: pgmap v480: 129 pgs: 129 active+clean; 20 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 1.7 MiB/s wr, 162 op/s 2024-06-06T04:33:08.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:08 smithi154 ceph-mon[36428]: pgmap v481: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 1.7 MiB/s wr, 223 op/s 2024-06-06T04:33:08.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:08 smithi045 ceph-mon[32299]: pgmap v481: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 1.7 MiB/s wr, 223 op/s 2024-06-06T04:33:08.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:08 smithi086 ceph-mon[36053]: pgmap v481: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 1.7 MiB/s wr, 223 op/s 2024-06-06T04:33:10.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:10 smithi154 ceph-mon[36428]: pgmap v482: 129 pgs: 129 active+clean; 26 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 2.3 MiB/s wr, 524 op/s 2024-06-06T04:33:10.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:10 smithi045 ceph-mon[32299]: pgmap v482: 129 pgs: 129 active+clean; 26 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 2.3 MiB/s wr, 524 op/s 2024-06-06T04:33:10.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:10 smithi086 ceph-mon[36053]: pgmap v482: 129 pgs: 129 active+clean; 26 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 2.3 MiB/s wr, 524 op/s 2024-06-06T04:33:12.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:12 smithi154 ceph-mon[36428]: pgmap v483: 129 pgs: 129 active+clean; 26 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.3 MiB/s wr, 547 op/s 2024-06-06T04:33:12.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:12 smithi045 ceph-mon[32299]: pgmap v483: 129 pgs: 129 active+clean; 26 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.3 MiB/s wr, 547 op/s 2024-06-06T04:33:12.608 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:12 smithi086 ceph-mon[36053]: pgmap v483: 129 pgs: 129 active+clean; 26 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.3 MiB/s wr, 547 op/s 2024-06-06T04:33:13.941 INFO:tasks.workunit.client.0.smithi045.stderr:Updating files: 8% (366/4296) Updating files: 9% (387/4296) Updating files: 10% (430/4296) Updating files: 11% (473/4296) Updating files: 11% (490/4296) Updating files: 12% (516/4296) Updating files: 13% (559/4296) Updating files: 14% (602/4296) Updating files: 15% (645/4296) Updating files: 16% (688/4296) Updating files: 17% (731/4296) Updating files: 18% (774/4296) Updating files: 18% (801/4296) Updating files: 19% (817/4296) Updating files: 20% (860/4296) Updating files: 21% (903/4296) Updating files: 22% (946/4296) Updating files: 23% (989/4296) Updating files: 24% (1032/4296) Updating files: 25% (1074/4296) Updating files: 26% (1117/4296) Updating files: 26% (1155/4296) Updating files: 27% (1160/4296) Updating files: 28% (1203/4296) Updating files: 29% (1246/4296) Updating files: 29% (1288/4296) Updating files: 30% (1289/4296) Updating files: 31% (1332/4296) Updating files: 32% (1375/4296) Updating files: 33% (1418/4296) Updating files: 34% (1461/4296) Updating files: 35% (1504/4296) Updating files: 35% (1545/4296) Updating files: 36% (1547/4296) Updating files: 37% (1590/4296) Updating files: 38% (1633/4296) Updating files: 39% (1676/4296) Updating files: 40% (1719/4296) Updating files: 41% (1762/4296) Updating files: 42% (1805/4296) Updating files: 43% (1848/4296) Updating files: 43% (1861/4296) Updating files: 44% (1891/4296) Updating files: 45% (1934/4296) Updating files: 46% (1977/4296) Updating files: 47% (2020/4296) Updating files: 48% (2063/4296) Updating files: 49% (2106/4296) Updating files: 50% (2148/4296) Updating files: 51% (2191/4296) Updating files: 52% (2234/4296) Updating files: 53% (2277/4296) Updating files: 54% (2320/4296) Updating files: 55% (2363/4296) Updating files: 55% (2398/4296) Updating files: 56% (2406/4296) Updating files: 57% (2449/4296) Updating files: 58% (2492/4296) Updating files: 59% (2535/4296) Updating files: 60% (2578/4296) Updating files: 61% (2621/4296) Updating files: 62% (2664/4296) Updating files: 63% (2707/4296) Updating files: 64% (2750/4296) Updating files: 65% (2793/4296) Updating files: 66% (2836/4296) Updating files: 67% (2879/4296) Updating files: 68% (2922/4296) Updating files: 69% (2965/4296) Updating files: 70% (3008/4296) Updating files: 70% (3028/4296) Updating files: 71% (3051/4296) Updating files: 72% (3094/4296) Updating files: 73% (3137/4296) Updating files: 74% (3180/4296) Updating files: 75% (3222/4296) Updating files: 76% (3265/4296) Updating files: 77% (3308/4296) Updating files: 78% (3351/4296) Updating files: 78% (3370/4296) Updating files: 79% (3394/4296) Updating files: 80% (3437/4296) Updating files: 81% (3480/4296) Updating files: 82% (3523/4296) Updating files: 83% (3566/4296) Updating files: 84% (3609/4296) Updating files: 85% (3652/4296) Updating files: 86% (3695/4296) Updating files: 86% (3716/4296) Updating files: 87% (3738/4296) Updating files: 88% (3781/4296) Updating files: 89% (3824/4296) Updating files: 90% (3867/4296) Updating files: 91% (3910/4296) Updating files: 91% (3915/4296) Updating files: 92% (3953/4296) Updating files: 93% (3996/4296) Updating files: 94% (4039/4296) Updating files: 95% (4082/4296) Updating files: 96% (4125/4296) Updating files: 96% (4160/4296) Updating files: 97% (4168/4296) Updating files: 98% (4211/4296) Updating files: 99% (4254/4296) Updating files: 100% (4296/4296) Updating files: 100% (4296/4296), done. 2024-06-06T04:33:13.952 INFO:tasks.workunit.client.0.smithi045.stderr:+ cd xfstests-dev 2024-06-06T04:33:13.953 INFO:tasks.workunit.client.0.smithi045.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2024-06-06T04:33:14.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:14 smithi154 ceph-mon[36428]: pgmap v484: 129 pgs: 129 active+clean; 28 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.2 MiB/s wr, 586 op/s 2024-06-06T04:33:14.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:14 smithi045 ceph-mon[32299]: pgmap v484: 129 pgs: 129 active+clean; 28 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.2 MiB/s wr, 586 op/s 2024-06-06T04:33:14.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:14 smithi086 ceph-mon[36053]: pgmap v484: 129 pgs: 129 active+clean; 28 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.2 MiB/s wr, 586 op/s 2024-06-06T04:33:16.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:16 smithi154 ceph-mon[36428]: pgmap v485: 129 pgs: 129 active+clean; 34 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.6 MiB/s wr, 649 op/s 2024-06-06T04:33:16.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:16 smithi045 ceph-mon[32299]: pgmap v485: 129 pgs: 129 active+clean; 34 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.6 MiB/s wr, 649 op/s 2024-06-06T04:33:16.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:16 smithi086 ceph-mon[36053]: pgmap v485: 129 pgs: 129 active+clean; 34 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.6 MiB/s wr, 649 op/s 2024-06-06T04:33:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.870+0000 7f57e2888640 1 -- 172.21.15.45:0/3618179447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc104e50 msgr2=0x7f57dc107240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.870+0000 7f57e2888640 1 --2- 172.21.15.45:0/3618179447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc104e50 0x7f57dc107240 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009900 tx=0x7f57d002a410 comp rx=0 tx=0).stop 2024-06-06T04:33:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.871+0000 7f57e2888640 1 -- 172.21.15.45:0/3618179447 shutdown_connections 2024-06-06T04:33:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.871+0000 7f57e2888640 1 --2- 172.21.15.45:0/3618179447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57dc107780 0x7f57dc109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.871+0000 7f57e2888640 1 --2- 172.21.15.45:0/3618179447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc104e50 0x7f57dc107240 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.871+0000 7f57e2888640 1 --2- 172.21.15.45:0/3618179447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc102520 0x7f57dc104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.871+0000 7f57e2888640 1 -- 172.21.15.45:0/3618179447 >> 172.21.15.45:0/3618179447 conn(0x7f57dc0fbeb0 msgr2=0x7f57dc0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.871+0000 7f57e2888640 1 -- 172.21.15.45:0/3618179447 shutdown_connections 2024-06-06T04:33:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.871+0000 7f57e2888640 1 -- 172.21.15.45:0/3618179447 wait complete. 2024-06-06T04:33:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.871+0000 7f57e2888640 1 Processor -- start 2024-06-06T04:33:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.871+0000 7f57e2888640 1 -- start start 2024-06-06T04:33:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57e2888640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc102520 0x7f57dc197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57e2888640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc104e50 0x7f57dc198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57e2888640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57dc107780 0x7f57dc19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57e2888640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc10d540 con 0x7f57dc102520 2024-06-06T04:33:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57e2888640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc10d3c0 con 0x7f57dc107780 2024-06-06T04:33:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57e2888640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc10d6c0 con 0x7f57dc104e50 2024-06-06T04:33:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57dbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc102520 0x7f57dc197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57dbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc102520 0x7f57dc197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40834/0 (socket says 172.21.15.45:40834) 2024-06-06T04:33:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57dbfff640 1 -- 172.21.15.45:0/3838542980 learned_addr learned my addr 172.21.15.45:0/3838542980 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57db7fe640 1 --2- 172.21.15.45:0/3838542980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc104e50 0x7f57dc198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57e0e25640 1 --2- 172.21.15.45:0/3838542980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57dc107780 0x7f57dc19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57dbfff640 1 -- 172.21.15.45:0/3838542980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc104e50 msgr2=0x7f57dc198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57dbfff640 1 --2- 172.21.15.45:0/3838542980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc104e50 0x7f57dc198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57dbfff640 1 -- 172.21.15.45:0/3838542980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57dc107780 msgr2=0x7f57dc19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57dbfff640 1 --2- 172.21.15.45:0/3838542980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57dc107780 0x7f57dc19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57dbfff640 1 -- 172.21.15.45:0/3838542980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc19f800 con 0x7f57dc102520 2024-06-06T04:33:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57dbfff640 1 --2- 172.21.15.45:0/3838542980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc102520 0x7f57dc197be0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00d950 tx=0x7f57cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.872+0000 7f57d97fa640 1 -- 172.21.15.45:0/3838542980 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc017070 con 0x7f57dc102520 2024-06-06T04:33:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.873+0000 7f57e2888640 1 -- 172.21.15.45:0/3838542980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57dc19faf0 con 0x7f57dc102520 2024-06-06T04:33:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.873+0000 7f57d97fa640 1 -- 172.21.15.45:0/3838542980 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57cc009c30 con 0x7f57dc102520 2024-06-06T04:33:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.873+0000 7f57d97fa640 1 -- 172.21.15.45:0/3838542980 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc020060 con 0x7f57dc102520 2024-06-06T04:33:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.873+0000 7f57e2888640 1 -- 172.21.15.45:0/3838542980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57dc1a0060 con 0x7f57dc102520 2024-06-06T04:33:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.875+0000 7f57d97fa640 1 -- 172.21.15.45:0/3838542980 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f57cc0109a0 con 0x7f57dc102520 2024-06-06T04:33:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.875+0000 7f57e5591740 1 Processor -- start 2024-06-06T04:33:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.875+0000 7f57e5591740 1 -- start start 2024-06-06T04:33:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.875+0000 7f57e5591740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a332010 0x556b7a332430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.875+0000 7f57e5591740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a337500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.875+0000 7f57e5591740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a337b70 0x556b7a33a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.875+0000 7f57e5591740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556b7a2954f0 con 0x556b7a332a00 2024-06-06T04:33:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.875+0000 7f57e5591740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556b7a233870 con 0x556b7a337b70 2024-06-06T04:33:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57e5591740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556b7a2914b0 con 0x556b7a332010 2024-06-06T04:33:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57d97fa640 1 --2- 172.21.15.45:0/3838542980 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f57c40a2c30 0x7f57c40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57dbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a337500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57db7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a337b70 0x556b7a33a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57dbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a337500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40840/0 (socket says 172.21.15.45:40840) 2024-06-06T04:33:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57db7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a337b70 0x556b7a33a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:33196/0 (socket says 172.21.15.45:33196) 2024-06-06T04:33:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57dbfff640 1 -- 172.21.15.45:0/3389267381 learned_addr learned my addr 172.21.15.45:0/3389267381 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57e0e25640 1 --2- 172.21.15.45:0/3838542980 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f57c40a2c30 0x7f57c40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57e0e25640 1 --2- 172.21.15.45:0/3389267381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a332010 0x556b7a332430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57d97fa640 1 -- 172.21.15.45:0/3838542980 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f57cc0cf000 con 0x7f57dc102520 2024-06-06T04:33:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57dbfff640 1 -- 172.21.15.45:0/3389267381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a332010 msgr2=0x556b7a332430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57dbfff640 1 --2- 172.21.15.45:0/3389267381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a332010 0x556b7a332430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57dbfff640 1 -- 172.21.15.45:0/3389267381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a337b70 msgr2=0x556b7a33a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57dbfff640 1 --2- 172.21.15.45:0/3389267381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a337b70 0x556b7a33a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57dbfff640 1 -- 172.21.15.45:0/3389267381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b7a24f9b0 con 0x556b7a332a00 2024-06-06T04:33:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57dbfff640 1 --2- 172.21.15.45:0/3389267381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a337500 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f57cc09b4e0 tx=0x7f57cc06e610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bff12f97e727d2c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57e0e25640 1 --2- 172.21.15.45:0/3838542980 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f57c40a2c30 0x7f57c40a50f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f57d4005fd0 tx=0x7f57d4009f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57c27fc640 1 -- 172.21.15.45:0/3389267381 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0a09e0 con 0x556b7a332a00 2024-06-06T04:33:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57c27fc640 1 -- 172.21.15.45:0/3389267381 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57cc09c590 con 0x556b7a332a00 2024-06-06T04:33:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.876+0000 7f57c27fc640 1 -- 172.21.15.45:0/3389267381 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc09c2b0 con 0x556b7a332a00 2024-06-06T04:33:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.877+0000 7f57e5591740 1 -- 172.21.15.45:0/3389267381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 msgr2=0x556b7a337500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.877+0000 7f57e5591740 1 --2- 172.21.15.45:0/3389267381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a337500 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f57cc09b4e0 tx=0x7f57cc06e610 comp rx=0 tx=0).stop 2024-06-06T04:33:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.877+0000 7f57e5591740 1 -- 172.21.15.45:0/3389267381 shutdown_connections 2024-06-06T04:33:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.877+0000 7f57e5591740 1 --2- 172.21.15.45:0/3389267381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a337b70 0x556b7a33a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.877+0000 7f57e5591740 1 --2- 172.21.15.45:0/3389267381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a337500 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.877+0000 7f57e5591740 1 --2- 172.21.15.45:0/3389267381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a332010 0x556b7a332430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.877+0000 7f57e5591740 1 -- 172.21.15.45:0/3389267381 >> 172.21.15.45:0/3389267381 conn(0x556b7a32d680 msgr2=0x556b7a32da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.877+0000 7f57e5591740 1 -- 172.21.15.45:0/3389267381 shutdown_connections 2024-06-06T04:33:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.877+0000 7f57e5591740 1 -- 172.21.15.45:0/3389267381 wait complete. 2024-06-06T04:33:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.877+0000 7f57e5591740 1 -- start start 2024-06-06T04:33:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57e5591740 1 Processor -- start 2024-06-06T04:33:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57e5591740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a332010 0x556b7a33c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57e5591740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a33c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57e5591740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a337b70 0x556b7a3436e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57e5591740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556b7a33aac0 con 0x556b7a332a00 2024-06-06T04:33:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57e5591740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556b7a21eae0 con 0x556b7a332010 2024-06-06T04:33:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57e5591740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556b7a33ac40 con 0x556b7a337b70 2024-06-06T04:33:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57e5591740 10 client.0 _ll_register_callbacks cb 0x556b7a2579b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57db7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a33c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57db7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a33c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40856/0 (socket says 172.21.15.45:40856) 2024-06-06T04:33:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57db7fe640 1 -- 172.21.15.45:0/3013349375 learned_addr learned my addr 172.21.15.45:0/3013349375 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57dbfff640 1 --2- 172.21.15.45:0/3013349375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a332010 0x556b7a33c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57e0e25640 1 --2- 172.21.15.45:0/3013349375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a337b70 0x556b7a3436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57db7fe640 1 -- 172.21.15.45:0/3013349375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a337b70 msgr2=0x556b7a3436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57db7fe640 1 --2- 172.21.15.45:0/3013349375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a337b70 0x556b7a3436e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57db7fe640 1 -- 172.21.15.45:0/3013349375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a332010 msgr2=0x556b7a33c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57db7fe640 1 --2- 172.21.15.45:0/3013349375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a332010 0x556b7a33c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57db7fe640 1 -- 172.21.15.45:0/3013349375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b79fa9210 con 0x556b7a332a00 2024-06-06T04:33:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.878+0000 7f57e0e25640 1 --2- 172.21.15.45:0/3013349375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a337b70 0x556b7a3436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:33:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.879+0000 7f57db7fe640 1 --2- 172.21.15.45:0/3013349375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a33c6a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009900 tx=0x7f57d0009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692e3a4e8f34ba53 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.879+0000 7f57bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:33:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.879+0000 7f57bb7fe640 1 -- 172.21.15.45:0/3013349375 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0035040 con 0x556b7a332a00 2024-06-06T04:33:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.879+0000 7f57e5591740 1 -- 172.21.15.45:0/3013349375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57cc09c590 con 0x556b7a332a00 2024-06-06T04:33:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.879+0000 7f57bb7fe640 1 -- 172.21.15.45:0/3013349375 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f57d0004240 con 0x556b7a332a00 2024-06-06T04:33:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.879+0000 7f57bb7fe640 1 -- 172.21.15.45:0/3013349375 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f57d004e050 con 0x556b7a332a00 2024-06-06T04:33:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.879+0000 7f57e5591740 10 client.15669 fetch_fsmap learned FSMap version 29 2024-06-06T04:33:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.879+0000 7f57e5591740 1 -- 172.21.15.45:0/3013349375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b7a075620 con 0x556b7a332a00 2024-06-06T04:33:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.879+0000 7f57bb7fe640 1 -- 172.21.15.45:0/3013349375 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57cc09c590 con 0x556b7a332a00 2024-06-06T04:33:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.879+0000 7f57bb7fe640 1 -- 172.21.15.45:0/3013349375 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f57d003c4b0 con 0x556b7a332a00 2024-06-06T04:33:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.879+0000 7f57e5591740 10 client.15669 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T04:33:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.879+0000 7f57e5591740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:33:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.879+0000 7f57e5591740 20 client.15669 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.879+0000 7f57e5591740 1 --2- 172.21.15.45:0/3013349375 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x556b7a348140 0x556b7a34a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.879+0000 7f57e5591740 4 client.15669 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:33:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.880+0000 7f57e5591740 1 -- 172.21.15.45:0/3013349375 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b7a2a8f10 con 0x556b7a348140 2024-06-06T04:33:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.880+0000 7f57dbfff640 1 --2- 172.21.15.45:0/3013349375 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x556b7a348140 0x556b7a34a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.880+0000 7f57bb7fe640 10 client.15669 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.880+0000 7f57dbfff640 1 --2- 172.21.15.45:0/3013349375 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x556b7a348140 0x556b7a34a570 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0ab28a2a748af79 server_cookie=90086cf5f233c065 in_seq=0 out_seq=0 2024-06-06T04:33:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.881+0000 7f57bb7fe640 1 -- 172.21.15.45:0/3013349375 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556b7a2a8f10 con 0x556b7a348140 2024-06-06T04:33:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.881+0000 7f57bb7fe640 10 client.15669 handle_command_reply: tid=0 2024-06-06T04:33:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.881+0000 7f57e5591740 1 client.15669 shutdown 2024-06-06T04:33:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.881+0000 7f57e5591740 1 -- 172.21.15.45:0/3013349375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 msgr2=0x556b7a33c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.881+0000 7f57e5591740 1 --2- 172.21.15.45:0/3013349375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a33c6a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009900 tx=0x7f57d0009670 comp rx=0 tx=0).stop 2024-06-06T04:33:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.881+0000 7f57e5591740 1 -- 172.21.15.45:0/3013349375 shutdown_connections 2024-06-06T04:33:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.881+0000 7f57e5591740 1 --2- 172.21.15.45:0/3013349375 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x556b7a348140 0x556b7a34a570 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.881+0000 7f57e5591740 1 --2- 172.21.15.45:0/3013349375 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a337b70 0x556b7a3436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.881+0000 7f57e5591740 1 --2- 172.21.15.45:0/3013349375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a33c6a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.881+0000 7f57e5591740 1 --2- 172.21.15.45:0/3013349375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a332010 0x556b7a33c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.881+0000 7f57bb7fe640 0 client.15669 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.881+0000 7f57e5591740 1 -- 172.21.15.45:0/3013349375 >> 172.21.15.45:0/3013349375 conn(0x556b7a32d680 msgr2=0x556b7a32c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.881+0000 7f57e5591740 1 -- 172.21.15.45:0/3013349375 shutdown_connections 2024-06-06T04:33:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.881+0000 7f57e5591740 1 -- 172.21.15.45:0/3013349375 wait complete. 2024-06-06T04:33:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.881+0000 7f57e5591740 20 client.15669 trim_cache size 0 max 16384 2024-06-06T04:33:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.897+0000 7f57e5591740 1 Processor -- start 2024-06-06T04:33:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.897+0000 7f57e5591740 1 -- start start 2024-06-06T04:33:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.897+0000 7f57e5591740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a332010 0x556b7a344130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.897+0000 7f57e5591740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a332a00 0x556b7a3436a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.897+0000 7f57e5591740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a337b70 0x556b7a33c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.897+0000 7f57e5591740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1acd50 con 0x556b7a337b70 2024-06-06T04:33:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.897+0000 7f57e5591740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1aa620 con 0x556b7a332a00 2024-06-06T04:33:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.897+0000 7f57e5591740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1a8470 con 0x556b7a332010 2024-06-06T04:33:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.898+0000 7f57e0e25640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a337b70 0x556b7a33c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.898+0000 7f57e0e25640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a337b70 0x556b7a33c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40862/0 (socket says 172.21.15.45:40862) 2024-06-06T04:33:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.898+0000 7f57e0e25640 1 -- 172.21.15.45:0/2397152332 learned_addr learned my addr 172.21.15.45:0/2397152332 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.898+0000 7f57db7fe640 1 --2- 172.21.15.45:0/2397152332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a332a00 0x556b7a3436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.898+0000 7f57dbfff640 1 --2- 172.21.15.45:0/2397152332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a332010 0x556b7a344130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.898+0000 7f57e0e25640 1 -- 172.21.15.45:0/2397152332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a332010 msgr2=0x556b7a344130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.898+0000 7f57e0e25640 1 --2- 172.21.15.45:0/2397152332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a332010 0x556b7a344130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.898+0000 7f57e0e25640 1 -- 172.21.15.45:0/2397152332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a332a00 msgr2=0x556b7a3436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.898+0000 7f57e0e25640 1 --2- 172.21.15.45:0/2397152332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a332a00 0x556b7a3436a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.898+0000 7f57e0e25640 1 -- 172.21.15.45:0/2397152332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b7a32cf30 con 0x556b7a337b70 2024-06-06T04:33:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.898+0000 7f57dbfff640 1 --2- 172.21.15.45:0/2397152332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a332010 0x556b7a344130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:33:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.898+0000 7f57e0e25640 1 --2- 172.21.15.45:0/2397152332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a337b70 0x556b7a33c160 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f57d400ff70 tx=0x7f57d4021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a35281e271c614 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.898+0000 7f57c1ffb640 1 -- 172.21.15.45:0/2397152332 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d402d070 con 0x556b7a337b70 2024-06-06T04:33:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.898+0000 7f57c1ffb640 1 -- 172.21.15.45:0/2397152332 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57d4011c50 con 0x556b7a337b70 2024-06-06T04:33:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.898+0000 7f57c1ffb640 1 -- 172.21.15.45:0/2397152332 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d401fbe0 con 0x556b7a337b70 2024-06-06T04:33:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.899+0000 7f57e5591740 1 -- 172.21.15.45:0/2397152332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a337b70 msgr2=0x556b7a33c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.899+0000 7f57e5591740 1 --2- 172.21.15.45:0/2397152332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a337b70 0x556b7a33c160 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f57d400ff70 tx=0x7f57d4021000 comp rx=0 tx=0).stop 2024-06-06T04:33:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.899+0000 7f57e5591740 1 -- 172.21.15.45:0/2397152332 shutdown_connections 2024-06-06T04:33:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.899+0000 7f57e5591740 1 --2- 172.21.15.45:0/2397152332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a332a00 0x556b7a3436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.899+0000 7f57e5591740 1 --2- 172.21.15.45:0/2397152332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a337b70 0x556b7a33c160 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.899+0000 7f57e5591740 1 --2- 172.21.15.45:0/2397152332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a332010 0x556b7a344130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.899+0000 7f57e5591740 1 -- 172.21.15.45:0/2397152332 >> 172.21.15.45:0/2397152332 conn(0x556b7a32d680 msgr2=0x556b7a32da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.899+0000 7f57e5591740 1 -- 172.21.15.45:0/2397152332 shutdown_connections 2024-06-06T04:33:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.899+0000 7f57e5591740 1 -- 172.21.15.45:0/2397152332 wait complete. 2024-06-06T04:33:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.899+0000 7f57e5591740 1 -- start start 2024-06-06T04:33:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57e5591740 1 Processor -- start 2024-06-06T04:33:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57e5591740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a332010 0x556b7a340460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57e5591740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a3409a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57e5591740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a337b70 0x556b7a3b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57e5591740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1a8ae0 con 0x556b7a332a00 2024-06-06T04:33:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57e5591740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1a9f50 con 0x556b7a332010 2024-06-06T04:33:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57e5591740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1a91b0 con 0x556b7a337b70 2024-06-06T04:33:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57e5591740 10 client.0 _ll_register_callbacks cb 0x7f57d0031af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57db7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a3409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57db7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a3409a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40876/0 (socket says 172.21.15.45:40876) 2024-06-06T04:33:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57db7fe640 1 -- 172.21.15.45:0/1321522871 learned_addr learned my addr 172.21.15.45:0/1321522871 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57dbfff640 1 --2- 172.21.15.45:0/1321522871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a332010 0x556b7a340460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57e0e25640 1 --2- 172.21.15.45:0/1321522871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a337b70 0x556b7a3b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57db7fe640 1 -- 172.21.15.45:0/1321522871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a337b70 msgr2=0x556b7a3b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57db7fe640 1 --2- 172.21.15.45:0/1321522871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a337b70 0x556b7a3b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57db7fe640 1 -- 172.21.15.45:0/1321522871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a332010 msgr2=0x556b7a340460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57db7fe640 1 --2- 172.21.15.45:0/1321522871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a332010 0x556b7a340460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57db7fe640 1 -- 172.21.15.45:0/1321522871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b7a341010 con 0x556b7a332a00 2024-06-06T04:33:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57db7fe640 1 --2- 172.21.15.45:0/1321522871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a3409a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f57d003c950 tx=0x7f57d0031ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2405f5917906d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.900+0000 7f57bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:33:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.901+0000 7f57bb7fe640 1 -- 172.21.15.45:0/1321522871 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0057060 con 0x556b7a332a00 2024-06-06T04:33:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.901+0000 7f57e5591740 1 -- 172.21.15.45:0/1321522871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57d4011c50 con 0x556b7a332a00 2024-06-06T04:33:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.901+0000 7f57bb7fe640 1 -- 172.21.15.45:0/1321522871 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f57d0004040 con 0x556b7a332a00 2024-06-06T04:33:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.901+0000 7f57bb7fe640 1 -- 172.21.15.45:0/1321522871 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f57d005e050 con 0x556b7a332a00 2024-06-06T04:33:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.901+0000 7f57e5591740 10 client.15675 fetch_fsmap learned FSMap version 29 2024-06-06T04:33:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.901+0000 7f57e5591740 1 -- 172.21.15.45:0/1321522871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b7a3b24a0 con 0x556b7a332a00 2024-06-06T04:33:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.901+0000 7f57bb7fe640 1 -- 172.21.15.45:0/1321522871 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57d4011c50 con 0x556b7a332a00 2024-06-06T04:33:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.901+0000 7f57bb7fe640 1 -- 172.21.15.45:0/1321522871 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f57d0003aa0 con 0x556b7a332a00 2024-06-06T04:33:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.901+0000 7f57e5591740 10 client.15675 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T04:33:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.901+0000 7f57e5591740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:33:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.901+0000 7f57e5591740 20 client.15675 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.901+0000 7f57e5591740 1 --2- 172.21.15.45:0/1321522871 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x556b7a3b2e80 0x556b7a3b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.901+0000 7f57e5591740 4 client.15675 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T04:33:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.901+0000 7f57e5591740 1 -- 172.21.15.45:0/1321522871 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b7a2a2c20 con 0x556b7a3b2e80 2024-06-06T04:33:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.901+0000 7f57dbfff640 1 --2- 172.21.15.45:0/1321522871 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x556b7a3b2e80 0x556b7a3b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.902+0000 7f57bb7fe640 10 client.15675 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:16.902+0000 7f57dbfff640 1 --2- 172.21.15.45:0/1321522871 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x556b7a3b2e80 0x556b7a3b52a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54b15db53e3a5470 server_cookie=238b81c3a9bb34c5 in_seq=0 out_seq=0 2024-06-06T04:33:18.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:18 smithi154 ceph-mon[36428]: pgmap v486: 129 pgs: 129 active+clean; 38 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 579 op/s 2024-06-06T04:33:18.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:18 smithi045 ceph-mon[32299]: pgmap v486: 129 pgs: 129 active+clean; 38 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 579 op/s 2024-06-06T04:33:18.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:18 smithi086 ceph-mon[36053]: pgmap v486: 129 pgs: 129 active+clean; 38 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 579 op/s 2024-06-06T04:33:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.562+0000 7f57bb7fe640 1 -- 172.21.15.45:0/1321522871 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556b7a2a2c20 con 0x556b7a3b2e80 2024-06-06T04:33:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.562+0000 7f57bb7fe640 10 client.15675 handle_command_reply: tid=0 2024-06-06T04:33:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.562+0000 7f57e5591740 1 client.15675 shutdown 2024-06-06T04:33:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.563+0000 7f57e5591740 1 -- 172.21.15.45:0/1321522871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 msgr2=0x556b7a3409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.563+0000 7f57e5591740 1 --2- 172.21.15.45:0/1321522871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a3409a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f57d003c950 tx=0x7f57d0031ba0 comp rx=0 tx=0).stop 2024-06-06T04:33:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.563+0000 7f57e5591740 1 -- 172.21.15.45:0/1321522871 shutdown_connections 2024-06-06T04:33:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.563+0000 7f57e5591740 1 --2- 172.21.15.45:0/1321522871 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x556b7a3b2e80 0x556b7a3b52a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.563+0000 7f57e5591740 1 --2- 172.21.15.45:0/1321522871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b7a337b70 0x556b7a3b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.563+0000 7f57e5591740 1 --2- 172.21.15.45:0/1321522871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b7a332a00 0x556b7a3409a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.563+0000 7f57bb7fe640 0 client.15675 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.563+0000 7f57e5591740 1 --2- 172.21.15.45:0/1321522871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b7a332010 0x556b7a340460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.563+0000 7f57e5591740 1 -- 172.21.15.45:0/1321522871 >> 172.21.15.45:0/1321522871 conn(0x556b7a32d680 msgr2=0x556b7a333ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.563+0000 7f57e5591740 1 -- 172.21.15.45:0/1321522871 shutdown_connections 2024-06-06T04:33:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.563+0000 7f57e5591740 1 -- 172.21.15.45:0/1321522871 wait complete. 2024-06-06T04:33:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.563+0000 7f57e5591740 20 client.15675 trim_cache size 0 max 16384 2024-06-06T04:33:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-06-06T04:33:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-06-06T04:33:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-06-06T04:33:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.563+0000 7f57e2888640 1 -- 172.21.15.45:0/3838542980 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f57c40a2c30 msgr2=0x7f57c40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.563+0000 7f57e2888640 1 --2- 172.21.15.45:0/3838542980 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f57c40a2c30 0x7f57c40a50f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f57d4005fd0 tx=0x7f57d4009f00 comp rx=0 tx=0).stop 2024-06-06T04:33:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.563+0000 7f57e2888640 1 -- 172.21.15.45:0/3838542980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc102520 msgr2=0x7f57dc197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.563+0000 7f57e2888640 1 --2- 172.21.15.45:0/3838542980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc102520 0x7f57dc197be0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00d950 tx=0x7f57cc00de10 comp rx=0 tx=0).stop 2024-06-06T04:33:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.564+0000 7f57e2888640 1 -- 172.21.15.45:0/3838542980 shutdown_connections 2024-06-06T04:33:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.564+0000 7f57e2888640 1 --2- 172.21.15.45:0/3838542980 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f57c40a2c30 0x7f57c40a50f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.564+0000 7f57e2888640 1 --2- 172.21.15.45:0/3838542980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57dc107780 0x7f57dc19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.564+0000 7f57e2888640 1 --2- 172.21.15.45:0/3838542980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc104e50 0x7f57dc198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.564+0000 7f57e2888640 1 --2- 172.21.15.45:0/3838542980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc102520 0x7f57dc197be0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.564+0000 7f57e2888640 1 -- 172.21.15.45:0/3838542980 >> 172.21.15.45:0/3838542980 conn(0x7f57dc0fbeb0 msgr2=0x7f57dc102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.564+0000 7f57e2888640 1 -- 172.21.15.45:0/3838542980 shutdown_connections 2024-06-06T04:33:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:19.564+0000 7f57e2888640 1 -- 172.21.15.45:0/3838542980 wait complete. 2024-06-06T04:33:20.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:20 smithi154 ceph-mon[36428]: pgmap v487: 129 pgs: 129 active+clean; 39 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 552 op/s 2024-06-06T04:33:20.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:20 smithi154 ceph-mon[36428]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:33:20.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:20 smithi045 ceph-mon[32299]: pgmap v487: 129 pgs: 129 active+clean; 39 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 552 op/s 2024-06-06T04:33:20.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:20 smithi045 ceph-mon[32299]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:33:20.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:20 smithi086 ceph-mon[36053]: pgmap v487: 129 pgs: 129 active+clean; 39 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 552 op/s 2024-06-06T04:33:20.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:20 smithi086 ceph-mon[36053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:33:22.410 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:22 smithi086 ceph-mon[36053]: pgmap v488: 129 pgs: 129 active+clean; 40 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.4 MiB/s wr, 385 op/s 2024-06-06T04:33:22.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:22 smithi154 ceph-mon[36428]: pgmap v488: 129 pgs: 129 active+clean; 40 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.4 MiB/s wr, 385 op/s 2024-06-06T04:33:22.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:22 smithi045 ceph-mon[32299]: pgmap v488: 129 pgs: 129 active+clean; 40 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.4 MiB/s wr, 385 op/s 2024-06-06T04:33:23.428 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T04:33:23.722 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:33:24.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:24 smithi154 ceph-mon[36428]: pgmap v489: 129 pgs: 129 active+clean; 31 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 626 op/s 2024-06-06T04:33:24.430 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:24 smithi045 ceph-mon[32299]: pgmap v489: 129 pgs: 129 active+clean; 31 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 626 op/s 2024-06-06T04:33:24.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:24 smithi086 ceph-mon[36053]: pgmap v489: 129 pgs: 129 active+clean; 31 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 626 op/s 2024-06-06T04:33:25.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.680+0000 7efe8fb40700 1 -- 172.21.15.45:0/2174161291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe88108500 msgr2=0x7efe881088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.680+0000 7efe8fb40700 1 --2- 172.21.15.45:0/2174161291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe88108500 0x7efe881088e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efe84009990 tx=0x7efe84009ca0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.680+0000 7efe8fb40700 1 -- 172.21.15.45:0/2174161291 shutdown_connections 2024-06-06T04:33:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.680+0000 7efe8fb40700 1 --2- 172.21.15.45:0/2174161291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe8806ab30 0x7efe8810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.680+0000 7efe8fb40700 1 --2- 172.21.15.45:0/2174161291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe88108500 0x7efe881088e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.680+0000 7efe8fb40700 1 --2- 172.21.15.45:0/2174161291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe8806a170 0x7efe8806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.680+0000 7efe8fb40700 1 -- 172.21.15.45:0/2174161291 >> 172.21.15.45:0/2174161291 conn(0x7efe88078340 msgr2=0x7efe88078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.680+0000 7efe8fb40700 1 -- 172.21.15.45:0/2174161291 shutdown_connections 2024-06-06T04:33:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.680+0000 7efe8fb40700 1 -- 172.21.15.45:0/2174161291 wait complete. 2024-06-06T04:33:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.681+0000 7efe8fb40700 1 Processor -- start 2024-06-06T04:33:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.681+0000 7efe8fb40700 1 -- start start 2024-06-06T04:33:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.681+0000 7efe8fb40700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe8806a170 0x7efe8810bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.681+0000 7efe8fb40700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe8806ab30 0x7efe8810c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.682+0000 7efe8fb40700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe88108500 0x7efe881a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.682+0000 7efe8fb40700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efe881026f0 con 0x7efe8806ab30 2024-06-06T04:33:25.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.682+0000 7efe8fb40700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe88102570 con 0x7efe8806a170 2024-06-06T04:33:25.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.682+0000 7efe8fb40700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efe88102870 con 0x7efe88108500 2024-06-06T04:33:25.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.682+0000 7efe8d0db700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe8806ab30 0x7efe8810c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.682+0000 7efe8e0dd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe88108500 0x7efe881a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.682+0000 7efe8e0dd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe88108500 0x7efe881a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:44546/0 (socket says 172.21.15.45:44546) 2024-06-06T04:33:25.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.682+0000 7efe8e0dd700 1 -- 172.21.15.45:0/506385260 learned_addr learned my addr 172.21.15.45:0/506385260 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:25.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.682+0000 7efe8d8dc700 1 --2- 172.21.15.45:0/506385260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe8806a170 0x7efe8810bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.682+0000 7efe8d8dc700 1 -- 172.21.15.45:0/506385260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe88108500 msgr2=0x7efe881a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:25.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.682+0000 7efe8d8dc700 1 --2- 172.21.15.45:0/506385260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe88108500 0x7efe881a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.682+0000 7efe8d8dc700 1 -- 172.21.15.45:0/506385260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe8806ab30 msgr2=0x7efe8810c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:25.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.682+0000 7efe8d8dc700 1 --2- 172.21.15.45:0/506385260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe8806ab30 0x7efe8810c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.682+0000 7efe8d8dc700 1 -- 172.21.15.45:0/506385260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe881a53a0 con 0x7efe8806a170 2024-06-06T04:33:25.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.682+0000 7efe8d8dc700 1 --2- 172.21.15.45:0/506385260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe8806a170 0x7efe8810bd80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe840059d0 tx=0x7efe84004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:25.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.683+0000 7efe7effd700 1 -- 172.21.15.45:0/506385260 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe84027070 con 0x7efe8806a170 2024-06-06T04:33:25.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.683+0000 7efe7effd700 1 -- 172.21.15.45:0/506385260 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe84022470 con 0x7efe8806a170 2024-06-06T04:33:25.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.683+0000 7efe8fb40700 1 -- 172.21.15.45:0/506385260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe881a5630 con 0x7efe8806a170 2024-06-06T04:33:25.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.683+0000 7efe8fb40700 1 -- 172.21.15.45:0/506385260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe881abd20 con 0x7efe8806a170 2024-06-06T04:33:25.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.683+0000 7efe7effd700 1 -- 172.21.15.45:0/506385260 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8400b610 con 0x7efe8806a170 2024-06-06T04:33:25.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.686+0000 7efe7effd700 1 -- 172.21.15.45:0/506385260 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efe8402a070 con 0x7efe8806a170 2024-06-06T04:33:25.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.686+0000 7efe7effd700 1 --2- 172.21.15.45:0/506385260 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7efe740a2ca0 0x7efe740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.687+0000 7efe7effd700 1 -- 172.21.15.45:0/506385260 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7efe840cffe0 con 0x7efe8806a170 2024-06-06T04:33:25.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.687+0000 7efe8d0db700 1 --2- 172.21.15.45:0/506385260 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7efe740a2ca0 0x7efe740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.687+0000 7efe8d0db700 1 --2- 172.21.15.45:0/506385260 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7efe740a2ca0 0x7efe740a5160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efe8810d370 tx=0x7efe78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:25.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.688+0000 7efe9a3c3b80 1 Processor -- start 2024-06-06T04:33:25.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.688+0000 7efe9a3c3b80 1 -- start start 2024-06-06T04:33:25.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe9a3c3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bb640 0x560fc50bba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe9a3c3b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50bbf60 0x560fc50b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe9a3c3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50b5f20 0x560fc50b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe9a3c3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560fc5007810 con 0x560fc50b5f20 2024-06-06T04:33:25.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe9a3c3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560fc4faf720 con 0x560fc50bb640 2024-06-06T04:33:25.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe9a3c3b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560fc50037d0 con 0x560fc50bbf60 2024-06-06T04:33:25.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe8d8dc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bb640 0x560fc50bba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe8d8dc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bb640 0x560fc50bba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:40544/0 (socket says 172.21.15.45:40544) 2024-06-06T04:33:25.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe8d8dc700 1 -- 172.21.15.45:0/4219408162 learned_addr learned my addr 172.21.15.45:0/4219408162 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:25.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe8e0dd700 1 --2- 172.21.15.45:0/4219408162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50b5f20 0x560fc50b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe8d0db700 1 --2- 172.21.15.45:0/4219408162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50bbf60 0x560fc50b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe8d8dc700 1 -- 172.21.15.45:0/4219408162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50bbf60 msgr2=0x560fc50b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:25.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe8d8dc700 1 --2- 172.21.15.45:0/4219408162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50bbf60 0x560fc50b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe8d8dc700 1 -- 172.21.15.45:0/4219408162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50b5f20 msgr2=0x560fc50b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:25.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe8d8dc700 1 --2- 172.21.15.45:0/4219408162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50b5f20 0x560fc50b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe8d8dc700 1 -- 172.21.15.45:0/4219408162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fc4fa5f50 con 0x560fc50bb640 2024-06-06T04:33:25.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe8e0dd700 1 --2- 172.21.15.45:0/4219408162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50b5f20 0x560fc50b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:25.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.689+0000 7efe8d0db700 1 --2- 172.21.15.45:0/4219408162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50bbf60 0x560fc50b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:33:25.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.690+0000 7efe8d8dc700 1 --2- 172.21.15.45:0/4219408162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bb640 0x560fc50bba20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efe84098ea0 tx=0x7efe8409c0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5983e02bdbd6b323 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:25.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.690+0000 7efe73fff700 1 -- 172.21.15.45:0/4219408162 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8409c4d0 con 0x560fc50bb640 2024-06-06T04:33:25.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.690+0000 7efe73fff700 1 -- 172.21.15.45:0/4219408162 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe840a0880 con 0x560fc50bb640 2024-06-06T04:33:25.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.690+0000 7efe73fff700 1 -- 172.21.15.45:0/4219408162 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe840a4c20 con 0x560fc50bb640 2024-06-06T04:33:25.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.691+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/4219408162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bb640 msgr2=0x560fc50bba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.691+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/4219408162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bb640 0x560fc50bba20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efe84098ea0 tx=0x7efe8409c0e0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.691+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/4219408162 shutdown_connections 2024-06-06T04:33:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.691+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/4219408162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50b5f20 0x560fc50b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.691+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/4219408162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bb640 0x560fc50bba20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.691+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/4219408162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50bbf60 0x560fc50b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.691+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/4219408162 >> 172.21.15.45:0/4219408162 conn(0x560fc4fe0cc0 msgr2=0x560fc50b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.691+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/4219408162 shutdown_connections 2024-06-06T04:33:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.691+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/4219408162 wait complete. 2024-06-06T04:33:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.692+0000 7efe9a3c3b80 1 -- start start 2024-06-06T04:33:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.692+0000 7efe9a3c3b80 1 Processor -- start 2024-06-06T04:33:25.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.692+0000 7efe9a3c3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50b5f20 0x560fc50c1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe9a3c3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bb640 0x560fc50bf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe9a3c3b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50bbf60 0x560fc50bf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe8d8dc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50b5f20 0x560fc50c1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe8d0db700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bb640 0x560fc50bf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe8e0dd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50bbf60 0x560fc50bf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe8e0dd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50bbf60 0x560fc50bf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:44564/0 (socket says 172.21.15.45:44564) 2024-06-06T04:33:25.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe8e0dd700 1 -- 172.21.15.45:0/3073319046 learned_addr learned my addr 172.21.15.45:0/3073319046 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:25.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3073319046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560fc50c26f0 con 0x560fc50bb640 2024-06-06T04:33:25.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3073319046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560fc4fa7bf0 con 0x560fc50b5f20 2024-06-06T04:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3073319046 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560fc50c2870 con 0x560fc50bbf60 2024-06-06T04:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe9a3c3b80 10 client.0 _ll_register_callbacks cb 0x560fc4fa1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe8e0dd700 1 -- 172.21.15.45:0/3073319046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50b5f20 msgr2=0x560fc50c1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe8e0dd700 1 --2- 172.21.15.45:0/3073319046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50b5f20 0x560fc50c1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe8e0dd700 1 -- 172.21.15.45:0/3073319046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bb640 msgr2=0x560fc50bf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe8e0dd700 1 --2- 172.21.15.45:0/3073319046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bb640 0x560fc50bf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe8e0dd700 1 -- 172.21.15.45:0/3073319046 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fc4f568d0 con 0x560fc50bbf60 2024-06-06T04:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.693+0000 7efe8d0db700 1 --2- 172.21.15.45:0/3073319046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bb640 0x560fc50bf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.694+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3073319046 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe840a0880 con 0x560fc50bbf60 2024-06-06T04:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.694+0000 7efe8e0dd700 1 --2- 172.21.15.45:0/3073319046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50bbf60 0x560fc50bf560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efe800143a0 tx=0x7efe800146b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61ae7af9855ee510 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.694+0000 7efe717fa700 10 client.25282 ms_handle_connect on v2:172.21.15.154:3300/0 2024-06-06T04:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.694+0000 7efe717fa700 1 -- 172.21.15.45:0/3073319046 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe800092d0 con 0x560fc50bbf60 2024-06-06T04:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.694+0000 7efe717fa700 1 -- 172.21.15.45:0/3073319046 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7efe80007500 con 0x560fc50bbf60 2024-06-06T04:33:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.694+0000 7efe717fa700 1 -- 172.21.15.45:0/3073319046 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efe80007760 con 0x560fc50bbf60 2024-06-06T04:33:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.694+0000 7efe9a3c3b80 10 client.25282 fetch_fsmap learned FSMap version 30 2024-06-06T04:33:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.694+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3073319046 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=30}) -- 0x560fc50ccee0 con 0x560fc50bbf60 2024-06-06T04:33:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.695+0000 7efe717fa700 1 -- 172.21.15.45:0/3073319046 <== mon.1 v2:172.21.15.154:3300/0 4 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7efe80007900 con 0x560fc50bbf60 2024-06-06T04:33:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.695+0000 7efe9a3c3b80 10 client.25282 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:33:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.695+0000 7efe9a3c3b80 10 client.25282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:33:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.695+0000 7efe9a3c3b80 20 client.25282 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.695+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/3073319046 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560fc50cf410 0x560fc50d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.695+0000 7efe9a3c3b80 4 client.25282 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:33:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.695+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3073319046 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fc5014d10 con 0x560fc50cf410 2024-06-06T04:33:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.695+0000 7efe8d8dc700 1 --2- 172.21.15.45:0/3073319046 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560fc50cf410 0x560fc50d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.695+0000 7efe717fa700 10 client.25282 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.695+0000 7efe8d8dc700 1 --2- 172.21.15.45:0/3073319046 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560fc50cf410 0x560fc50d1860 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f4e743612279826 server_cookie=6ac428543d585a0a in_seq=0 out_seq=0 2024-06-06T04:33:25.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.696+0000 7efe717fa700 1 -- 172.21.15.45:0/3073319046 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560fc5014d10 con 0x560fc50cf410 2024-06-06T04:33:25.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.696+0000 7efe717fa700 10 client.25282 handle_command_reply: tid=0 2024-06-06T04:33:25.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.696+0000 7efe9a3c3b80 1 client.25282 shutdown 2024-06-06T04:33:25.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.696+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3073319046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50bbf60 msgr2=0x560fc50bf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:25.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.696+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/3073319046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50bbf60 0x560fc50bf560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efe800143a0 tx=0x7efe800146b0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.697+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3073319046 shutdown_connections 2024-06-06T04:33:25.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.697+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/3073319046 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560fc50cf410 0x560fc50d1860 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.697+0000 7efe717fa700 0 client.25282 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:25.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.697+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/3073319046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50bbf60 0x560fc50bf560 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.697+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/3073319046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50b5f20 0x560fc50c1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.697+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/3073319046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bb640 0x560fc50bf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.697+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3073319046 >> 172.21.15.45:0/3073319046 conn(0x560fc4fe0cc0 msgr2=0x560fc50b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:25.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.697+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3073319046 shutdown_connections 2024-06-06T04:33:25.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.697+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3073319046 wait complete. 2024-06-06T04:33:25.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.697+0000 7efe9a3c3b80 20 client.25282 trim_cache size 0 max 16384 2024-06-06T04:33:25.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.738+0000 7efe9a3c3b80 1 Processor -- start 2024-06-06T04:33:25.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.738+0000 7efe9a3c3b80 1 -- start start 2024-06-06T04:33:25.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe9a3c3b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50b5f20 0x560fc50b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe9a3c3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bb640 0x560fc50bdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe9a3c3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bbf60 0x560fc50bcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe9a3c3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efe881b70d0 con 0x560fc50bb640 2024-06-06T04:33:25.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe9a3c3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe881b4930 con 0x560fc50bbf60 2024-06-06T04:33:25.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe9a3c3b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efe881b2780 con 0x560fc50b5f20 2024-06-06T04:33:25.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe8e0dd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bbf60 0x560fc50bcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe8e0dd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bbf60 0x560fc50bcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:40570/0 (socket says 172.21.15.45:40570) 2024-06-06T04:33:25.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe8e0dd700 1 -- 172.21.15.45:0/3017018937 learned_addr learned my addr 172.21.15.45:0/3017018937 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:25.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe8d8dc700 1 --2- 172.21.15.45:0/3017018937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50b5f20 0x560fc50b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe8d0db700 1 --2- 172.21.15.45:0/3017018937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bb640 0x560fc50bdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe8e0dd700 1 -- 172.21.15.45:0/3017018937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50b5f20 msgr2=0x560fc50b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:25.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe8e0dd700 1 --2- 172.21.15.45:0/3017018937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50b5f20 0x560fc50b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe8e0dd700 1 -- 172.21.15.45:0/3017018937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bb640 msgr2=0x560fc50bdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:25.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe8e0dd700 1 --2- 172.21.15.45:0/3017018937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bb640 0x560fc50bdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe8e0dd700 1 -- 172.21.15.45:0/3017018937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fc50b6300 con 0x560fc50bbf60 2024-06-06T04:33:25.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.739+0000 7efe8d0db700 1 --2- 172.21.15.45:0/3017018937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bb640 0x560fc50bdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:33:25.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.740+0000 7efe8e0dd700 1 --2- 172.21.15.45:0/3017018937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bbf60 0x560fc50bcb10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efe80015010 tx=0x7efe80019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c5612e95c8fa4c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:25.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.740+0000 7efe737fe700 1 -- 172.21.15.45:0/3017018937 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80007500 con 0x560fc50bbf60 2024-06-06T04:33:25.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.740+0000 7efe737fe700 1 -- 172.21.15.45:0/3017018937 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe840a0880 con 0x560fc50bbf60 2024-06-06T04:33:25.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.740+0000 7efe737fe700 1 -- 172.21.15.45:0/3017018937 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80003a00 con 0x560fc50bbf60 2024-06-06T04:33:25.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.741+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3017018937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bbf60 msgr2=0x560fc50bcb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:25.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.741+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/3017018937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bbf60 0x560fc50bcb10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efe80015010 tx=0x7efe80019000 comp rx=0 tx=0).stop 2024-06-06T04:33:25.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.741+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3017018937 shutdown_connections 2024-06-06T04:33:25.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.741+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/3017018937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bbf60 0x560fc50bcb10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.741+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/3017018937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50b5f20 0x560fc50b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.741+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/3017018937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bb640 0x560fc50bdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.741+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3017018937 >> 172.21.15.45:0/3017018937 conn(0x560fc4fe0cc0 msgr2=0x560fc50bc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.741+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3017018937 shutdown_connections 2024-06-06T04:33:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.741+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3017018937 wait complete. 2024-06-06T04:33:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.742+0000 7efe9a3c3b80 1 -- start start 2024-06-06T04:33:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.742+0000 7efe9a3c3b80 1 Processor -- start 2024-06-06T04:33:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.742+0000 7efe9a3c3b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50b5f20 0x560fc50d6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.742+0000 7efe9a3c3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bb640 0x560fc50d72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.742+0000 7efe9a3c3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bbf60 0x560fc50d77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.742+0000 7efe9a3c3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efe881b2df0 con 0x560fc50bbf60 2024-06-06T04:33:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.742+0000 7efe9a3c3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe881b4260 con 0x560fc50bb640 2024-06-06T04:33:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.742+0000 7efe9a3c3b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efe881b34c0 con 0x560fc50b5f20 2024-06-06T04:33:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.743+0000 7efe9a3c3b80 10 client.0 _ll_register_callbacks cb 0x560fc4fa1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.743+0000 7efe8d0db700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bb640 0x560fc50d72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.743+0000 7efe8d0db700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bb640 0x560fc50d72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:40580/0 (socket says 172.21.15.45:40580) 2024-06-06T04:33:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.743+0000 7efe8d0db700 1 -- 172.21.15.45:0/3112113998 learned_addr learned my addr 172.21.15.45:0/3112113998 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.743+0000 7efe8d8dc700 1 --2- 172.21.15.45:0/3112113998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50b5f20 0x560fc50d6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.743+0000 7efe8e0dd700 1 --2- 172.21.15.45:0/3112113998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bbf60 0x560fc50d77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.743+0000 7efe8d8dc700 1 -- 172.21.15.45:0/3112113998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bb640 msgr2=0x560fc50d72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.743+0000 7efe8d8dc700 1 --2- 172.21.15.45:0/3112113998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bb640 0x560fc50d72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.743+0000 7efe8d8dc700 1 -- 172.21.15.45:0/3112113998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bbf60 msgr2=0x560fc50d77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.743+0000 7efe8d8dc700 1 --2- 172.21.15.45:0/3112113998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bbf60 0x560fc50d77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.743+0000 7efe8d8dc700 1 -- 172.21.15.45:0/3112113998 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fc5139130 con 0x560fc50b5f20 2024-06-06T04:33:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.743+0000 7efe8d8dc700 1 --2- 172.21.15.45:0/3112113998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50b5f20 0x560fc50d6d60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efe84005f50 tx=0x7efe84022900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adcc7378dc330b71 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.743+0000 7efe717fa700 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-06-06T04:33:25.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.743+0000 7efe717fa700 1 -- 172.21.15.45:0/3112113998 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe840229b0 con 0x560fc50b5f20 2024-06-06T04:33:25.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.744+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3112113998 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe840a0880 con 0x560fc50b5f20 2024-06-06T04:33:25.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.744+0000 7efe717fa700 1 -- 172.21.15.45:0/3112113998 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7efe8409c730 con 0x560fc50b5f20 2024-06-06T04:33:25.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.744+0000 7efe717fa700 1 -- 172.21.15.45:0/3112113998 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efe84022b50 con 0x560fc50b5f20 2024-06-06T04:33:25.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.744+0000 7efe9a3c3b80 10 client.25288 fetch_fsmap learned FSMap version 30 2024-06-06T04:33:25.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.744+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3112113998 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560fc513a3e0 con 0x560fc50b5f20 2024-06-06T04:33:25.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.745+0000 7efe717fa700 1 -- 172.21.15.45:0/3112113998 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe840a0880 con 0x560fc50b5f20 2024-06-06T04:33:25.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.745+0000 7efe717fa700 1 -- 172.21.15.45:0/3112113998 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7efe840cdc30 con 0x560fc50b5f20 2024-06-06T04:33:25.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.745+0000 7efe9a3c3b80 10 client.25288 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:33:25.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.745+0000 7efe9a3c3b80 10 client.25288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:33:25.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.745+0000 7efe9a3c3b80 20 client.25288 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:25.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.745+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/3112113998 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560fc50cf410 0x560fc513c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:25.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.745+0000 7efe9a3c3b80 4 client.25288 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T04:33:25.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.745+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3112113998 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560fc500f290 con 0x560fc50cf410 2024-06-06T04:33:25.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.745+0000 7efe8d0db700 1 --2- 172.21.15.45:0/3112113998 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560fc50cf410 0x560fc513c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:25.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.746+0000 7efe8d0db700 1 --2- 172.21.15.45:0/3112113998 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560fc50cf410 0x560fc513c970 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a2a94cb3a8b4ac server_cookie=a1e2222d02dd9b40 in_seq=0 out_seq=0 2024-06-06T04:33:25.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.746+0000 7efe717fa700 10 client.25288 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:25.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.746+0000 7efe717fa700 1 -- 172.21.15.45:0/3112113998 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560fc500f290 con 0x560fc50cf410 2024-06-06T04:33:25.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.746+0000 7efe717fa700 10 client.25288 handle_command_reply: tid=0 2024-06-06T04:33:25.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.746+0000 7efe9a3c3b80 1 client.25288 shutdown 2024-06-06T04:33:25.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.747+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3112113998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50b5f20 msgr2=0x560fc50d6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:25.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.747+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/3112113998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50b5f20 0x560fc50d6d60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efe84005f50 tx=0x7efe84022900 comp rx=0 tx=0).stop 2024-06-06T04:33:25.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.747+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3112113998 shutdown_connections 2024-06-06T04:33:25.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.747+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/3112113998 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560fc50cf410 0x560fc513c970 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.747+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/3112113998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fc50bbf60 0x560fc50d77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.747+0000 7efe717fa700 0 client.25288 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:25.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.747+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/3112113998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fc50b5f20 0x560fc50d6d60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.747+0000 7efe9a3c3b80 1 --2- 172.21.15.45:0/3112113998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560fc50bb640 0x560fc50d72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.747+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3112113998 >> 172.21.15.45:0/3112113998 conn(0x560fc4fe0cc0 msgr2=0x560fc50a9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:25.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.747+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3112113998 shutdown_connections 2024-06-06T04:33:25.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.747+0000 7efe9a3c3b80 1 -- 172.21.15.45:0/3112113998 wait complete. 2024-06-06T04:33:25.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.747+0000 7efe9a3c3b80 20 client.25288 trim_cache size 0 max 16384 2024-06-06T04:33:25.751 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:33:25.751 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-06-06T04:33:25.751 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-06-06T04:33:25.751 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:33:25.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.748+0000 7efe8fb40700 1 -- 172.21.15.45:0/506385260 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7efe740a2ca0 msgr2=0x7efe740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:25.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.748+0000 7efe8fb40700 1 --2- 172.21.15.45:0/506385260 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7efe740a2ca0 0x7efe740a5160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efe8810d370 tx=0x7efe78009260 comp rx=0 tx=0).stop 2024-06-06T04:33:25.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.748+0000 7efe8fb40700 1 -- 172.21.15.45:0/506385260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe8806a170 msgr2=0x7efe8810bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:25.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.748+0000 7efe8fb40700 1 --2- 172.21.15.45:0/506385260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe8806a170 0x7efe8810bd80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe840059d0 tx=0x7efe84004790 comp rx=0 tx=0).stop 2024-06-06T04:33:25.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.748+0000 7efe8fb40700 1 -- 172.21.15.45:0/506385260 shutdown_connections 2024-06-06T04:33:25.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.748+0000 7efe8fb40700 1 --2- 172.21.15.45:0/506385260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe88108500 0x7efe881a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.748+0000 7efe8fb40700 1 --2- 172.21.15.45:0/506385260 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7efe740a2ca0 0x7efe740a5160 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.748+0000 7efe8fb40700 1 --2- 172.21.15.45:0/506385260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe8806a170 0x7efe8810bd80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.748+0000 7efe8fb40700 1 --2- 172.21.15.45:0/506385260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe8806ab30 0x7efe8810c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:25.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.748+0000 7efe8fb40700 1 -- 172.21.15.45:0/506385260 >> 172.21.15.45:0/506385260 conn(0x7efe88078340 msgr2=0x7efe88106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:25.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.748+0000 7efe8fb40700 1 -- 172.21.15.45:0/506385260 shutdown_connections 2024-06-06T04:33:25.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:25.748+0000 7efe8fb40700 1 -- 172.21.15.45:0/506385260 wait complete. 2024-06-06T04:33:26.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:26 smithi154 ceph-mon[36428]: pgmap v490: 129 pgs: 129 active+clean; 28 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 1.15k op/s 2024-06-06T04:33:26.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:26 smithi045 ceph-mon[32299]: pgmap v490: 129 pgs: 129 active+clean; 28 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 1.15k op/s 2024-06-06T04:33:26.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:26 smithi086 ceph-mon[36053]: pgmap v490: 129 pgs: 129 active+clean; 28 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 1.15k op/s 2024-06-06T04:33:26.707 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-06T04:33:26.707 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:33:27.040 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:33:28.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:28 smithi154 ceph-mon[36428]: pgmap v491: 129 pgs: 129 active+clean; 28 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.2 MiB/s wr, 1.09k op/s 2024-06-06T04:33:28.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:28 smithi045 ceph-mon[32299]: pgmap v491: 129 pgs: 129 active+clean; 28 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.2 MiB/s wr, 1.09k op/s 2024-06-06T04:33:28.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:28 smithi086 ceph-mon[36053]: pgmap v491: 129 pgs: 129 active+clean; 28 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.2 MiB/s wr, 1.09k op/s 2024-06-06T04:33:29.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.622+0000 7f3c4ccdb700 1 -- 172.21.15.45:0/3626264647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c48108500 msgr2=0x7f3c481088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:29.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.622+0000 7f3c4ccdb700 1 --2- 172.21.15.45:0/3626264647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c48108500 0x7f3c481088e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009990 tx=0x7f3c38009ca0 comp rx=0 tx=0).stop 2024-06-06T04:33:29.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.622+0000 7f3c4ccdb700 1 -- 172.21.15.45:0/3626264647 shutdown_connections 2024-06-06T04:33:29.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.622+0000 7f3c4ccdb700 1 --2- 172.21.15.45:0/3626264647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4806b580 0x7f3c4810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:29.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.622+0000 7f3c4ccdb700 1 --2- 172.21.15.45:0/3626264647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c48108500 0x7f3c481088e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:29.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.622+0000 7f3c4ccdb700 1 --2- 172.21.15.45:0/3626264647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c4806abc0 0x7f3c4806b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:29.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.622+0000 7f3c4ccdb700 1 -- 172.21.15.45:0/3626264647 >> 172.21.15.45:0/3626264647 conn(0x7f3c48078340 msgr2=0x7f3c48078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:29.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.622+0000 7f3c4ccdb700 1 -- 172.21.15.45:0/3626264647 shutdown_connections 2024-06-06T04:33:29.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.622+0000 7f3c4ccdb700 1 -- 172.21.15.45:0/3626264647 wait complete. 2024-06-06T04:33:29.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.623+0000 7f3c4ccdb700 1 Processor -- start 2024-06-06T04:33:29.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.623+0000 7f3c4ccdb700 1 -- start start 2024-06-06T04:33:29.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c4ccdb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c4806abc0 0x7f3c48101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:29.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c4ccdb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c4806b580 0x7f3c481019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:29.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c4ccdb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c48108500 0x7f3c48101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:29.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c4ccdb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48112610 con 0x7f3c4806b580 2024-06-06T04:33:29.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c4ccdb700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48112490 con 0x7f3c4806abc0 2024-06-06T04:33:29.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c4ccdb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48112790 con 0x7f3c48108500 2024-06-06T04:33:29.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c4659c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c4806abc0 0x7f3c48101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:29.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c46d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c48108500 0x7f3c48101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:29.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c4659c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c4806abc0 0x7f3c48101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:59068/0 (socket says 172.21.15.45:59068) 2024-06-06T04:33:29.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c46d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c48108500 0x7f3c48101f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:42440/0 (socket says 172.21.15.45:42440) 2024-06-06T04:33:29.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c46d9d700 1 -- 172.21.15.45:0/2015353584 learned_addr learned my addr 172.21.15.45:0/2015353584 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:29.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c3dd9b700 1 --2- 172.21.15.45:0/2015353584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c4806b580 0x7f3c481019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:29.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c46d9d700 1 -- 172.21.15.45:0/2015353584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c4806abc0 msgr2=0x7f3c48101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:29.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c46d9d700 1 --2- 172.21.15.45:0/2015353584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c4806abc0 0x7f3c48101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:29.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c46d9d700 1 -- 172.21.15.45:0/2015353584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c4806b580 msgr2=0x7f3c481019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:29.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c46d9d700 1 --2- 172.21.15.45:0/2015353584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c4806b580 0x7f3c481019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:29.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c46d9d700 1 -- 172.21.15.45:0/2015353584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c481a7110 con 0x7f3c48108500 2024-06-06T04:33:29.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.624+0000 7f3c4659c700 1 --2- 172.21.15.45:0/2015353584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c4806abc0 0x7f3c48101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:29.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.625+0000 7f3c46d9d700 1 --2- 172.21.15.45:0/2015353584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c48108500 0x7f3c48101f10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000ef30 tx=0x7f3c4000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:29.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.625+0000 7f3c3ffff700 1 -- 172.21.15.45:0/2015353584 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400092e0 con 0x7f3c48108500 2024-06-06T04:33:29.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.625+0000 7f3c3ffff700 1 -- 172.21.15.45:0/2015353584 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c40004bb0 con 0x7f3c48108500 2024-06-06T04:33:29.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.625+0000 7f3c4ccdb700 1 -- 172.21.15.45:0/2015353584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c481a73a0 con 0x7f3c48108500 2024-06-06T04:33:29.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.625+0000 7f3c3ffff700 1 -- 172.21.15.45:0/2015353584 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40004020 con 0x7f3c48108500 2024-06-06T04:33:29.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.625+0000 7f3c4ccdb700 1 -- 172.21.15.45:0/2015353584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c481a78e0 con 0x7f3c48108500 2024-06-06T04:33:29.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.626+0000 7f3c4ccdb700 1 -- 172.21.15.45:0/2015353584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c4810a9e0 con 0x7f3c48108500 2024-06-06T04:33:29.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.627+0000 7f3c3ffff700 1 -- 172.21.15.45:0/2015353584 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3c40044d10 con 0x7f3c48108500 2024-06-06T04:33:29.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.628+0000 7f3c3ffff700 1 --2- 172.21.15.45:0/2015353584 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3c340a2ca0 0x7f3c340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:29.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.628+0000 7f3c4659c700 1 --2- 172.21.15.45:0/2015353584 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3c340a2ca0 0x7f3c340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:29.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.628+0000 7f3c3ffff700 1 -- 172.21.15.45:0/2015353584 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f3c400cf270 con 0x7f3c48108500 2024-06-06T04:33:29.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.628+0000 7f3c4659c700 1 --2- 172.21.15.45:0/2015353584 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3c340a2ca0 0x7f3c340a5160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800b2e0 tx=0x7f3c3801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:29.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.631+0000 7f3c3ffff700 1 -- 172.21.15.45:0/2015353584 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3c40022020 con 0x7f3c48108500 2024-06-06T04:33:29.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.934+0000 7f3c4ccdb700 1 -- 172.21.15.45:0/2015353584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c481029c0 con 0x7f3c48108500 2024-06-06T04:33:29.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.935+0000 7f3c3ffff700 1 -- 172.21.15.45:0/2015353584 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11862 (secure 0 0 0) 0x7f3c40010040 con 0x7f3c48108500 2024-06-06T04:33:29.939 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:33:29.939 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T04:27:24.462897+0000","modified":"2024-06-06T04:33:18.907139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24451,24463,24473]},"id":1}]} 2024-06-06T04:33:29.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.938+0000 7f3c4ccdb700 1 -- 172.21.15.45:0/2015353584 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3c340a2ca0 msgr2=0x7f3c340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:29.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.938+0000 7f3c4ccdb700 1 --2- 172.21.15.45:0/2015353584 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3c340a2ca0 0x7f3c340a5160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800b2e0 tx=0x7f3c3801a040 comp rx=0 tx=0).stop 2024-06-06T04:33:29.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.938+0000 7f3c4ccdb700 1 -- 172.21.15.45:0/2015353584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c48108500 msgr2=0x7f3c48101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.938+0000 7f3c4ccdb700 1 --2- 172.21.15.45:0/2015353584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c48108500 0x7f3c48101f10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000ef30 tx=0x7f3c4000c3a0 comp rx=0 tx=0).stop 2024-06-06T04:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.938+0000 7f3c4ccdb700 1 -- 172.21.15.45:0/2015353584 shutdown_connections 2024-06-06T04:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.938+0000 7f3c4ccdb700 1 --2- 172.21.15.45:0/2015353584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c48108500 0x7f3c48101f10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.938+0000 7f3c4ccdb700 1 --2- 172.21.15.45:0/2015353584 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3c340a2ca0 0x7f3c340a5160 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.938+0000 7f3c4ccdb700 1 --2- 172.21.15.45:0/2015353584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c4806abc0 0x7f3c48101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.938+0000 7f3c4ccdb700 1 --2- 172.21.15.45:0/2015353584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c4806b580 0x7f3c481019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.938+0000 7f3c4ccdb700 1 -- 172.21.15.45:0/2015353584 >> 172.21.15.45:0/2015353584 conn(0x7f3c48078340 msgr2=0x7f3c48100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.938+0000 7f3c4ccdb700 1 -- 172.21.15.45:0/2015353584 shutdown_connections 2024-06-06T04:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:29.938+0000 7f3c4ccdb700 1 -- 172.21.15.45:0/2015353584 wait complete. 2024-06-06T04:33:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 30 2024-06-06T04:33:30.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:30 smithi045 ceph-mon[32299]: pgmap v492: 129 pgs: 129 active+clean; 35 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 1.22k op/s 2024-06-06T04:33:30.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:30 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2015353584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:33:30.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:30 smithi154 ceph-mon[36428]: pgmap v492: 129 pgs: 129 active+clean; 35 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 1.22k op/s 2024-06-06T04:33:30.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:30 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2015353584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:33:30.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:30 smithi086 ceph-mon[36053]: pgmap v492: 129 pgs: 129 active+clean; 35 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 1.22k op/s 2024-06-06T04:33:30.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:30 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2015353584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:33:30.783 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-06T04:33:31.083 INFO:tasks.workunit.client.0.smithi045.stderr:Note: switching to '12973fc04fd10d4af086901e10ffa8e48866b735'. 2024-06-06T04:33:31.083 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-06-06T04:33:31.083 INFO:tasks.workunit.client.0.smithi045.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-06T04:33:31.083 INFO:tasks.workunit.client.0.smithi045.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-06T04:33:31.083 INFO:tasks.workunit.client.0.smithi045.stderr:state without impacting any branches by switching back to a branch. 2024-06-06T04:33:31.084 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-06-06T04:33:31.084 INFO:tasks.workunit.client.0.smithi045.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-06T04:33:31.084 INFO:tasks.workunit.client.0.smithi045.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-06T04:33:31.084 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-06-06T04:33:31.084 INFO:tasks.workunit.client.0.smithi045.stderr: git switch -c 2024-06-06T04:33:31.084 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-06-06T04:33:31.084 INFO:tasks.workunit.client.0.smithi045.stderr:Or undo this operation with: 2024-06-06T04:33:31.084 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-06-06T04:33:31.084 INFO:tasks.workunit.client.0.smithi045.stderr: git switch - 2024-06-06T04:33:31.084 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-06-06T04:33:31.084 INFO:tasks.workunit.client.0.smithi045.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-06T04:33:31.084 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-06-06T04:33:31.084 INFO:tasks.workunit.client.0.smithi045.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2024-06-06T04:33:31.087 INFO:tasks.workunit.client.0.smithi045.stderr:+ make -j4 2024-06-06T04:33:31.089 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:33:31.112 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize -cfi 2024-06-06T04:33:32.233 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: putting auxiliary files in '.'. 2024-06-06T04:33:32.234 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file './config.guess' 2024-06-06T04:33:32.244 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file './config.sub' 2024-06-06T04:33:32.255 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file './install-sh' 2024-06-06T04:33:32.268 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file './ltmain.sh' 2024-06-06T04:33:32.303 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-06-06T04:33:32.303 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-06-06T04:33:32.314 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-06-06T04:33:32.323 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-06-06T04:33:32.334 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-06-06T04:33:32.344 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-06-06T04:33:32.353 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-06-06T04:33:32.354 INFO:tasks.workunit.client.0.smithi045.stdout:cp include/install-sh . 2024-06-06T04:33:32.420 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:32 smithi045 ceph-mon[32299]: pgmap v493: 129 pgs: 129 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 1.25k op/s 2024-06-06T04:33:32.429 INFO:tasks.workunit.client.0.smithi045.stdout:aclocal -I m4 2024-06-06T04:33:32.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:32 smithi154 ceph-mon[36428]: pgmap v493: 129 pgs: 129 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 1.25k op/s 2024-06-06T04:33:32.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:32 smithi086 ceph-mon[36053]: pgmap v493: 129 pgs: 129 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 1.25k op/s 2024-06-06T04:33:32.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.660+0000 7f3361e5d640 1 -- 172.21.15.45:0/3982549023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f335c101a30 msgr2=0x7f335c101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.660+0000 7f3361e5d640 1 --2- 172.21.15.45:0/3982549023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f335c101a30 0x7f335c101e50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f334c009900 tx=0x7f334c02a410 comp rx=0 tx=0).stop 2024-06-06T04:33:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.660+0000 7f3361e5d640 1 -- 172.21.15.45:0/3982549023 shutdown_connections 2024-06-06T04:33:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.660+0000 7f3361e5d640 1 --2- 172.21.15.45:0/3982549023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f335c1035e0 0x7f335c109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.660+0000 7f3361e5d640 1 --2- 172.21.15.45:0/3982549023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f335c102c20 0x7f335c1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.660+0000 7f3361e5d640 1 --2- 172.21.15.45:0/3982549023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f335c101a30 0x7f335c101e50 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.660+0000 7f3361e5d640 1 -- 172.21.15.45:0/3982549023 >> 172.21.15.45:0/3982549023 conn(0x7f335c0fcfc0 msgr2=0x7f335c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.660+0000 7f3361e5d640 1 -- 172.21.15.45:0/3982549023 shutdown_connections 2024-06-06T04:33:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.660+0000 7f3361e5d640 1 -- 172.21.15.45:0/3982549023 wait complete. 2024-06-06T04:33:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.660+0000 7f3361e5d640 1 Processor -- start 2024-06-06T04:33:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.660+0000 7f3361e5d640 1 -- start start 2024-06-06T04:33:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f3361e5d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f335c101a30 0x7f335c075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f3361e5d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f335c102c20 0x7f335c073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f3361e5d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f335c1035e0 0x7f335c0742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f3361e5d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f335c10d550 con 0x7f335c1035e0 2024-06-06T04:33:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f3361e5d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f335c10d3d0 con 0x7f335c101a30 2024-06-06T04:33:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f3361e5d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f335c10d6d0 con 0x7f335c102c20 2024-06-06T04:33:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f335bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f335c1035e0 0x7f335c0742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f335b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f335c101a30 0x7f335c075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f335bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f335c1035e0 0x7f335c0742d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57824/0 (socket says 172.21.15.45:57824) 2024-06-06T04:33:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f3352ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f335c102c20 0x7f335c073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f335bfff640 1 -- 172.21.15.45:0/794077722 learned_addr learned my addr 172.21.15.45:0/794077722 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f335bfff640 1 -- 172.21.15.45:0/794077722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f335c102c20 msgr2=0x7f335c073d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f335bfff640 1 --2- 172.21.15.45:0/794077722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f335c102c20 0x7f335c073d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f335bfff640 1 -- 172.21.15.45:0/794077722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f335c101a30 msgr2=0x7f335c075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f335bfff640 1 --2- 172.21.15.45:0/794077722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f335c101a30 0x7f335c075750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f335bfff640 1 -- 172.21.15.45:0/794077722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335c074b60 con 0x7f335c1035e0 2024-06-06T04:33:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f335b7fe640 1 --2- 172.21.15.45:0/794077722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f335c101a30 0x7f335c075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f335bfff640 1 --2- 172.21.15.45:0/794077722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f335c1035e0 0x7f335c0742d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f335400efd0 tx=0x7f335400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.661+0000 7f33597fa640 1 -- 172.21.15.45:0/794077722 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354003b70 con 0x7f335c1035e0 2024-06-06T04:33:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.662+0000 7f33597fa640 1 -- 172.21.15.45:0/794077722 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3354003d10 con 0x7f335c1035e0 2024-06-06T04:33:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.662+0000 7f33597fa640 1 -- 172.21.15.45:0/794077722 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354010980 con 0x7f335c1035e0 2024-06-06T04:33:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.662+0000 7f3361e5d640 1 -- 172.21.15.45:0/794077722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f335c1a1d40 con 0x7f335c1035e0 2024-06-06T04:33:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.662+0000 7f3361e5d640 1 -- 172.21.15.45:0/794077722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f335c1a2180 con 0x7f335c1035e0 2024-06-06T04:33:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.664+0000 7f33597fa640 1 -- 172.21.15.45:0/794077722 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3354004020 con 0x7f335c1035e0 2024-06-06T04:33:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.664+0000 7f3364b6c740 1 Processor -- start 2024-06-06T04:33:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.664+0000 7f3364b6c740 1 -- start start 2024-06-06T04:33:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.664+0000 7f3364b6c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00e010 0x564a0f00e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.664+0000 7f3364b6c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00ea00 0x564a0f013500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.664+0000 7f3364b6c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f013b70 0x564a0f016000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.664+0000 7f3364b6c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564a0ef714f0 con 0x564a0f00ea00 2024-06-06T04:33:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.664+0000 7f3364b6c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564a0ef0f870 con 0x564a0f00e010 2024-06-06T04:33:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.664+0000 7f3364b6c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564a0ef6d4b0 con 0x564a0f013b70 2024-06-06T04:33:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.664+0000 7f335b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00ea00 0x564a0f013500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.664+0000 7f335b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00ea00 0x564a0f013500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57838/0 (socket says 172.21.15.45:57838) 2024-06-06T04:33:32.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.664+0000 7f335b7fe640 1 -- 172.21.15.45:0/332180992 learned_addr learned my addr 172.21.15.45:0/332180992 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.665+0000 7f3352ffd640 1 --2- 172.21.15.45:0/332180992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00e010 0x564a0f00e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.665+0000 7f3352ffd640 1 --2- 172.21.15.45:0/332180992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f013b70 0x564a0f016000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.665+0000 7f335b7fe640 1 -- 172.21.15.45:0/332180992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f013b70 msgr2=0x564a0f016000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.665+0000 7f335b7fe640 1 --2- 172.21.15.45:0/332180992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f013b70 0x564a0f016000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.665+0000 7f335b7fe640 1 -- 172.21.15.45:0/332180992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00e010 msgr2=0x564a0f00e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.665+0000 7f335b7fe640 1 --2- 172.21.15.45:0/332180992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00e010 0x564a0f00e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.665+0000 7f335b7fe640 1 -- 172.21.15.45:0/332180992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a0ef2b9b0 con 0x564a0f00ea00 2024-06-06T04:33:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.665+0000 7f3352ffd640 1 --2- 172.21.15.45:0/332180992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00e010 0x564a0f00e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.665+0000 7f335b7fe640 1 --2- 172.21.15.45:0/332180992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00ea00 0x564a0f013500 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f334c009900 tx=0x7f334c032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89567802f26a9249 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.665+0000 7f33597fa640 1 --2- 172.21.15.45:0/794077722 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f33480a2cb0 0x7f33480a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.665+0000 7f3351ffb640 1 -- 172.21.15.45:0/332180992 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c035070 con 0x564a0f00ea00 2024-06-06T04:33:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.665+0000 7f3351ffb640 1 -- 172.21.15.45:0/332180992 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f334c0035f0 con 0x564a0f00ea00 2024-06-06T04:33:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.665+0000 7f3351ffb640 1 -- 172.21.15.45:0/332180992 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c0044e0 con 0x564a0f00ea00 2024-06-06T04:33:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.665+0000 7f3352ffd640 1 --2- 172.21.15.45:0/794077722 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f33480a2cb0 0x7f33480a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.666+0000 7f33597fa640 1 -- 172.21.15.45:0/794077722 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f33540cf7c0 con 0x7f335c1035e0 2024-06-06T04:33:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.666+0000 7f3364b6c740 1 -- 172.21.15.45:0/332180992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00ea00 msgr2=0x564a0f013500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.666+0000 7f3352ffd640 1 --2- 172.21.15.45:0/794077722 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f33480a2cb0 0x7f33480a5170 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x564a0f00f260 tx=0x7f3344009940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.666+0000 7f3364b6c740 1 --2- 172.21.15.45:0/332180992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00ea00 0x564a0f013500 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f334c009900 tx=0x7f334c032eb0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.666+0000 7f3364b6c740 1 -- 172.21.15.45:0/332180992 shutdown_connections 2024-06-06T04:33:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.666+0000 7f3364b6c740 1 --2- 172.21.15.45:0/332180992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f013b70 0x564a0f016000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.666+0000 7f3364b6c740 1 --2- 172.21.15.45:0/332180992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00ea00 0x564a0f013500 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.666+0000 7f3364b6c740 1 --2- 172.21.15.45:0/332180992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00e010 0x564a0f00e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.666+0000 7f3364b6c740 1 -- 172.21.15.45:0/332180992 >> 172.21.15.45:0/332180992 conn(0x564a0f009680 msgr2=0x564a0f009a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.666+0000 7f3364b6c740 1 -- 172.21.15.45:0/332180992 shutdown_connections 2024-06-06T04:33:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.666+0000 7f3364b6c740 1 -- 172.21.15.45:0/332180992 wait complete. 2024-06-06T04:33:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.667+0000 7f3364b6c740 1 -- start start 2024-06-06T04:33:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.667+0000 7f3364b6c740 1 Processor -- start 2024-06-06T04:33:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.667+0000 7f3364b6c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00e010 0x564a0f0181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.667+0000 7f3364b6c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f00ea00 0x564a0f018730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.667+0000 7f3364b6c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f013b70 0x564a0f01f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.667+0000 7f3364b6c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564a0f016ac0 con 0x564a0f00e010 2024-06-06T04:33:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.667+0000 7f3364b6c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564a0eefaae0 con 0x564a0f013b70 2024-06-06T04:33:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.667+0000 7f3364b6c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564a0f016c40 con 0x564a0f00ea00 2024-06-06T04:33:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.667+0000 7f3364b6c740 10 client.0 _ll_register_callbacks cb 0x564a0ef339b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.667+0000 7f335b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00e010 0x564a0f0181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.667+0000 7f335b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00e010 0x564a0f0181f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57844/0 (socket says 172.21.15.45:57844) 2024-06-06T04:33:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.667+0000 7f335b7fe640 1 -- 172.21.15.45:0/2927319598 learned_addr learned my addr 172.21.15.45:0/2927319598 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.667+0000 7f335bfff640 1 --2- 172.21.15.45:0/2927319598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f013b70 0x564a0f01f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.667+0000 7f3352ffd640 1 --2- 172.21.15.45:0/2927319598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f00ea00 0x564a0f018730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.668+0000 7f335b7fe640 1 -- 172.21.15.45:0/2927319598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f00ea00 msgr2=0x564a0f018730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.668+0000 7f335b7fe640 1 --2- 172.21.15.45:0/2927319598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f00ea00 0x564a0f018730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.668+0000 7f335b7fe640 1 -- 172.21.15.45:0/2927319598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f013b70 msgr2=0x564a0f01f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.668+0000 7f335b7fe640 1 --2- 172.21.15.45:0/2927319598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f013b70 0x564a0f01f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.668+0000 7f335b7fe640 1 -- 172.21.15.45:0/2927319598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a0ec85210 con 0x564a0f00e010 2024-06-06T04:33:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.668+0000 7f335bfff640 1 --2- 172.21.15.45:0/2927319598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f013b70 0x564a0f01f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.668+0000 7f335b7fe640 1 --2- 172.21.15.45:0/2927319598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00e010 0x564a0f0181f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f334c0058c0 tx=0x7f334c031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fc9320724c7cff server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.668+0000 7f333affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:33:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.668+0000 7f333affd640 1 -- 172.21.15.45:0/2927319598 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c035050 con 0x564a0f00e010 2024-06-06T04:33:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.668+0000 7f333affd640 1 -- 172.21.15.45:0/2927319598 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f334c0039b0 con 0x564a0f00e010 2024-06-06T04:33:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.668+0000 7f3364b6c740 1 -- 172.21.15.45:0/2927319598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f334c0035f0 con 0x564a0f00e010 2024-06-06T04:33:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.668+0000 7f333affd640 1 -- 172.21.15.45:0/2927319598 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f334c003bb0 con 0x564a0f00e010 2024-06-06T04:33:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.668+0000 7f3364b6c740 10 client.15711 fetch_fsmap learned FSMap version 30 2024-06-06T04:33:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.668+0000 7f3364b6c740 1 -- 172.21.15.45:0/2927319598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a0ed51620 con 0x564a0f00e010 2024-06-06T04:33:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.668+0000 7f333affd640 1 -- 172.21.15.45:0/2927319598 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f334c0035f0 con 0x564a0f00e010 2024-06-06T04:33:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.669+0000 7f333affd640 1 -- 172.21.15.45:0/2927319598 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f334c0459d0 con 0x564a0f00e010 2024-06-06T04:33:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.669+0000 7f3364b6c740 10 client.15711 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:33:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.669+0000 7f3364b6c740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:33:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.669+0000 7f3364b6c740 20 client.15711 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.669+0000 7f3364b6c740 1 --2- 172.21.15.45:0/2927319598 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x564a0f023c60 0x564a0f026090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.669+0000 7f3364b6c740 4 client.15711 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:33:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.669+0000 7f3364b6c740 1 -- 172.21.15.45:0/2927319598 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a0ef84f10 con 0x564a0f023c60 2024-06-06T04:33:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.669+0000 7f3352ffd640 1 --2- 172.21.15.45:0/2927319598 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x564a0f023c60 0x564a0f026090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.669+0000 7f333affd640 10 client.15711 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.669+0000 7f3352ffd640 1 --2- 172.21.15.45:0/2927319598 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x564a0f023c60 0x564a0f026090 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4d297378843e336 server_cookie=b300cd10a72d7d12 in_seq=0 out_seq=0 2024-06-06T04:33:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.671+0000 7f333affd640 1 -- 172.21.15.45:0/2927319598 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564a0ef84f10 con 0x564a0f023c60 2024-06-06T04:33:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.671+0000 7f333affd640 10 client.15711 handle_command_reply: tid=0 2024-06-06T04:33:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.671+0000 7f3364b6c740 1 client.15711 shutdown 2024-06-06T04:33:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.671+0000 7f3364b6c740 1 -- 172.21.15.45:0/2927319598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00e010 msgr2=0x564a0f0181f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.671+0000 7f3364b6c740 1 --2- 172.21.15.45:0/2927319598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00e010 0x564a0f0181f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f334c0058c0 tx=0x7f334c031740 comp rx=0 tx=0).stop 2024-06-06T04:33:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.671+0000 7f3364b6c740 1 -- 172.21.15.45:0/2927319598 shutdown_connections 2024-06-06T04:33:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.671+0000 7f3364b6c740 1 --2- 172.21.15.45:0/2927319598 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x564a0f023c60 0x564a0f026090 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.671+0000 7f3364b6c740 1 --2- 172.21.15.45:0/2927319598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f013b70 0x564a0f01f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.671+0000 7f3364b6c740 1 --2- 172.21.15.45:0/2927319598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f00ea00 0x564a0f018730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.671+0000 7f333affd640 0 client.15711 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.671+0000 7f3364b6c740 1 --2- 172.21.15.45:0/2927319598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00e010 0x564a0f0181f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.671+0000 7f3364b6c740 1 -- 172.21.15.45:0/2927319598 >> 172.21.15.45:0/2927319598 conn(0x564a0f009680 msgr2=0x564a0f0087e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.671+0000 7f3364b6c740 1 -- 172.21.15.45:0/2927319598 shutdown_connections 2024-06-06T04:33:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.671+0000 7f3364b6c740 1 -- 172.21.15.45:0/2927319598 wait complete. 2024-06-06T04:33:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.671+0000 7f3364b6c740 20 client.15711 trim_cache size 0 max 16384 2024-06-06T04:33:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.686+0000 7f3364b6c740 1 Processor -- start 2024-06-06T04:33:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.686+0000 7f3364b6c740 1 -- start start 2024-06-06T04:33:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f3364b6c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00e010 0x564a0f00fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f3364b6c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f00ea00 0x564a0f010160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f3364b6c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f013b70 0x564a0f01f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f3364b6c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f335c1aefe0 con 0x564a0f013b70 2024-06-06T04:33:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f3364b6c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f335c1ac880 con 0x564a0f00e010 2024-06-06T04:33:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f3364b6c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f335c1aa6d0 con 0x564a0f00ea00 2024-06-06T04:33:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f335bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f013b70 0x564a0f01f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f335bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f013b70 0x564a0f01f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57860/0 (socket says 172.21.15.45:57860) 2024-06-06T04:33:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f335bfff640 1 -- 172.21.15.45:0/1557758636 learned_addr learned my addr 172.21.15.45:0/1557758636 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f335b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00e010 0x564a0f00fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f3352ffd640 1 --2- 172.21.15.45:0/1557758636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f00ea00 0x564a0f010160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f335bfff640 1 -- 172.21.15.45:0/1557758636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f00ea00 msgr2=0x564a0f010160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f335bfff640 1 --2- 172.21.15.45:0/1557758636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f00ea00 0x564a0f010160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f335bfff640 1 -- 172.21.15.45:0/1557758636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00e010 msgr2=0x564a0f00fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f335bfff640 1 --2- 172.21.15.45:0/1557758636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00e010 0x564a0f00fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f335bfff640 1 -- 172.21.15.45:0/1557758636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a0f0094b0 con 0x564a0f013b70 2024-06-06T04:33:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f335b7fe640 1 --2- 172.21.15.45:0/1557758636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00e010 0x564a0f00fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f335bfff640 1 --2- 172.21.15.45:0/1557758636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f013b70 0x564a0f01f730 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f33540cd270 tx=0x7f335409b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a875bf4172ea876c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f33517fa640 1 -- 172.21.15.45:0/1557758636 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33540b9070 con 0x564a0f013b70 2024-06-06T04:33:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f33517fa640 1 -- 172.21.15.45:0/1557758636 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33540b1030 con 0x564a0f013b70 2024-06-06T04:33:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.687+0000 7f33517fa640 1 -- 172.21.15.45:0/1557758636 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33540b4e60 con 0x564a0f013b70 2024-06-06T04:33:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.688+0000 7f3364b6c740 1 -- 172.21.15.45:0/1557758636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f013b70 msgr2=0x564a0f01f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.688+0000 7f3364b6c740 1 --2- 172.21.15.45:0/1557758636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f013b70 0x564a0f01f730 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f33540cd270 tx=0x7f335409b5c0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.688+0000 7f3364b6c740 1 -- 172.21.15.45:0/1557758636 shutdown_connections 2024-06-06T04:33:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.688+0000 7f3364b6c740 1 --2- 172.21.15.45:0/1557758636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f013b70 0x564a0f01f730 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.688+0000 7f3364b6c740 1 --2- 172.21.15.45:0/1557758636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f00ea00 0x564a0f010160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.688+0000 7f3364b6c740 1 --2- 172.21.15.45:0/1557758636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00e010 0x564a0f00fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.688+0000 7f3364b6c740 1 -- 172.21.15.45:0/1557758636 >> 172.21.15.45:0/1557758636 conn(0x564a0f009680 msgr2=0x564a0f016fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.688+0000 7f3364b6c740 1 -- 172.21.15.45:0/1557758636 shutdown_connections 2024-06-06T04:33:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.688+0000 7f3364b6c740 1 -- 172.21.15.45:0/1557758636 wait complete. 2024-06-06T04:33:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.688+0000 7f3364b6c740 1 -- start start 2024-06-06T04:33:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f3364b6c740 1 Processor -- start 2024-06-06T04:33:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f3364b6c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00e010 0x564a0f01ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f3364b6c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00ea00 0x564a0f01cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f3364b6c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f013b70 0x564a0f01df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f3364b6c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f335c1aad40 con 0x564a0f00e010 2024-06-06T04:33:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f3364b6c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f335c1ac1b0 con 0x564a0f00ea00 2024-06-06T04:33:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f3364b6c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f335c1ab410 con 0x564a0f013b70 2024-06-06T04:33:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f3364b6c740 10 client.0 _ll_register_callbacks cb 0x7f33440097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f335b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00e010 0x564a0f01ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f335b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00e010 0x564a0f01ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57868/0 (socket says 172.21.15.45:57868) 2024-06-06T04:33:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f3352ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00ea00 0x564a0f01cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f335b7fe640 1 -- 172.21.15.45:0/702867608 learned_addr learned my addr 172.21.15.45:0/702867608 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f335bfff640 1 --2- 172.21.15.45:0/702867608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f013b70 0x564a0f01df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f335b7fe640 1 -- 172.21.15.45:0/702867608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f013b70 msgr2=0x564a0f01df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f335b7fe640 1 --2- 172.21.15.45:0/702867608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f013b70 0x564a0f01df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f335b7fe640 1 -- 172.21.15.45:0/702867608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00ea00 msgr2=0x564a0f01cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f335b7fe640 1 --2- 172.21.15.45:0/702867608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00ea00 0x564a0f01cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f335b7fe640 1 -- 172.21.15.45:0/702867608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a0f016d80 con 0x564a0f00e010 2024-06-06T04:33:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f335b7fe640 1 --2- 172.21.15.45:0/702867608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00e010 0x564a0f01ca40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f334c033000 tx=0x7f334c03c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220ed0c46d1d4ae3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f333affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:33:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.689+0000 7f333affd640 1 -- 172.21.15.45:0/702867608 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c035040 con 0x564a0f00e010 2024-06-06T04:33:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.690+0000 7f333affd640 1 -- 172.21.15.45:0/702867608 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f334c0039b0 con 0x564a0f00e010 2024-06-06T04:33:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.690+0000 7f3364b6c740 1 -- 172.21.15.45:0/702867608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33540b1030 con 0x564a0f00e010 2024-06-06T04:33:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.690+0000 7f333affd640 1 -- 172.21.15.45:0/702867608 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f334c0035f0 con 0x564a0f00e010 2024-06-06T04:33:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.690+0000 7f3364b6c740 10 client.15717 fetch_fsmap learned FSMap version 30 2024-06-06T04:33:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.690+0000 7f3364b6c740 1 -- 172.21.15.45:0/702867608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a0f01f730 con 0x564a0f00e010 2024-06-06T04:33:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.690+0000 7f333affd640 1 -- 172.21.15.45:0/702867608 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33540b1030 con 0x564a0f00e010 2024-06-06T04:33:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.690+0000 7f333affd640 1 -- 172.21.15.45:0/702867608 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f334c0459d0 con 0x564a0f00e010 2024-06-06T04:33:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.690+0000 7f3364b6c740 10 client.15717 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:33:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.690+0000 7f3364b6c740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:33:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.690+0000 7f3364b6c740 20 client.15717 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.690+0000 7f3364b6c740 1 --2- 172.21.15.45:0/702867608 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x564a0f08ec70 0x564a0f091090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.690+0000 7f3364b6c740 4 client.15717 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T04:33:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.690+0000 7f3364b6c740 1 -- 172.21.15.45:0/702867608 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a0ef7ec20 con 0x564a0f08ec70 2024-06-06T04:33:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.690+0000 7f3352ffd640 1 --2- 172.21.15.45:0/702867608 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x564a0f08ec70 0x564a0f091090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.691+0000 7f333affd640 10 client.15717 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:32.691+0000 7f3352ffd640 1 --2- 172.21.15.45:0/702867608 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x564a0f08ec70 0x564a0f091090 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ecaa8f77b0ddc0e server_cookie=1660930de32b04a7 in_seq=0 out_seq=0 2024-06-06T04:33:33.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.027+0000 7f8f93921700 1 -- 172.21.15.45:0/2588642463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c1027a0 msgr2=0x7f8f8c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:33.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.027+0000 7f8f93921700 1 --2- 172.21.15.45:0/2588642463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c1027a0 0x7f8f8c102b80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800b0d0 tx=0x7f8f8800b3e0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.028+0000 7f8f93921700 1 -- 172.21.15.45:0/2588642463 shutdown_connections 2024-06-06T04:33:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.028+0000 7f8f93921700 1 --2- 172.21.15.45:0/2588642463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f8c107870 0x7f8f8c10f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.028+0000 7f8f93921700 1 --2- 172.21.15.45:0/2588642463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c1027a0 0x7f8f8c102b80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.028+0000 7f8f93921700 1 --2- 172.21.15.45:0/2588642463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f8c103150 0x7f8f8c107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.028+0000 7f8f93921700 1 -- 172.21.15.45:0/2588642463 >> 172.21.15.45:0/2588642463 conn(0x7f8f8c079570 msgr2=0x7f8f8c079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.028+0000 7f8f93921700 1 -- 172.21.15.45:0/2588642463 shutdown_connections 2024-06-06T04:33:33.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.028+0000 7f8f93921700 1 -- 172.21.15.45:0/2588642463 wait complete. 2024-06-06T04:33:33.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.028+0000 7f8f93921700 1 Processor -- start 2024-06-06T04:33:33.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.028+0000 7f8f93921700 1 -- start start 2024-06-06T04:33:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.029+0000 7f8f93921700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c1027a0 0x7f8f8c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.029+0000 7f8f93921700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f8c103150 0x7f8f8c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.029+0000 7f8f93921700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f8c107870 0x7f8f8c10e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.029+0000 7f8f93921700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c112640 con 0x7f8f8c1027a0 2024-06-06T04:33:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.029+0000 7f8f93921700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1124c0 con 0x7f8f8c103150 2024-06-06T04:33:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.029+0000 7f8f93921700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1127c0 con 0x7f8f8c107870 2024-06-06T04:33:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.029+0000 7f8f916bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c1027a0 0x7f8f8c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.029+0000 7f8f916bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c1027a0 0x7f8f8c19f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57880/0 (socket says 172.21.15.45:57880) 2024-06-06T04:33:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.029+0000 7f8f916bd700 1 -- 172.21.15.45:0/2224886297 learned_addr learned my addr 172.21.15.45:0/2224886297 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.029+0000 7f8f91ebe700 1 --2- 172.21.15.45:0/2224886297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f8c107870 0x7f8f8c10e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.029+0000 7f8f90ebc700 1 --2- 172.21.15.45:0/2224886297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f8c103150 0x7f8f8c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.029+0000 7f8f916bd700 1 -- 172.21.15.45:0/2224886297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f8c107870 msgr2=0x7f8f8c10e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.029+0000 7f8f916bd700 1 --2- 172.21.15.45:0/2224886297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f8c107870 0x7f8f8c10e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.029+0000 7f8f916bd700 1 -- 172.21.15.45:0/2224886297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f8c103150 msgr2=0x7f8f8c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:33.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.029+0000 7f8f916bd700 1 --2- 172.21.15.45:0/2224886297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f8c103150 0x7f8f8c19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.029+0000 7f8f916bd700 1 -- 172.21.15.45:0/2224886297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8c10ec80 con 0x7f8f8c1027a0 2024-06-06T04:33:33.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.030+0000 7f8f916bd700 1 --2- 172.21.15.45:0/2224886297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c1027a0 0x7f8f8c19f740 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8f88007a30 tx=0x7f8f88007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:33.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.030+0000 7f8f827fc700 1 -- 172.21.15.45:0/2224886297 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8801f070 con 0x7f8f8c1027a0 2024-06-06T04:33:33.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.030+0000 7f8f827fc700 1 -- 172.21.15.45:0/2224886297 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f88004c60 con 0x7f8f8c1027a0 2024-06-06T04:33:33.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.030+0000 7f8f93921700 1 -- 172.21.15.45:0/2224886297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8c10ef10 con 0x7f8f8c1027a0 2024-06-06T04:33:33.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.030+0000 7f8f827fc700 1 -- 172.21.15.45:0/2224886297 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8801b690 con 0x7f8f8c1027a0 2024-06-06T04:33:33.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.030+0000 7f8f93921700 1 -- 172.21.15.45:0/2224886297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8c19b2a0 con 0x7f8f8c1027a0 2024-06-06T04:33:33.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.034+0000 7f8f9e1a4b80 1 Processor -- start 2024-06-06T04:33:33.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.034+0000 7f8f827fc700 1 -- 172.21.15.45:0/2224886297 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8f88022070 con 0x7f8f8c1027a0 2024-06-06T04:33:33.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.034+0000 7f8f9e1a4b80 1 -- start start 2024-06-06T04:33:33.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.034+0000 7f8f9e1a4b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e785810f60 0x55e785811340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.035+0000 7f8f9e1a4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785811880 0x55e78580b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.035+0000 7f8f9e1a4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e78580b7a0 0x55e78580bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.035+0000 7f8f9e1a4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e7856fdf60 con 0x55e785811880 2024-06-06T04:33:33.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.035+0000 7f8f9e1a4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e78575c850 con 0x55e78580b7a0 2024-06-06T04:33:33.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.035+0000 7f8f916bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785811880 0x55e78580b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.035+0000 7f8f91ebe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e785810f60 0x55e785811340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.035+0000 7f8f9e1a4b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e7856c4460 con 0x55e785810f60 2024-06-06T04:33:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.035+0000 7f8f91ebe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e785810f60 0x55e785811340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:42542/0 (socket says 172.21.15.45:42542) 2024-06-06T04:33:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.035+0000 7f8f91ebe700 1 -- 172.21.15.45:0/1564161247 learned_addr learned my addr 172.21.15.45:0/1564161247 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.035+0000 7f8f90ebc700 1 --2- 172.21.15.45:0/1564161247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e78580b7a0 0x55e78580bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.035+0000 7f8f90ebc700 1 -- 172.21.15.45:0/1564161247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e785810f60 msgr2=0x55e785811340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.035+0000 7f8f90ebc700 1 --2- 172.21.15.45:0/1564161247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e785810f60 0x55e785811340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.035+0000 7f8f90ebc700 1 -- 172.21.15.45:0/1564161247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785811880 msgr2=0x55e78580b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.035+0000 7f8f90ebc700 1 --2- 172.21.15.45:0/1564161247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785811880 0x55e78580b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.035+0000 7f8f90ebc700 1 -- 172.21.15.45:0/1564161247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7856ab8d0 con 0x55e78580b7a0 2024-06-06T04:33:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.035+0000 7f8f827fc700 1 --2- 172.21.15.45:0/2224886297 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8f780a2ca0 0x7f8f780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.036+0000 7f8f916bd700 1 --2- 172.21.15.45:0/1564161247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785811880 0x55e78580b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:33:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.036+0000 7f8f90ebc700 1 --2- 172.21.15.45:0/1564161247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e78580b7a0 0x55e78580bbc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c10f640 tx=0x7f8f7c009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dacda8ff7cf10948 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.036+0000 7f8f827fc700 1 -- 172.21.15.45:0/2224886297 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f8f8800f020 con 0x7f8f8c1027a0 2024-06-06T04:33:33.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.036+0000 7f8f91ebe700 1 --2- 172.21.15.45:0/2224886297 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8f780a2ca0 0x7f8f780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.036+0000 7f8f777fe700 1 -- 172.21.15.45:0/1564161247 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c011c80 con 0x55e78580b7a0 2024-06-06T04:33:33.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.036+0000 7f8f777fe700 1 -- 172.21.15.45:0/1564161247 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f7c011e20 con 0x55e78580b7a0 2024-06-06T04:33:33.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.036+0000 7f8f777fe700 1 -- 172.21.15.45:0/1564161247 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c005280 con 0x55e78580b7a0 2024-06-06T04:33:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.037+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1564161247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e78580b7a0 msgr2=0x55e78580bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.037+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/1564161247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e78580b7a0 0x55e78580bbc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c10f640 tx=0x7f8f7c009f60 comp rx=0 tx=0).stop 2024-06-06T04:33:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.037+0000 7f8f91ebe700 1 --2- 172.21.15.45:0/2224886297 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8f780a2ca0 0x7f8f780a5160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8f84009ee0 tx=0x7f8f84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.037+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1564161247 shutdown_connections 2024-06-06T04:33:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.037+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/1564161247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e78580b7a0 0x55e78580bbc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.037+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/1564161247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e785810f60 0x55e785811340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.037+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/1564161247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785811880 0x55e78580b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.037+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1564161247 >> 172.21.15.45:0/1564161247 conn(0x55e785738080 msgr2=0x55e785806220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.037+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1564161247 shutdown_connections 2024-06-06T04:33:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.037+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1564161247 wait complete. 2024-06-06T04:33:33.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.037+0000 7f8f9e1a4b80 1 -- start start 2024-06-06T04:33:33.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.037+0000 7f8f9e1a4b80 1 Processor -- start 2024-06-06T04:33:33.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f9e1a4b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e78580b7a0 0x55e7858175f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f9e1a4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785810f60 0x55e7858148f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f9e1a4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e785811880 0x55e785814e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f9e1a4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e7856fcfc0 con 0x55e785810f60 2024-06-06T04:33:33.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f9e1a4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e785758810 con 0x55e785811880 2024-06-06T04:33:33.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f9e1a4b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e785818260 con 0x55e78580b7a0 2024-06-06T04:33:33.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f9e1a4b80 10 client.0 _ll_register_callbacks cb 0x55e7856f25b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:33.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f90ebc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785810f60 0x55e7858148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f90ebc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785810f60 0x55e7858148f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57892/0 (socket says 172.21.15.45:57892) 2024-06-06T04:33:33.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f90ebc700 1 -- 172.21.15.45:0/486176002 learned_addr learned my addr 172.21.15.45:0/486176002 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f916bd700 1 --2- 172.21.15.45:0/486176002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e78580b7a0 0x55e7858175f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f91ebe700 1 --2- 172.21.15.45:0/486176002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e785811880 0x55e785814e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f90ebc700 1 -- 172.21.15.45:0/486176002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e78580b7a0 msgr2=0x55e7858175f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f90ebc700 1 --2- 172.21.15.45:0/486176002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e78580b7a0 0x55e7858175f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f90ebc700 1 -- 172.21.15.45:0/486176002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e785811880 msgr2=0x55e785814e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f90ebc700 1 --2- 172.21.15.45:0/486176002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e785811880 0x55e785814e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f90ebc700 1 -- 172.21.15.45:0/486176002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e785722750 con 0x55e785810f60 2024-06-06T04:33:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.038+0000 7f8f91ebe700 1 --2- 172.21.15.45:0/486176002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e785811880 0x55e785814e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:33:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.039+0000 7f8f90ebc700 1 --2- 172.21.15.45:0/486176002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785810f60 0x55e7858148f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c004da0 tx=0x7f8f7c011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69d9517ecdb2972 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.039+0000 7f8f74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:33:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.039+0000 7f8f74ff9700 1 -- 172.21.15.45:0/486176002 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c026070 con 0x55e785810f60 2024-06-06T04:33:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.039+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/486176002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f7c011e20 con 0x55e785810f60 2024-06-06T04:33:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.039+0000 7f8f74ff9700 1 -- 172.21.15.45:0/486176002 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f8f7c005010 con 0x55e785810f60 2024-06-06T04:33:33.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.039+0000 7f8f74ff9700 1 -- 172.21.15.45:0/486176002 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8f7c032050 con 0x55e785810f60 2024-06-06T04:33:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.039+0000 7f8f9e1a4b80 10 client.15726 fetch_fsmap learned FSMap version 30 2024-06-06T04:33:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.039+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/486176002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e785822950 con 0x55e785810f60 2024-06-06T04:33:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.039+0000 7f8f74ff9700 1 -- 172.21.15.45:0/486176002 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f7c011e20 con 0x55e785810f60 2024-06-06T04:33:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.040+0000 7f8f74ff9700 1 -- 172.21.15.45:0/486176002 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f8f7c018070 con 0x55e785810f60 2024-06-06T04:33:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.040+0000 7f8f9e1a4b80 10 client.15726 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:33:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.040+0000 7f8f9e1a4b80 10 client.15726 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-06T04:33:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.040+0000 7f8f9e1a4b80 20 client.15726 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:33.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.040+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/486176002 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55e785824e80 0x55e785827270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.040+0000 7f8f9e1a4b80 4 client.15726 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:33:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.040+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/486176002 --> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e785769d50 con 0x55e785824e80 2024-06-06T04:33:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.040+0000 7f8f916bd700 1 --2- 172.21.15.45:0/486176002 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55e785824e80 0x55e785827270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.040+0000 7f8f74ff9700 10 client.15726 ms_handle_connect on v2:172.21.15.154:6836/1140071744 2024-06-06T04:33:33.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.040+0000 7f8f916bd700 1 --2- 172.21.15.45:0/486176002 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55e785824e80 0x55e785827270 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=412e597d8f32a1f server_cookie=78c262c3e0bc9cc5 in_seq=0 out_seq=0 2024-06-06T04:33:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.041+0000 7f8f74ff9700 1 -- 172.21.15.45:0/486176002 <== mds.2 v2:172.21.15.154:6836/1140071744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e785769d50 con 0x55e785824e80 2024-06-06T04:33:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.041+0000 7f8f74ff9700 10 client.15726 handle_command_reply: tid=0 2024-06-06T04:33:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.041+0000 7f8f9e1a4b80 1 client.15726 shutdown 2024-06-06T04:33:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.041+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/486176002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785810f60 msgr2=0x55e7858148f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.041+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/486176002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785810f60 0x55e7858148f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c004da0 tx=0x7f8f7c011b30 comp rx=0 tx=0).stop 2024-06-06T04:33:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.041+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/486176002 shutdown_connections 2024-06-06T04:33:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.041+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/486176002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e785811880 0x55e785814e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.041+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/486176002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e78580b7a0 0x55e7858175f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.041+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/486176002 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55e785824e80 0x55e785827270 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.041+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/486176002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785810f60 0x55e7858148f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.041+0000 7f8f74ff9700 0 client.15726 ms_handle_reset on v2:172.21.15.154:6836/1140071744 2024-06-06T04:33:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.041+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/486176002 >> 172.21.15.45:0/486176002 conn(0x55e785738080 msgr2=0x55e785805700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.042+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/486176002 shutdown_connections 2024-06-06T04:33:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.042+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/486176002 wait complete. 2024-06-06T04:33:33.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.042+0000 7f8f9e1a4b80 20 client.15726 trim_cache size 0 max 16384 2024-06-06T04:33:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.057+0000 7f8f9e1a4b80 1 Processor -- start 2024-06-06T04:33:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.057+0000 7f8f9e1a4b80 1 -- start start 2024-06-06T04:33:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.058+0000 7f8f9e1a4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78580b7a0 0x55e78580b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.058+0000 7f8f9e1a4b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e785810f60 0x55e785805e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.058+0000 7f8f916bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78580b7a0 0x55e78580b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.058+0000 7f8f916bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78580b7a0 0x55e78580b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57904/0 (socket says 172.21.15.45:57904) 2024-06-06T04:33:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.058+0000 7f8f916bd700 1 -- 172.21.15.45:0/1791549443 learned_addr learned my addr 172.21.15.45:0/1791549443 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.058+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/1791549443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e785811880 0x55e7858063c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.058+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1791549443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1bf880 con 0x55e78580b7a0 2024-06-06T04:33:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.058+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1791549443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1bd0e0 con 0x55e785811880 2024-06-06T04:33:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.058+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1791549443 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1baf30 con 0x55e785810f60 2024-06-06T04:33:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.058+0000 7f8f90ebc700 1 --2- 172.21.15.45:0/1791549443 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e785810f60 0x55e785805e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.058+0000 7f8f916bd700 1 -- 172.21.15.45:0/1791549443 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e785810f60 msgr2=0x55e785805e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.058+0000 7f8f916bd700 1 --2- 172.21.15.45:0/1791549443 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e785810f60 0x55e785805e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.058+0000 7f8f916bd700 1 -- 172.21.15.45:0/1791549443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e785811880 msgr2=0x55e7858063c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.058+0000 7f8f916bd700 1 --2- 172.21.15.45:0/1791549443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e785811880 0x55e7858063c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.058+0000 7f8f916bd700 1 -- 172.21.15.45:0/1791549443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e785802810 con 0x55e78580b7a0 2024-06-06T04:33:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.059+0000 7f8f916bd700 1 --2- 172.21.15.45:0/1791549443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78580b7a0 0x55e78580b260 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8f88098d40 tx=0x7f8f880cd9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bb28cc4bfd8d5b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.059+0000 7f8f76ffd700 1 -- 172.21.15.45:0/1791549443 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88005a90 con 0x55e78580b7a0 2024-06-06T04:33:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.059+0000 7f8f76ffd700 1 -- 172.21.15.45:0/1791549443 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f8809cfe0 con 0x55e78580b7a0 2024-06-06T04:33:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.059+0000 7f8f76ffd700 1 -- 172.21.15.45:0/1791549443 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f880074e0 con 0x55e78580b7a0 2024-06-06T04:33:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.059+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1791549443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78580b7a0 msgr2=0x55e78580b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.059+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/1791549443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78580b7a0 0x55e78580b260 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8f88098d40 tx=0x7f8f880cd9c0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.059+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1791549443 shutdown_connections 2024-06-06T04:33:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.059+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/1791549443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e785811880 0x55e7858063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.059+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/1791549443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78580b7a0 0x55e78580b260 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.059+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/1791549443 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e785810f60 0x55e785805e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.059+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1791549443 >> 172.21.15.45:0/1791549443 conn(0x55e785738080 msgr2=0x55e7858169b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.060+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1791549443 shutdown_connections 2024-06-06T04:33:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.060+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1791549443 wait complete. 2024-06-06T04:33:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.060+0000 7f8f9e1a4b80 1 -- start start 2024-06-06T04:33:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.060+0000 7f8f9e1a4b80 1 Processor -- start 2024-06-06T04:33:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.060+0000 7f8f9e1a4b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e78580b7a0 0x55e78581d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.060+0000 7f8f9e1a4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785810f60 0x55e78581da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.060+0000 7f8f9e1a4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e785811880 0x55e78582ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f9e1a4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1bb5a0 con 0x55e785810f60 2024-06-06T04:33:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f9e1a4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1bca10 con 0x55e785811880 2024-06-06T04:33:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f9e1a4b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1bbc70 con 0x55e78580b7a0 2024-06-06T04:33:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f9e1a4b80 10 client.0 _ll_register_callbacks cb 0x55e78580f530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f90ebc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785810f60 0x55e78581da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f90ebc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785810f60 0x55e78581da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57914/0 (socket says 172.21.15.45:57914) 2024-06-06T04:33:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f90ebc700 1 -- 172.21.15.45:0/1802879857 learned_addr learned my addr 172.21.15.45:0/1802879857 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f916bd700 1 --2- 172.21.15.45:0/1802879857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e78580b7a0 0x55e78581d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f91ebe700 1 --2- 172.21.15.45:0/1802879857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e785811880 0x55e78582ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f90ebc700 1 -- 172.21.15.45:0/1802879857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e78580b7a0 msgr2=0x55e78581d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f90ebc700 1 --2- 172.21.15.45:0/1802879857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e78580b7a0 0x55e78581d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f90ebc700 1 -- 172.21.15.45:0/1802879857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e785811880 msgr2=0x55e78582ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f90ebc700 1 --2- 172.21.15.45:0/1802879857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e785811880 0x55e78582ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f90ebc700 1 -- 172.21.15.45:0/1802879857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e78582bfb0 con 0x55e785810f60 2024-06-06T04:33:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f90ebc700 1 --2- 172.21.15.45:0/1802879857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785810f60 0x55e78581da60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c004dd0 tx=0x7f8f7c004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7089b87b813841e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:33:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f74ff9700 1 -- 172.21.15.45:0/1802879857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c02f070 con 0x55e785810f60 2024-06-06T04:33:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1802879857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f8809cfe0 con 0x55e785810f60 2024-06-06T04:33:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.061+0000 7f8f74ff9700 1 -- 172.21.15.45:0/1802879857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f8f7c022610 con 0x55e785810f60 2024-06-06T04:33:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.062+0000 7f8f74ff9700 1 -- 172.21.15.45:0/1802879857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8f7c026040 con 0x55e785810f60 2024-06-06T04:33:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.062+0000 7f8f9e1a4b80 10 client.15732 fetch_fsmap learned FSMap version 30 2024-06-06T04:33:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.062+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1802879857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e78588f0f0 con 0x55e785810f60 2024-06-06T04:33:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.062+0000 7f8f74ff9700 1 -- 172.21.15.45:0/1802879857 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f8809cfe0 con 0x55e785810f60 2024-06-06T04:33:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.062+0000 7f8f74ff9700 1 -- 172.21.15.45:0/1802879857 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f8f7c018050 con 0x55e785810f60 2024-06-06T04:33:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.062+0000 7f8f9e1a4b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:33:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.062+0000 7f8f9e1a4b80 10 client.15732 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-06T04:33:33.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.062+0000 7f8f9e1a4b80 20 client.15732 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.062+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/1802879857 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55e78588f820 0x55e785891c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.062+0000 7f8f9e1a4b80 4 client.15732 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T04:33:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.062+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1802879857 --> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7857642d0 con 0x55e78588f820 2024-06-06T04:33:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.062+0000 7f8f916bd700 1 --2- 172.21.15.45:0/1802879857 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55e78588f820 0x55e785891c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f916bd700 1 --2- 172.21.15.45:0/1802879857 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55e78588f820 0x55e785891c40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff5bc64647013bf3 server_cookie=1e47ab526d631e0f in_seq=0 out_seq=0 2024-06-06T04:33:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f74ff9700 10 client.15732 ms_handle_connect on v2:172.21.15.154:6836/1140071744 2024-06-06T04:33:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f74ff9700 1 -- 172.21.15.45:0/1802879857 <== mds.2 v2:172.21.15.154:6836/1140071744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7857642d0 con 0x55e78588f820 2024-06-06T04:33:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f74ff9700 10 client.15732 handle_command_reply: tid=0 2024-06-06T04:33:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f9e1a4b80 1 client.15732 shutdown 2024-06-06T04:33:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1802879857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785810f60 msgr2=0x55e78581da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/1802879857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785810f60 0x55e78581da60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c004dd0 tx=0x7f8f7c004c90 comp rx=0 tx=0).stop 2024-06-06T04:33:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1802879857 shutdown_connections 2024-06-06T04:33:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/1802879857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e785811880 0x55e78582ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/1802879857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e78580b7a0 0x55e78581d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/1802879857 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x55e78588f820 0x55e785891c40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f9e1a4b80 1 --2- 172.21.15.45:0/1802879857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e785810f60 0x55e78581da60 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f74ff9700 0 client.15732 ms_handle_reset on v2:172.21.15.154:6836/1140071744 2024-06-06T04:33:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1802879857 >> 172.21.15.45:0/1802879857 conn(0x55e785738080 msgr2=0x55e785816120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1802879857 shutdown_connections 2024-06-06T04:33:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f9e1a4b80 1 -- 172.21.15.45:0/1802879857 wait complete. 2024-06-06T04:33:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.063+0000 7f8f9e1a4b80 20 client.15732 trim_cache size 0 max 16384 2024-06-06T04:33:33.069 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-06-06T04:33:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.064+0000 7f8f93921700 1 -- 172.21.15.45:0/2224886297 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8f780a2ca0 msgr2=0x7f8f780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.064+0000 7f8f93921700 1 --2- 172.21.15.45:0/2224886297 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8f780a2ca0 0x7f8f780a5160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8f84009ee0 tx=0x7f8f84009260 comp rx=0 tx=0).stop 2024-06-06T04:33:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.064+0000 7f8f93921700 1 -- 172.21.15.45:0/2224886297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c1027a0 msgr2=0x7f8f8c19f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.064+0000 7f8f93921700 1 --2- 172.21.15.45:0/2224886297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c1027a0 0x7f8f8c19f740 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8f88007a30 tx=0x7f8f88007a60 comp rx=0 tx=0).stop 2024-06-06T04:33:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.064+0000 7f8f93921700 1 -- 172.21.15.45:0/2224886297 shutdown_connections 2024-06-06T04:33:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.064+0000 7f8f93921700 1 --2- 172.21.15.45:0/2224886297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f8c107870 0x7f8f8c10e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.064+0000 7f8f93921700 1 --2- 172.21.15.45:0/2224886297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c1027a0 0x7f8f8c19f740 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.064+0000 7f8f93921700 1 --2- 172.21.15.45:0/2224886297 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f8f780a2ca0 0x7f8f780a5160 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.064+0000 7f8f93921700 1 --2- 172.21.15.45:0/2224886297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f8c103150 0x7f8f8c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:33.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.064+0000 7f8f93921700 1 -- 172.21.15.45:0/2224886297 >> 172.21.15.45:0/2224886297 conn(0x7f8f8c079570 msgr2=0x7f8f8c1089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:33.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.064+0000 7f8f93921700 1 -- 172.21.15.45:0/2224886297 shutdown_connections 2024-06-06T04:33:33.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:33.064+0000 7f8f93921700 1 -- 172.21.15.45:0/2224886297 wait complete. 2024-06-06T04:33:34.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:34 smithi154 ceph-mon[36428]: pgmap v494: 129 pgs: 129 active+clean; 35 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.6 MiB/s wr, 1.16k op/s 2024-06-06T04:33:34.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:34 smithi045 ceph-mon[32299]: pgmap v494: 129 pgs: 129 active+clean; 35 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.6 MiB/s wr, 1.16k op/s 2024-06-06T04:33:34.483 INFO:tasks.workunit.client.0.smithi045.stdout:autoheader 2024-06-06T04:33:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.585+0000 7f333affd640 1 -- 172.21.15.45:0/702867608 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564a0ef7ec20 con 0x564a0f08ec70 2024-06-06T04:33:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.585+0000 7f333affd640 10 client.15717 handle_command_reply: tid=0 2024-06-06T04:33:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.585+0000 7f3364b6c740 1 client.15717 shutdown 2024-06-06T04:33:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.586+0000 7f3364b6c740 1 -- 172.21.15.45:0/702867608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00e010 msgr2=0x564a0f01ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.586+0000 7f3364b6c740 1 --2- 172.21.15.45:0/702867608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00e010 0x564a0f01ca40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f334c033000 tx=0x7f334c03c740 comp rx=0 tx=0).stop 2024-06-06T04:33:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.586+0000 7f3364b6c740 1 -- 172.21.15.45:0/702867608 shutdown_connections 2024-06-06T04:33:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.586+0000 7f3364b6c740 1 --2- 172.21.15.45:0/702867608 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x564a0f08ec70 0x564a0f091090 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.586+0000 7f3364b6c740 1 --2- 172.21.15.45:0/702867608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0f00ea00 0x564a0f01cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.586+0000 7f3364b6c740 1 --2- 172.21.15.45:0/702867608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a0f013b70 0x564a0f01df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.586+0000 7f3364b6c740 1 --2- 172.21.15.45:0/702867608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0f00e010 0x564a0f01ca40 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.586+0000 7f3364b6c740 1 -- 172.21.15.45:0/702867608 >> 172.21.15.45:0/702867608 conn(0x564a0f009680 msgr2=0x564a0f00fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.586+0000 7f333affd640 0 client.15717 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.586+0000 7f3364b6c740 1 -- 172.21.15.45:0/702867608 shutdown_connections 2024-06-06T04:33:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.586+0000 7f3364b6c740 1 -- 172.21.15.45:0/702867608 wait complete. 2024-06-06T04:33:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.586+0000 7f3364b6c740 20 client.15717 trim_cache size 0 max 16384 2024-06-06T04:33:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-06-06T04:33:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-06-06T04:33:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-06-06T04:33:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.587+0000 7f3361e5d640 1 -- 172.21.15.45:0/794077722 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f33480a2cb0 msgr2=0x7f33480a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.587+0000 7f3361e5d640 1 --2- 172.21.15.45:0/794077722 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f33480a2cb0 0x7f33480a5170 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x564a0f00f260 tx=0x7f3344009940 comp rx=0 tx=0).stop 2024-06-06T04:33:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.587+0000 7f3361e5d640 1 -- 172.21.15.45:0/794077722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f335c1035e0 msgr2=0x7f335c0742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.587+0000 7f3361e5d640 1 --2- 172.21.15.45:0/794077722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f335c1035e0 0x7f335c0742d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f335400efd0 tx=0x7f335400c6a0 comp rx=0 tx=0).stop 2024-06-06T04:33:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.587+0000 7f3361e5d640 1 -- 172.21.15.45:0/794077722 shutdown_connections 2024-06-06T04:33:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.587+0000 7f3361e5d640 1 --2- 172.21.15.45:0/794077722 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f33480a2cb0 0x7f33480a5170 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.587+0000 7f3361e5d640 1 --2- 172.21.15.45:0/794077722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f335c1035e0 0x7f335c0742d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.587+0000 7f3361e5d640 1 --2- 172.21.15.45:0/794077722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f335c102c20 0x7f335c073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.587+0000 7f3361e5d640 1 --2- 172.21.15.45:0/794077722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f335c101a30 0x7f335c075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.587+0000 7f3361e5d640 1 -- 172.21.15.45:0/794077722 >> 172.21.15.45:0/794077722 conn(0x7f335c0fcfc0 msgr2=0x7f335c105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.587+0000 7f3361e5d640 1 -- 172.21.15.45:0/794077722 shutdown_connections 2024-06-06T04:33:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:34.587+0000 7f3361e5d640 1 -- 172.21.15.45:0/794077722 wait complete. 2024-06-06T04:33:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:34 smithi086 ceph-mon[36053]: pgmap v494: 129 pgs: 129 active+clean; 35 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.6 MiB/s wr, 1.16k op/s 2024-06-06T04:33:34.779 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-06T04:33:34.996 INFO:tasks.workunit.client.0.smithi045.stdout:autoconf 2024-06-06T04:33:35.046 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:33:35.249 INFO:tasks.workunit.client.0.smithi045.stdout:./configure \ 2024-06-06T04:33:35.249 INFO:tasks.workunit.client.0.smithi045.stdout: --libexecdir=/usr/lib \ 2024-06-06T04:33:35.249 INFO:tasks.workunit.client.0.smithi045.stdout: --exec_prefix=/var/lib 2024-06-06T04:33:35.468 INFO:tasks.workunit.client.0.smithi045.stdout:checking build system type... x86_64-pc-linux-gnu 2024-06-06T04:33:35.468 INFO:tasks.workunit.client.0.smithi045.stdout:checking host system type... x86_64-pc-linux-gnu 2024-06-06T04:33:35.470 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to print strings... printf 2024-06-06T04:33:35.471 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc... gcc 2024-06-06T04:33:35.604 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether the C compiler works... yes 2024-06-06T04:33:35.605 INFO:tasks.workunit.client.0.smithi045.stdout:checking for C compiler default output file name... a.out 2024-06-06T04:33:35.719 INFO:tasks.workunit.client.0.smithi045.stdout:checking for suffix of executables... 2024-06-06T04:33:35.782 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether we are cross compiling... no 2024-06-06T04:33:35.842 INFO:tasks.workunit.client.0.smithi045.stdout:checking for suffix of object files... o 2024-06-06T04:33:35.882 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether we are using the GNU C compiler... yes 2024-06-06T04:33:35.919 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether gcc accepts -g... yes 2024-06-06T04:33:36.070 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc option to accept ISO C89... none needed 2024-06-06T04:33:36.076 INFO:tasks.workunit.client.0.smithi045.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-06-06T04:33:36.081 INFO:tasks.workunit.client.0.smithi045.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-06-06T04:33:36.083 INFO:tasks.workunit.client.0.smithi045.stdout:checking for egrep... /usr/bin/grep -E 2024-06-06T04:33:36.085 INFO:tasks.workunit.client.0.smithi045.stdout:checking for fgrep... /usr/bin/grep -F 2024-06-06T04:33:36.088 INFO:tasks.workunit.client.0.smithi045.stdout:checking for ld used by gcc... /usr/bin/ld 2024-06-06T04:33:36.091 INFO:tasks.workunit.client.0.smithi045.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-06-06T04:33:36.125 INFO:tasks.workunit.client.0.smithi045.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-06-06T04:33:36.427 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:36 smithi045 ceph-mon[32299]: pgmap v495: 129 pgs: 129 active+clean; 41 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 986 op/s 2024-06-06T04:33:36.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:36 smithi154 ceph-mon[36428]: pgmap v495: 129 pgs: 129 active+clean; 41 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 986 op/s 2024-06-06T04:33:36.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:36 smithi086 ceph-mon[36053]: pgmap v495: 129 pgs: 129 active+clean; 41 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 986 op/s 2024-06-06T04:33:37.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.130+0000 7fa44a79f700 1 -- 172.21.15.45:0/4097957385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa444108580 msgr2=0x7fa444108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:37.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.130+0000 7fa44a79f700 1 --2- 172.21.15.45:0/4097957385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa444108580 0x7fa444108960 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa4340099c0 tx=0x7fa434009cd0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.130+0000 7fa44a79f700 1 -- 172.21.15.45:0/4097957385 shutdown_connections 2024-06-06T04:33:37.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.130+0000 7fa44a79f700 1 --2- 172.21.15.45:0/4097957385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa44406b620 0x7fa44410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.130+0000 7fa44a79f700 1 --2- 172.21.15.45:0/4097957385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa444108580 0x7fa444108960 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.130+0000 7fa44a79f700 1 --2- 172.21.15.45:0/4097957385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa44406ac60 0x7fa44406b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.130+0000 7fa44a79f700 1 -- 172.21.15.45:0/4097957385 >> 172.21.15.45:0/4097957385 conn(0x7fa444078430 msgr2=0x7fa444078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:37.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.130+0000 7fa44a79f700 1 -- 172.21.15.45:0/4097957385 shutdown_connections 2024-06-06T04:33:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.130+0000 7fa44a79f700 1 -- 172.21.15.45:0/4097957385 wait complete. 2024-06-06T04:33:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.130+0000 7fa44a79f700 1 Processor -- start 2024-06-06T04:33:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.130+0000 7fa44a79f700 1 -- start start 2024-06-06T04:33:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa44a79f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa44406ac60 0x7fa44419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa44a79f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa44406b620 0x7fa44419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa44a79f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa444108580 0x7fa444198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa44a79f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa444112680 con 0x7fa44406b620 2024-06-06T04:33:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa44a79f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa444112500 con 0x7fa444108580 2024-06-06T04:33:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa44a79f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa444112800 con 0x7fa44406ac60 2024-06-06T04:33:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa4437fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa44406b620 0x7fa44419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa4437fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa44406b620 0x7fa44419ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57936/0 (socket says 172.21.15.45:57936) 2024-06-06T04:33:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa4437fe700 1 -- 172.21.15.45:0/310860445 learned_addr learned my addr 172.21.15.45:0/310860445 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa443fff700 1 --2- 172.21.15.45:0/310860445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa44406ac60 0x7fa44419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa448d3c700 1 --2- 172.21.15.45:0/310860445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa444108580 0x7fa444198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa4437fe700 1 -- 172.21.15.45:0/310860445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa44406ac60 msgr2=0x7fa44419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa4437fe700 1 --2- 172.21.15.45:0/310860445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa44406ac60 0x7fa44419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa4437fe700 1 -- 172.21.15.45:0/310860445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa444108580 msgr2=0x7fa444198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa4437fe700 1 --2- 172.21.15.45:0/310860445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa444108580 0x7fa444198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa4437fe700 1 -- 172.21.15.45:0/310860445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4441990a0 con 0x7fa44406b620 2024-06-06T04:33:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.131+0000 7fa4437fe700 1 --2- 172.21.15.45:0/310860445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa44406b620 0x7fa44419ecc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00d980 tx=0x7fa42c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.132+0000 7fa4417fa700 1 -- 172.21.15.45:0/310860445 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c017070 con 0x7fa44406b620 2024-06-06T04:33:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.132+0000 7fa44a79f700 1 -- 172.21.15.45:0/310860445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa444199390 con 0x7fa44406b620 2024-06-06T04:33:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.132+0000 7fa4417fa700 1 -- 172.21.15.45:0/310860445 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa42c005020 con 0x7fa44406b620 2024-06-06T04:33:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.132+0000 7fa4417fa700 1 -- 172.21.15.45:0/310860445 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c009b50 con 0x7fa44406b620 2024-06-06T04:33:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.132+0000 7fa44a79f700 1 -- 172.21.15.45:0/310860445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4441a5650 con 0x7fa44406b620 2024-06-06T04:33:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.134+0000 7fa4417fa700 1 -- 172.21.15.45:0/310860445 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa42c009cf0 con 0x7fa44406b620 2024-06-06T04:33:37.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.134+0000 7fa4417fa700 1 --2- 172.21.15.45:0/310860445 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa4300a2c20 0x7fa4300a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.135+0000 7fa443fff700 1 --2- 172.21.15.45:0/310860445 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa4300a2c20 0x7fa4300a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.135+0000 7fa4417fa700 1 -- 172.21.15.45:0/310860445 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fa42c0cf3d0 con 0x7fa44406b620 2024-06-06T04:33:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.135+0000 7fa443fff700 1 --2- 172.21.15.45:0/310860445 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa4300a2c20 0x7fa4300a50e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa434000c00 tx=0x7fa434005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.135+0000 7fa455022b80 1 Processor -- start 2024-06-06T04:33:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.135+0000 7fa455022b80 1 -- start start 2024-06-06T04:33:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa455022b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705590fd0 0x55a7055913b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa455022b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a7055956f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa455022b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a705595c30 0x55a705596090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa455022b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a7054de810 con 0x55a7055918f0 2024-06-06T04:33:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa455022b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a705486720 con 0x55a705590fd0 2024-06-06T04:33:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa455022b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a7054da7d0 con 0x55a705595c30 2024-06-06T04:33:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa4437fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a7055956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa4437fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a7055956f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57948/0 (socket says 172.21.15.45:57948) 2024-06-06T04:33:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa4437fe700 1 -- 172.21.15.45:0/1990360348 learned_addr learned my addr 172.21.15.45:0/1990360348 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa443fff700 1 --2- 172.21.15.45:0/1990360348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705590fd0 0x55a7055913b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa448d3c700 1 --2- 172.21.15.45:0/1990360348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a705595c30 0x55a705596090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa4437fe700 1 -- 172.21.15.45:0/1990360348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a705595c30 msgr2=0x55a705596090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa4437fe700 1 --2- 172.21.15.45:0/1990360348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a705595c30 0x55a705596090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa4437fe700 1 -- 172.21.15.45:0/1990360348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705590fd0 msgr2=0x55a7055913b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa4437fe700 1 --2- 172.21.15.45:0/1990360348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705590fd0 0x55a7055913b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa4437fe700 1 -- 172.21.15.45:0/1990360348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a70547cf50 con 0x55a7055918f0 2024-06-06T04:33:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa4437fe700 1 --2- 172.21.15.45:0/1990360348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a7055956f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa42c09b7a0 tx=0x7fa42c005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5614924c68d0ce82 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa43a7fc700 1 -- 172.21.15.45:0/1990360348 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c09fbc0 con 0x55a7055918f0 2024-06-06T04:33:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa43a7fc700 1 -- 172.21.15.45:0/1990360348 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa42c0a4480 con 0x55a7055918f0 2024-06-06T04:33:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.136+0000 7fa43a7fc700 1 -- 172.21.15.45:0/1990360348 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c09ba70 con 0x55a7055918f0 2024-06-06T04:33:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.137+0000 7fa455022b80 1 -- 172.21.15.45:0/1990360348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 msgr2=0x55a7055956f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:37.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.137+0000 7fa455022b80 1 --2- 172.21.15.45:0/1990360348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a7055956f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa42c09b7a0 tx=0x7fa42c005690 comp rx=0 tx=0).stop 2024-06-06T04:33:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.137+0000 7fa455022b80 1 -- 172.21.15.45:0/1990360348 shutdown_connections 2024-06-06T04:33:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.137+0000 7fa455022b80 1 --2- 172.21.15.45:0/1990360348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a705595c30 0x55a705596090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.137+0000 7fa455022b80 1 --2- 172.21.15.45:0/1990360348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705590fd0 0x55a7055913b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.137+0000 7fa455022b80 1 --2- 172.21.15.45:0/1990360348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a7055956f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.137+0000 7fa455022b80 1 -- 172.21.15.45:0/1990360348 >> 172.21.15.45:0/1990360348 conn(0x55a7054b7cc0 msgr2=0x55a705587850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.137+0000 7fa455022b80 1 -- 172.21.15.45:0/1990360348 shutdown_connections 2024-06-06T04:33:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.137+0000 7fa455022b80 1 -- 172.21.15.45:0/1990360348 wait complete. 2024-06-06T04:33:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.138+0000 7fa455022b80 1 -- start start 2024-06-06T04:33:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.138+0000 7fa455022b80 1 Processor -- start 2024-06-06T04:33:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.138+0000 7fa455022b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a705590fd0 0x55a70559d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.138+0000 7fa455022b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a70559a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.138+0000 7fa455022b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705595c30 0x55a70559aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.138+0000 7fa455022b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a70559dd70 con 0x55a7055918f0 2024-06-06T04:33:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.138+0000 7fa455022b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a70547ebf0 con 0x55a705595c30 2024-06-06T04:33:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.138+0000 7fa455022b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a70559def0 con 0x55a705590fd0 2024-06-06T04:33:37.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.138+0000 7fa455022b80 10 client.0 _ll_register_callbacks cb 0x55a705478920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:37.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.138+0000 7fa4437fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a70559a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.181 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-06-06T04:33:37.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.138+0000 7fa4437fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a70559a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57964/0 (socket says 172.21.15.45:57964) 2024-06-06T04:33:37.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.138+0000 7fa4437fe700 1 -- 172.21.15.45:0/3223644671 learned_addr learned my addr 172.21.15.45:0/3223644671 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:37.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.138+0000 7fa448d3c700 1 --2- 172.21.15.45:0/3223644671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705595c30 0x55a70559aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.138+0000 7fa443fff700 1 --2- 172.21.15.45:0/3223644671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a705590fd0 0x55a70559d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.139+0000 7fa4437fe700 1 -- 172.21.15.45:0/3223644671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a705590fd0 msgr2=0x55a70559d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:37.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.139+0000 7fa4437fe700 1 --2- 172.21.15.45:0/3223644671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a705590fd0 0x55a70559d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.139+0000 7fa4437fe700 1 -- 172.21.15.45:0/3223644671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705595c30 msgr2=0x55a70559aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:37.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.139+0000 7fa4437fe700 1 --2- 172.21.15.45:0/3223644671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705595c30 0x55a70559aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.139+0000 7fa4437fe700 1 -- 172.21.15.45:0/3223644671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a70542d8d0 con 0x55a7055918f0 2024-06-06T04:33:37.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.139+0000 7fa4437fe700 1 --2- 172.21.15.45:0/3223644671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a70559a550 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0047d0 tx=0x7fa42c0860c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb8a8e01fcc7608 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:37.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.139+0000 7fa4237fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:33:37.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.139+0000 7fa4237fe700 1 -- 172.21.15.45:0/3223644671 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c09fbc0 con 0x55a7055918f0 2024-06-06T04:33:37.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.139+0000 7fa4237fe700 1 -- 172.21.15.45:0/3223644671 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fa42c00e350 con 0x55a7055918f0 2024-06-06T04:33:37.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.139+0000 7fa455022b80 1 -- 172.21.15.45:0/3223644671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa42c0a4480 con 0x55a7055918f0 2024-06-06T04:33:37.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.139+0000 7fa4237fe700 1 -- 172.21.15.45:0/3223644671 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa42c00e550 con 0x55a7055918f0 2024-06-06T04:33:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.139+0000 7fa455022b80 10 client.15744 fetch_fsmap learned FSMap version 30 2024-06-06T04:33:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.139+0000 7fa455022b80 1 -- 172.21.15.45:0/3223644671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a7055a8750 con 0x55a7055918f0 2024-06-06T04:33:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.139+0000 7fa4237fe700 1 -- 172.21.15.45:0/3223644671 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa42c0a4480 con 0x55a7055918f0 2024-06-06T04:33:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.139+0000 7fa4237fe700 1 -- 172.21.15.45:0/3223644671 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fa42c0a4080 con 0x55a7055918f0 2024-06-06T04:33:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.140+0000 7fa455022b80 10 client.15744 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:33:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.140+0000 7fa455022b80 10 client.15744 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T04:33:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.140+0000 7fa455022b80 20 client.15744 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.140+0000 7fa455022b80 1 --2- 172.21.15.45:0/3223644671 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55a7055aac20 0x55a7055ad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.140+0000 7fa455022b80 4 client.15744 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:33:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.140+0000 7fa455022b80 1 -- 172.21.15.45:0/3223644671 --> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7054ebd10 con 0x55a7055aac20 2024-06-06T04:33:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.140+0000 7fa443fff700 1 --2- 172.21.15.45:0/3223644671 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55a7055aac20 0x55a7055ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.140+0000 7fa443fff700 1 --2- 172.21.15.45:0/3223644671 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55a7055aac20 0x55a7055ad010 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50ca41bdda1c9464 server_cookie=207351716403023f in_seq=0 out_seq=0 2024-06-06T04:33:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.140+0000 7fa4237fe700 10 client.15744 ms_handle_connect on v2:172.21.15.154:6838/982197202 2024-06-06T04:33:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.141+0000 7fa4237fe700 1 -- 172.21.15.45:0/3223644671 <== mds.1 v2:172.21.15.154:6838/982197202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a7054ebd10 con 0x55a7055aac20 2024-06-06T04:33:37.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.141+0000 7fa4237fe700 10 client.15744 handle_command_reply: tid=0 2024-06-06T04:33:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.141+0000 7fa455022b80 1 client.15744 shutdown 2024-06-06T04:33:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.141+0000 7fa455022b80 1 -- 172.21.15.45:0/3223644671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 msgr2=0x55a70559a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.141+0000 7fa455022b80 1 --2- 172.21.15.45:0/3223644671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a70559a550 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0047d0 tx=0x7fa42c0860c0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.141+0000 7fa455022b80 1 -- 172.21.15.45:0/3223644671 shutdown_connections 2024-06-06T04:33:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.141+0000 7fa455022b80 1 --2- 172.21.15.45:0/3223644671 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55a7055aac20 0x55a7055ad010 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.141+0000 7fa455022b80 1 --2- 172.21.15.45:0/3223644671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705595c30 0x55a70559aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.141+0000 7fa455022b80 1 --2- 172.21.15.45:0/3223644671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a705590fd0 0x55a70559d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.141+0000 7fa4237fe700 0 client.15744 ms_handle_reset on v2:172.21.15.154:6838/982197202 2024-06-06T04:33:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.141+0000 7fa455022b80 1 --2- 172.21.15.45:0/3223644671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a70559a550 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.141+0000 7fa455022b80 1 -- 172.21.15.45:0/3223644671 >> 172.21.15.45:0/3223644671 conn(0x55a7054b7cc0 msgr2=0x55a70558b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.141+0000 7fa455022b80 1 -- 172.21.15.45:0/3223644671 shutdown_connections 2024-06-06T04:33:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.141+0000 7fa455022b80 1 -- 172.21.15.45:0/3223644671 wait complete. 2024-06-06T04:33:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.142+0000 7fa455022b80 20 client.15744 trim_cache size 0 max 16384 2024-06-06T04:33:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.157+0000 7fa455022b80 1 Processor -- start 2024-06-06T04:33:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.157+0000 7fa455022b80 1 -- start start 2024-06-06T04:33:37.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.157+0000 7fa455022b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705590fd0 0x55a70559d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.157+0000 7fa455022b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a7055879f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.157+0000 7fa455022b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a705595c30 0x55a705581690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.157+0000 7fa455022b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4441ae930 con 0x55a7055918f0 2024-06-06T04:33:37.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.157+0000 7fa455022b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4441ac190 con 0x55a705590fd0 2024-06-06T04:33:37.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.157+0000 7fa455022b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa4441a9fe0 con 0x55a705595c30 2024-06-06T04:33:37.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.157+0000 7fa4437fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a7055879f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.157+0000 7fa4437fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a7055879f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57972/0 (socket says 172.21.15.45:57972) 2024-06-06T04:33:37.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.157+0000 7fa4437fe700 1 -- 172.21.15.45:0/4260649718 learned_addr learned my addr 172.21.15.45:0/4260649718 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:37.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.157+0000 7fa443fff700 1 --2- 172.21.15.45:0/4260649718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705590fd0 0x55a70559d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.157+0000 7fa4437fe700 1 -- 172.21.15.45:0/4260649718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a705595c30 msgr2=0x55a705581690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:37.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.157+0000 7fa448d3c700 1 --2- 172.21.15.45:0/4260649718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a705595c30 0x55a705581690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.157+0000 7fa4437fe700 1 --2- 172.21.15.45:0/4260649718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a705595c30 0x55a705581690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.157+0000 7fa4437fe700 1 -- 172.21.15.45:0/4260649718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705590fd0 msgr2=0x55a70559d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:37.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.158+0000 7fa4437fe700 1 --2- 172.21.15.45:0/4260649718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705590fd0 0x55a70559d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.158+0000 7fa4437fe700 1 -- 172.21.15.45:0/4260649718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a705598ac0 con 0x55a7055918f0 2024-06-06T04:33:37.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.158+0000 7fa443fff700 1 --2- 172.21.15.45:0/4260649718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705590fd0 0x55a70559d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:33:37.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.158+0000 7fa448d3c700 1 --2- 172.21.15.45:0/4260649718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a705595c30 0x55a705581690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:33:37.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.158+0000 7fa4437fe700 1 --2- 172.21.15.45:0/4260649718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a7055879f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0981e0 tx=0x7fa42c099d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67349f0bc2d8199 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:37.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.158+0000 7fa439ffb700 1 -- 172.21.15.45:0/4260649718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c00ee30 con 0x55a7055918f0 2024-06-06T04:33:37.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.158+0000 7fa439ffb700 1 -- 172.21.15.45:0/4260649718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa42c0a4480 con 0x55a7055918f0 2024-06-06T04:33:37.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.158+0000 7fa439ffb700 1 -- 172.21.15.45:0/4260649718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c09ba70 con 0x55a7055918f0 2024-06-06T04:33:37.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.158+0000 7fa455022b80 1 -- 172.21.15.45:0/4260649718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 msgr2=0x55a7055879f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:37.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.158+0000 7fa455022b80 1 --2- 172.21.15.45:0/4260649718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a7055879f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0981e0 tx=0x7fa42c099d20 comp rx=0 tx=0).stop 2024-06-06T04:33:37.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.158+0000 7fa455022b80 1 -- 172.21.15.45:0/4260649718 shutdown_connections 2024-06-06T04:33:37.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.159+0000 7fa455022b80 1 --2- 172.21.15.45:0/4260649718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a705595c30 0x55a705581690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.159+0000 7fa455022b80 1 --2- 172.21.15.45:0/4260649718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705590fd0 0x55a70559d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.159+0000 7fa455022b80 1 --2- 172.21.15.45:0/4260649718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a7055918f0 0x55a7055879f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.159+0000 7fa455022b80 1 -- 172.21.15.45:0/4260649718 >> 172.21.15.45:0/4260649718 conn(0x55a7054b7cc0 msgr2=0x55a705598e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:37.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.159+0000 7fa455022b80 1 -- 172.21.15.45:0/4260649718 shutdown_connections 2024-06-06T04:33:37.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.159+0000 7fa455022b80 1 -- 172.21.15.45:0/4260649718 wait complete. 2024-06-06T04:33:37.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.159+0000 7fa455022b80 1 -- start start 2024-06-06T04:33:37.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.159+0000 7fa455022b80 1 Processor -- start 2024-06-06T04:33:37.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.159+0000 7fa455022b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a705590fd0 0x55a70559e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.159+0000 7fa455022b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7055918f0 0x55a70559e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.159+0000 7fa455022b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705595c30 0x55a705613660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.159+0000 7fa455022b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4441aa650 con 0x55a705590fd0 2024-06-06T04:33:37.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.159+0000 7fa455022b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4441abac0 con 0x55a705595c30 2024-06-06T04:33:37.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.159+0000 7fa455022b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa4441aad20 con 0x55a7055918f0 2024-06-06T04:33:37.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.159+0000 7fa455022b80 10 client.0 _ll_register_callbacks cb 0x55a705478920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:37.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa443fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a705590fd0 0x55a70559e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa443fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a705590fd0 0x55a70559e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57974/0 (socket says 172.21.15.45:57974) 2024-06-06T04:33:37.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa443fff700 1 -- 172.21.15.45:0/3522582293 learned_addr learned my addr 172.21.15.45:0/3522582293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:37.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa448d3c700 1 --2- 172.21.15.45:0/3522582293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705595c30 0x55a705613660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa4437fe700 1 --2- 172.21.15.45:0/3522582293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7055918f0 0x55a70559e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa443fff700 1 -- 172.21.15.45:0/3522582293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7055918f0 msgr2=0x55a70559e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:37.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa443fff700 1 --2- 172.21.15.45:0/3522582293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7055918f0 0x55a70559e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa443fff700 1 -- 172.21.15.45:0/3522582293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705595c30 msgr2=0x55a705613660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:37.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa443fff700 1 --2- 172.21.15.45:0/3522582293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705595c30 0x55a705613660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa443fff700 1 -- 172.21.15.45:0/3522582293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a705613ba0 con 0x55a705590fd0 2024-06-06T04:33:37.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa443fff700 1 --2- 172.21.15.45:0/3522582293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a705590fd0 0x55a70559e320 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa4340099c0 tx=0x7fa434009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6316fee76942e84 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:37.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa4237fe700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:33:37.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa4237fe700 1 -- 172.21.15.45:0/3522582293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434037030 con 0x55a705590fd0 2024-06-06T04:33:37.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa455022b80 1 -- 172.21.15.45:0/3522582293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa42c0a4480 con 0x55a705590fd0 2024-06-06T04:33:37.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa4237fe700 1 -- 172.21.15.45:0/3522582293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fa43400b9a0 con 0x55a705590fd0 2024-06-06T04:33:37.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa4237fe700 1 -- 172.21.15.45:0/3522582293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa434044050 con 0x55a705590fd0 2024-06-06T04:33:37.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa455022b80 10 client.15750 fetch_fsmap learned FSMap version 30 2024-06-06T04:33:37.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.160+0000 7fa455022b80 1 -- 172.21.15.45:0/3522582293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a705614d00 con 0x55a705590fd0 2024-06-06T04:33:37.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.161+0000 7fa4237fe700 1 -- 172.21.15.45:0/3522582293 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa42c0a4480 con 0x55a705590fd0 2024-06-06T04:33:37.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.161+0000 7fa4237fe700 1 -- 172.21.15.45:0/3522582293 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fa43400fce0 con 0x55a705590fd0 2024-06-06T04:33:37.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.161+0000 7fa455022b80 10 client.15750 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:33:37.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.161+0000 7fa455022b80 10 client.15750 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T04:33:37.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.161+0000 7fa455022b80 20 client.15750 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:37.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.161+0000 7fa455022b80 1 --2- 172.21.15.45:0/3522582293 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55a705615420 0x55a705617840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:37.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.161+0000 7fa455022b80 4 client.15750 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T04:33:37.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.161+0000 7fa455022b80 1 -- 172.21.15.45:0/3522582293 --> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7054e6290 con 0x55a705615420 2024-06-06T04:33:37.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.161+0000 7fa4437fe700 1 --2- 172.21.15.45:0/3522582293 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55a705615420 0x55a705617840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:37.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa4237fe700 10 client.15750 ms_handle_connect on v2:172.21.15.154:6838/982197202 2024-06-06T04:33:37.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa4437fe700 1 --2- 172.21.15.45:0/3522582293 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55a705615420 0x55a705617840 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3cc003ddd1b9822 server_cookie=cd7baf61b082f9f2 in_seq=0 out_seq=0 2024-06-06T04:33:37.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa4237fe700 1 -- 172.21.15.45:0/3522582293 <== mds.1 v2:172.21.15.154:6838/982197202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7054e6290 con 0x55a705615420 2024-06-06T04:33:37.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa4237fe700 10 client.15750 handle_command_reply: tid=0 2024-06-06T04:33:37.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa455022b80 1 client.15750 shutdown 2024-06-06T04:33:37.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa455022b80 1 -- 172.21.15.45:0/3522582293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a705590fd0 msgr2=0x55a70559e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:37.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa455022b80 1 --2- 172.21.15.45:0/3522582293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a705590fd0 0x55a70559e320 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa4340099c0 tx=0x7fa434009cd0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa455022b80 1 -- 172.21.15.45:0/3522582293 shutdown_connections 2024-06-06T04:33:37.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa455022b80 1 --2- 172.21.15.45:0/3522582293 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55a705615420 0x55a705617840 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa455022b80 1 --2- 172.21.15.45:0/3522582293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a705595c30 0x55a705613660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa455022b80 1 --2- 172.21.15.45:0/3522582293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a705590fd0 0x55a70559e320 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa455022b80 1 --2- 172.21.15.45:0/3522582293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7055918f0 0x55a70559e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa4237fe700 0 client.15750 ms_handle_reset on v2:172.21.15.154:6838/982197202 2024-06-06T04:33:37.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa455022b80 1 -- 172.21.15.45:0/3522582293 >> 172.21.15.45:0/3522582293 conn(0x55a7054b7cc0 msgr2=0x55a705594bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa455022b80 1 -- 172.21.15.45:0/3522582293 shutdown_connections 2024-06-06T04:33:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa455022b80 1 -- 172.21.15.45:0/3522582293 wait complete. 2024-06-06T04:33:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.162+0000 7fa455022b80 20 client.15750 trim_cache size 0 max 16384 2024-06-06T04:33:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.163+0000 7fa44a79f700 1 -- 172.21.15.45:0/310860445 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa4300a2c20 msgr2=0x7fa4300a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.163+0000 7fa44a79f700 1 --2- 172.21.15.45:0/310860445 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa4300a2c20 0x7fa4300a50e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa434000c00 tx=0x7fa434005fb0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.163+0000 7fa44a79f700 1 -- 172.21.15.45:0/310860445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa44406b620 msgr2=0x7fa44419ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.163+0000 7fa44a79f700 1 --2- 172.21.15.45:0/310860445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa44406b620 0x7fa44419ecc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00d980 tx=0x7fa42c00dc90 comp rx=0 tx=0).stop 2024-06-06T04:33:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.163+0000 7fa44a79f700 1 -- 172.21.15.45:0/310860445 shutdown_connections 2024-06-06T04:33:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.163+0000 7fa44a79f700 1 --2- 172.21.15.45:0/310860445 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa4300a2c20 0x7fa4300a50e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.163+0000 7fa44a79f700 1 --2- 172.21.15.45:0/310860445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa444108580 0x7fa444198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.163+0000 7fa44a79f700 1 --2- 172.21.15.45:0/310860445 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa44406ac60 0x7fa44419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.163+0000 7fa44a79f700 1 --2- 172.21.15.45:0/310860445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa44406b620 0x7fa44419ecc0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.163+0000 7fa44a79f700 1 -- 172.21.15.45:0/310860445 >> 172.21.15.45:0/310860445 conn(0x7fa444078430 msgr2=0x7fa4440fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.163+0000 7fa44a79f700 1 -- 172.21.15.45:0/310860445 shutdown_connections 2024-06-06T04:33:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:37.163+0000 7fa44a79f700 1 -- 172.21.15.45:0/310860445 wait complete. 2024-06-06T04:33:38.184 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-06T04:33:38.429 INFO:tasks.workunit.client.0.smithi045.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-06-06T04:33:38.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:38 smithi154 ceph-mon[36428]: pgmap v496: 129 pgs: 129 active+clean; 41 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 492 op/s 2024-06-06T04:33:38.430 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether ln -s works... yes 2024-06-06T04:33:38.444 INFO:tasks.workunit.client.0.smithi045.stdout:checking the maximum length of command line arguments... 1572864 2024-06-06T04:33:38.444 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:33:38.447 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-06-06T04:33:38.448 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-06-06T04:33:38.448 INFO:tasks.workunit.client.0.smithi045.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-06-06T04:33:38.449 INFO:tasks.workunit.client.0.smithi045.stdout:checking for objdump... objdump 2024-06-06T04:33:38.450 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to recognize dependent libraries... pass_all 2024-06-06T04:33:38.451 INFO:tasks.workunit.client.0.smithi045.stdout:checking for dlltool... no 2024-06-06T04:33:38.451 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-06-06T04:33:38.452 INFO:tasks.workunit.client.0.smithi045.stdout:checking for ar... ar 2024-06-06T04:33:38.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:38 smithi045 ceph-mon[32299]: pgmap v496: 129 pgs: 129 active+clean; 41 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 492 op/s 2024-06-06T04:33:38.603 INFO:tasks.workunit.client.0.smithi045.stdout:checking for archiver @FILE support... @ 2024-06-06T04:33:38.603 INFO:tasks.workunit.client.0.smithi045.stdout:checking for strip... strip 2024-06-06T04:33:38.604 INFO:tasks.workunit.client.0.smithi045.stdout:checking for ranlib... ranlib 2024-06-06T04:33:38.605 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gawk... gawk 2024-06-06T04:33:38.608 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:38 smithi086 ceph-mon[36053]: pgmap v496: 129 pgs: 129 active+clean; 41 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 492 op/s 2024-06-06T04:33:38.782 INFO:tasks.workunit.client.0.smithi045.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-06-06T04:33:38.783 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sysroot... no 2024-06-06T04:33:38.805 INFO:tasks.workunit.client.0.smithi045.stdout:checking for a working dd... /usr/bin/dd 2024-06-06T04:33:38.824 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-06-06T04:33:38.914 INFO:tasks.workunit.client.0.smithi045.stdout:checking for mt... no 2024-06-06T04:33:38.935 INFO:tasks.workunit.client.0.smithi045.stdout:checking if : is a manifest tool... no 2024-06-06T04:33:39.503 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to run the C preprocessor... gcc -E 2024-06-06T04:33:40.424 INFO:tasks.workunit.client.0.smithi045.stdout:checking for ANSI C header files... yes 2024-06-06T04:33:40.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:40 smithi154 ceph-mon[36428]: pgmap v497: 129 pgs: 129 active+clean; 26 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 852 op/s 2024-06-06T04:33:40.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:40 smithi045 ceph-mon[32299]: pgmap v497: 129 pgs: 129 active+clean; 26 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 852 op/s 2024-06-06T04:33:40.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:40 smithi086 ceph-mon[36053]: pgmap v497: 129 pgs: 129 active+clean; 26 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 852 op/s 2024-06-06T04:33:40.678 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/types.h... yes 2024-06-06T04:33:40.791 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/stat.h... yes 2024-06-06T04:33:40.955 INFO:tasks.workunit.client.0.smithi045.stdout:checking for stdlib.h... yes 2024-06-06T04:33:41.075 INFO:tasks.workunit.client.0.smithi045.stdout:checking for string.h... yes 2024-06-06T04:33:41.333 INFO:tasks.workunit.client.0.smithi045.stdout:checking for memory.h... yes 2024-06-06T04:33:41.524 INFO:tasks.workunit.client.0.smithi045.stdout:checking for strings.h... yes 2024-06-06T04:33:41.710 INFO:tasks.workunit.client.0.smithi045.stdout:checking for inttypes.h... yes 2024-06-06T04:33:41.844 INFO:tasks.workunit.client.0.smithi045.stdout:checking for stdint.h... yes 2024-06-06T04:33:42.020 INFO:tasks.workunit.client.0.smithi045.stdout:checking for unistd.h... yes 2024-06-06T04:33:42.302 INFO:tasks.workunit.client.0.smithi045.stdout:checking for dlfcn.h... yes 2024-06-06T04:33:42.314 INFO:tasks.workunit.client.0.smithi045.stdout:checking for objdir... .libs 2024-06-06T04:33:42.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:42 smithi154 ceph-mon[36428]: pgmap v498: 129 pgs: 129 active+clean; 28 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.2 MiB/s wr, 831 op/s 2024-06-06T04:33:42.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:42 smithi045 ceph-mon[32299]: pgmap v498: 129 pgs: 129 active+clean; 28 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.2 MiB/s wr, 831 op/s 2024-06-06T04:33:42.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:42 smithi086 ceph-mon[36053]: pgmap v498: 129 pgs: 129 active+clean; 28 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.2 MiB/s wr, 831 op/s 2024-06-06T04:33:42.811 INFO:tasks.workunit.client.0.smithi045.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-06-06T04:33:42.811 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-06-06T04:33:42.960 INFO:tasks.workunit.client.0.smithi045.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-06-06T04:33:43.417 INFO:tasks.workunit.client.0.smithi045.stdout:checking if gcc static flag -static works... no 2024-06-06T04:33:43.556 INFO:tasks.workunit.client.0.smithi045.stdout:checking if gcc supports -c -o file.o... yes 2024-06-06T04:33:43.556 INFO:tasks.workunit.client.0.smithi045.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-06-06T04:33:43.564 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-06-06T04:33:43.793 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether -lc should be explicitly linked in... no 2024-06-06T04:33:43.916 INFO:tasks.workunit.client.0.smithi045.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-06-06T04:33:43.917 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to hardcode library paths into programs... immediate 2024-06-06T04:33:43.920 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether stripping libraries is possible... yes 2024-06-06T04:33:43.921 INFO:tasks.workunit.client.0.smithi045.stdout:checking if libtool supports shared libraries... yes 2024-06-06T04:33:43.922 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether to build shared libraries... yes 2024-06-06T04:33:43.923 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether to build static libraries... yes 2024-06-06T04:33:43.932 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc... (cached) gcc 2024-06-06T04:33:43.988 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-06-06T04:33:43.988 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether gcc accepts -g... (cached) yes 2024-06-06T04:33:43.989 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-06-06T04:33:43.989 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gmake... /usr/bin/gmake 2024-06-06T04:33:43.990 INFO:tasks.workunit.client.0.smithi045.stderr:./configure: line 11968: test: $(SHELL): binary operator expected 2024-06-06T04:33:43.990 INFO:tasks.workunit.client.0.smithi045.stdout:checking for tar... /bin/tar 2024-06-06T04:33:43.991 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gzip... /bin/gzip 2024-06-06T04:33:44.047 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether gcc -MM is supported... yes 2024-06-06T04:33:44.048 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sort... /bin/sort 2024-06-06T04:33:44.048 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether ln -s works... yes 2024-06-06T04:33:44.049 INFO:tasks.workunit.client.0.smithi045.stdout:checking for rpm... /bin/rpm 2024-06-06T04:33:44.054 INFO:tasks.workunit.client.0.smithi045.stdout:checking for rpmbuild... no 2024-06-06T04:33:44.054 INFO:tasks.workunit.client.0.smithi045.stdout:checking for ANSI C header files... (cached) yes 2024-06-06T04:33:44.157 INFO:tasks.workunit.client.0.smithi045.stdout:checking assert.h usability... yes 2024-06-06T04:33:44.183 INFO:tasks.workunit.client.0.smithi045.stdout:checking assert.h presence... yes 2024-06-06T04:33:44.183 INFO:tasks.workunit.client.0.smithi045.stdout:checking for assert.h... yes 2024-06-06T04:33:44.279 INFO:tasks.workunit.client.0.smithi045.stdout:checking bstring.h usability... no 2024-06-06T04:33:44.324 INFO:tasks.workunit.client.0.smithi045.stdout:checking bstring.h presence... no 2024-06-06T04:33:44.324 INFO:tasks.workunit.client.0.smithi045.stdout:checking for bstring.h... no 2024-06-06T04:33:44.397 INFO:tasks.workunit.client.0.smithi045.stdout:checking libgen.h usability... yes 2024-06-06T04:33:44.421 INFO:tasks.workunit.client.0.smithi045.stdout:checking libgen.h presence... yes 2024-06-06T04:33:44.421 INFO:tasks.workunit.client.0.smithi045.stdout:checking for libgen.h... yes 2024-06-06T04:33:44.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:44 smithi045 ceph-mon[32299]: pgmap v499: 129 pgs: 129 active+clean; 28 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.1 MiB/s wr, 768 op/s 2024-06-06T04:33:44.574 INFO:tasks.workunit.client.0.smithi045.stdout:checking dirent.h usability... yes 2024-06-06T04:33:44.601 INFO:tasks.workunit.client.0.smithi045.stdout:checking dirent.h presence... yes 2024-06-06T04:33:44.602 INFO:tasks.workunit.client.0.smithi045.stdout:checking for dirent.h... yes 2024-06-06T04:33:44.608 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:44 smithi086 ceph-mon[36053]: pgmap v499: 129 pgs: 129 active+clean; 28 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.1 MiB/s wr, 768 op/s 2024-06-06T04:33:44.679 INFO:tasks.workunit.client.0.smithi045.stdout:checking errno.h usability... yes 2024-06-06T04:33:44.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:44 smithi154 ceph-mon[36428]: pgmap v499: 129 pgs: 129 active+clean; 28 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.1 MiB/s wr, 768 op/s 2024-06-06T04:33:44.702 INFO:tasks.workunit.client.0.smithi045.stdout:checking errno.h presence... yes 2024-06-06T04:33:44.702 INFO:tasks.workunit.client.0.smithi045.stdout:checking for errno.h... yes 2024-06-06T04:33:44.787 INFO:tasks.workunit.client.0.smithi045.stdout:checking malloc.h usability... yes 2024-06-06T04:33:44.814 INFO:tasks.workunit.client.0.smithi045.stdout:checking malloc.h presence... yes 2024-06-06T04:33:44.814 INFO:tasks.workunit.client.0.smithi045.stdout:checking for malloc.h... yes 2024-06-06T04:33:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.862+0000 7fa20a220700 1 -- 172.21.15.45:0/1699386912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa20406b810 msgr2=0x7fa2041070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.862+0000 7fa20a220700 1 --2- 172.21.15.45:0/1699386912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa20406b810 0x7fa2041070d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80099c0 tx=0x7fa1f8009cd0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.862+0000 7fa20a220700 1 -- 172.21.15.45:0/1699386912 shutdown_connections 2024-06-06T04:33:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.862+0000 7fa20a220700 1 --2- 172.21.15.45:0/1699386912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2041077b0 0x7fa20410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.862+0000 7fa20a220700 1 --2- 172.21.15.45:0/1699386912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa20406ae60 0x7fa20406b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.862+0000 7fa20a220700 1 --2- 172.21.15.45:0/1699386912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa20406b810 0x7fa2041070d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.862+0000 7fa20a220700 1 -- 172.21.15.45:0/1699386912 >> 172.21.15.45:0/1699386912 conn(0x7fa2040773e0 msgr2=0x7fa2040777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.862+0000 7fa20a220700 1 -- 172.21.15.45:0/1699386912 shutdown_connections 2024-06-06T04:33:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.862+0000 7fa20a220700 1 -- 172.21.15.45:0/1699386912 wait complete. 2024-06-06T04:33:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.863+0000 7fa20a220700 1 Processor -- start 2024-06-06T04:33:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.863+0000 7fa20a220700 1 -- start start 2024-06-06T04:33:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.863+0000 7fa20a220700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa20406ae60 0x7fa2041014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.863+0000 7fa20a220700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa20406b810 0x7fa204101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.863+0000 7fa20a220700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2041077b0 0x7fa204101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.863+0000 7fa20a220700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa204112660 con 0x7fa20406ae60 2024-06-06T04:33:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.863+0000 7fa20a220700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041124e0 con 0x7fa2041077b0 2024-06-06T04:33:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.863+0000 7fa20a220700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041127e0 con 0x7fa20406b810 2024-06-06T04:33:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.863+0000 7fa2037fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa20406ae60 0x7fa2041014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.863+0000 7fa2037fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa20406ae60 0x7fa2041014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33818/0 (socket says 172.21.15.45:33818) 2024-06-06T04:33:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.863+0000 7fa203fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2041077b0 0x7fa204101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.863+0000 7fa2037fe700 1 -- 172.21.15.45:0/719627472 learned_addr learned my addr 172.21.15.45:0/719627472 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.863+0000 7fa202ffd700 1 --2- 172.21.15.45:0/719627472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa20406b810 0x7fa204101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.864+0000 7fa2037fe700 1 -- 172.21.15.45:0/719627472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa20406b810 msgr2=0x7fa204101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.864+0000 7fa2037fe700 1 --2- 172.21.15.45:0/719627472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa20406b810 0x7fa204101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.864+0000 7fa2037fe700 1 -- 172.21.15.45:0/719627472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2041077b0 msgr2=0x7fa204101f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.864+0000 7fa2037fe700 1 --2- 172.21.15.45:0/719627472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2041077b0 0x7fa204101f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.864+0000 7fa2037fe700 1 -- 172.21.15.45:0/719627472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2041a70e0 con 0x7fa20406ae60 2024-06-06T04:33:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.864+0000 7fa203fff700 1 --2- 172.21.15.45:0/719627472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2041077b0 0x7fa204101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.864+0000 7fa2037fe700 1 --2- 172.21.15.45:0/719627472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa20406ae60 0x7fa2041014f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa1f400d950 tx=0x7fa1f400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.864+0000 7fa200ff9700 1 -- 172.21.15.45:0/719627472 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4016070 con 0x7fa20406ae60 2024-06-06T04:33:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.864+0000 7fa20a220700 1 -- 172.21.15.45:0/719627472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2041a73d0 con 0x7fa20406ae60 2024-06-06T04:33:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.864+0000 7fa200ff9700 1 -- 172.21.15.45:0/719627472 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1f400fd40 con 0x7fa20406ae60 2024-06-06T04:33:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.864+0000 7fa200ff9700 1 -- 172.21.15.45:0/719627472 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f400e3b0 con 0x7fa20406ae60 2024-06-06T04:33:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.864+0000 7fa20a220700 1 -- 172.21.15.45:0/719627472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2041a78e0 con 0x7fa20406ae60 2024-06-06T04:33:44.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.867+0000 7fa200ff9700 1 -- 172.21.15.45:0/719627472 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa1f4019070 con 0x7fa20406ae60 2024-06-06T04:33:44.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.868+0000 7fa214aa3b80 1 Processor -- start 2024-06-06T04:33:44.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.868+0000 7fa214aa3b80 1 -- start start 2024-06-06T04:33:44.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.868+0000 7fa200ff9700 1 --2- 172.21.15.45:0/719627472 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa1ec0a2c20 0x7fa1ec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.868+0000 7fa214aa3b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d38f60 0x560db0d39340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.868+0000 7fa214aa3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d39880 0x560db0d33260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.868+0000 7fa214aa3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d337a0 0x560db0d33bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.868+0000 7fa214aa3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560db0c25f60 con 0x560db0d337a0 2024-06-06T04:33:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.868+0000 7fa214aa3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560db0c84850 con 0x560db0d39880 2024-06-06T04:33:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.868+0000 7fa214aa3b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560db0bec460 con 0x560db0d38f60 2024-06-06T04:33:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.868+0000 7fa203fff700 1 --2- 172.21.15.45:0/719627472 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa1ec0a2c20 0x7fa1ec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.868+0000 7fa203fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d337a0 0x560db0d33bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.869+0000 7fa202ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d39880 0x560db0d33260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.869+0000 7fa203fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d337a0 0x560db0d33bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33826/0 (socket says 172.21.15.45:33826) 2024-06-06T04:33:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.869+0000 7fa203fff700 1 -- 172.21.15.45:0/1307515886 learned_addr learned my addr 172.21.15.45:0/1307515886 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:44.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.869+0000 7fa200ff9700 1 -- 172.21.15.45:0/719627472 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fa1f40cef20 con 0x7fa20406ae60 2024-06-06T04:33:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.869+0000 7fa2037fe700 1 --2- 172.21.15.45:0/1307515886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d38f60 0x560db0d39340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.869+0000 7fa203fff700 1 -- 172.21.15.45:0/1307515886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d38f60 msgr2=0x560db0d39340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.869+0000 7fa203fff700 1 --2- 172.21.15.45:0/1307515886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d38f60 0x560db0d39340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.869+0000 7fa203fff700 1 -- 172.21.15.45:0/1307515886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d39880 msgr2=0x560db0d33260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.869+0000 7fa203fff700 1 --2- 172.21.15.45:0/1307515886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d39880 0x560db0d33260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.869+0000 7fa203fff700 1 -- 172.21.15.45:0/1307515886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560db0bd38d0 con 0x560db0d337a0 2024-06-06T04:33:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.869+0000 7fa202ffd700 1 --2- 172.21.15.45:0/1307515886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d39880 0x560db0d33260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:44.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.869+0000 7fa203fff700 1 --2- 172.21.15.45:0/1307515886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d337a0 0x560db0d33bc0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0094c0 tx=0x7fa1fc007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c245254e8a174bf4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.869+0000 7fa1e9ffb700 1 -- 172.21.15.45:0/1307515886 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc017db0 con 0x560db0d337a0 2024-06-06T04:33:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.869+0000 7fa1e9ffb700 1 -- 172.21.15.45:0/1307515886 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1fc004bb0 con 0x560db0d337a0 2024-06-06T04:33:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.869+0000 7fa1e9ffb700 1 -- 172.21.15.45:0/1307515886 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc022400 con 0x560db0d337a0 2024-06-06T04:33:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.870+0000 7fa214aa3b80 1 -- 172.21.15.45:0/1307515886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d337a0 msgr2=0x560db0d33bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.870+0000 7fa203fff700 1 --2- 172.21.15.45:0/719627472 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa1ec0a2c20 0x7fa1ec0a50e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc021000 tx=0x7fa1fc01f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.870+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/1307515886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d337a0 0x560db0d33bc0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0094c0 tx=0x7fa1fc007e80 comp rx=0 tx=0).stop 2024-06-06T04:33:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.870+0000 7fa214aa3b80 1 -- 172.21.15.45:0/1307515886 shutdown_connections 2024-06-06T04:33:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.870+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/1307515886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d337a0 0x560db0d33bc0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.870+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/1307515886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d38f60 0x560db0d39340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.870+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/1307515886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d39880 0x560db0d33260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.870+0000 7fa214aa3b80 1 -- 172.21.15.45:0/1307515886 >> 172.21.15.45:0/1307515886 conn(0x560db0c60080 msgr2=0x560db0d2e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.870+0000 7fa214aa3b80 1 -- 172.21.15.45:0/1307515886 shutdown_connections 2024-06-06T04:33:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.870+0000 7fa214aa3b80 1 -- 172.21.15.45:0/1307515886 wait complete. 2024-06-06T04:33:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.871+0000 7fa214aa3b80 1 -- start start 2024-06-06T04:33:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.871+0000 7fa214aa3b80 1 Processor -- start 2024-06-06T04:33:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.871+0000 7fa214aa3b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d337a0 0x560db0d3f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.871+0000 7fa214aa3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d38f60 0x560db0d3c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.871+0000 7fa214aa3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d39880 0x560db0d3ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.871+0000 7fa214aa3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560db0c24fc0 con 0x560db0d39880 2024-06-06T04:33:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.871+0000 7fa214aa3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560db0c80810 con 0x560db0d38f60 2024-06-06T04:33:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.871+0000 7fa214aa3b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560db0d40260 con 0x560db0d337a0 2024-06-06T04:33:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.871+0000 7fa214aa3b80 10 client.0 _ll_register_callbacks cb 0x560db0c1a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.872+0000 7fa203fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d39880 0x560db0d3ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.872+0000 7fa203fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d39880 0x560db0d3ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33842/0 (socket says 172.21.15.45:33842) 2024-06-06T04:33:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.872+0000 7fa203fff700 1 -- 172.21.15.45:0/1680454072 learned_addr learned my addr 172.21.15.45:0/1680454072 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.872+0000 7fa202ffd700 1 --2- 172.21.15.45:0/1680454072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d38f60 0x560db0d3c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.872+0000 7fa2037fe700 1 --2- 172.21.15.45:0/1680454072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d337a0 0x560db0d3f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.872+0000 7fa203fff700 1 -- 172.21.15.45:0/1680454072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d337a0 msgr2=0x560db0d3f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.872+0000 7fa203fff700 1 --2- 172.21.15.45:0/1680454072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d337a0 0x560db0d3f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.872+0000 7fa203fff700 1 -- 172.21.15.45:0/1680454072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d38f60 msgr2=0x560db0d3c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.872+0000 7fa203fff700 1 --2- 172.21.15.45:0/1680454072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d38f60 0x560db0d3c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.872+0000 7fa203fff700 1 -- 172.21.15.45:0/1680454072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560db0c4a750 con 0x560db0d39880 2024-06-06T04:33:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.872+0000 7fa202ffd700 1 --2- 172.21.15.45:0/1680454072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d38f60 0x560db0d3c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.872+0000 7fa203fff700 1 --2- 172.21.15.45:0/1680454072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d39880 0x560db0d3ce30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009fd0 tx=0x7fa1fc01a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544ccbaf2a6ecaa7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.872+0000 7fa1e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:33:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.872+0000 7fa1e2ffd700 1 -- 172.21.15.45:0/1680454072 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc02f020 con 0x560db0d39880 2024-06-06T04:33:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.872+0000 7fa1e2ffd700 1 -- 172.21.15.45:0/1680454072 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fa1fc009580 con 0x560db0d39880 2024-06-06T04:33:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.872+0000 7fa214aa3b80 1 -- 172.21.15.45:0/1680454072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1fc004bb0 con 0x560db0d39880 2024-06-06T04:33:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.873+0000 7fa1e2ffd700 1 -- 172.21.15.45:0/1680454072 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa1fc022400 con 0x560db0d39880 2024-06-06T04:33:44.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.873+0000 7fa214aa3b80 10 client.15762 fetch_fsmap learned FSMap version 30 2024-06-06T04:33:44.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.873+0000 7fa214aa3b80 1 -- 172.21.15.45:0/1680454072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560db0d4aa10 con 0x560db0d39880 2024-06-06T04:33:44.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.873+0000 7fa1e2ffd700 1 -- 172.21.15.45:0/1680454072 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1fc004bb0 con 0x560db0d39880 2024-06-06T04:33:44.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.873+0000 7fa1e2ffd700 1 -- 172.21.15.45:0/1680454072 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fa1fc018040 con 0x560db0d39880 2024-06-06T04:33:44.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.873+0000 7fa214aa3b80 10 client.15762 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:33:44.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.873+0000 7fa214aa3b80 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:33:44.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.873+0000 7fa214aa3b80 20 client.15762 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:44.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.873+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/1680454072 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560db0d4cff0 0x560db0d4f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.873+0000 7fa214aa3b80 4 client.15762 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:33:44.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.873+0000 7fa214aa3b80 1 -- 172.21.15.45:0/1680454072 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560db0c91d50 con 0x560db0d4cff0 2024-06-06T04:33:44.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.874+0000 7fa2037fe700 1 --2- 172.21.15.45:0/1680454072 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560db0d4cff0 0x560db0d4f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.874+0000 7fa1e2ffd700 10 client.15762 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:44.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.874+0000 7fa2037fe700 1 --2- 172.21.15.45:0/1680454072 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560db0d4cff0 0x560db0d4f3e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2777a343a12167cb server_cookie=3af06fde12bfafed in_seq=0 out_seq=0 2024-06-06T04:33:44.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.875+0000 7fa1e2ffd700 1 -- 172.21.15.45:0/1680454072 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560db0c91d50 con 0x560db0d4cff0 2024-06-06T04:33:44.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.875+0000 7fa1e2ffd700 10 client.15762 handle_command_reply: tid=0 2024-06-06T04:33:44.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.875+0000 7fa214aa3b80 1 client.15762 shutdown 2024-06-06T04:33:44.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.875+0000 7fa214aa3b80 1 -- 172.21.15.45:0/1680454072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d39880 msgr2=0x560db0d3ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:44.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.875+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/1680454072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d39880 0x560db0d3ce30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009fd0 tx=0x7fa1fc01a9c0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.875+0000 7fa214aa3b80 1 -- 172.21.15.45:0/1680454072 shutdown_connections 2024-06-06T04:33:44.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.875+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/1680454072 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560db0d4cff0 0x560db0d4f3e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.875+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/1680454072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d39880 0x560db0d3ce30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.875+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/1680454072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d337a0 0x560db0d3f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.875+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/1680454072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d38f60 0x560db0d3c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.875+0000 7fa214aa3b80 1 -- 172.21.15.45:0/1680454072 >> 172.21.15.45:0/1680454072 conn(0x560db0c60080 msgr2=0x560db0d2d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:44.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.875+0000 7fa1e2ffd700 0 client.15762 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:44.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.875+0000 7fa214aa3b80 1 -- 172.21.15.45:0/1680454072 shutdown_connections 2024-06-06T04:33:44.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.875+0000 7fa214aa3b80 1 -- 172.21.15.45:0/1680454072 wait complete. 2024-06-06T04:33:44.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.876+0000 7fa214aa3b80 20 client.15762 trim_cache size 0 max 16384 2024-06-06T04:33:44.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.893+0000 7fa214aa3b80 1 Processor -- start 2024-06-06T04:33:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.893+0000 7fa214aa3b80 1 -- start start 2024-06-06T04:33:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa214aa3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d337a0 0x560db0d33260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa214aa3b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d38f60 0x560db0d3ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa214aa3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d39880 0x560db0d3a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa214aa3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041b2eb0 con 0x560db0d337a0 2024-06-06T04:33:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa214aa3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041b0710 con 0x560db0d39880 2024-06-06T04:33:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa214aa3b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041ae560 con 0x560db0d38f60 2024-06-06T04:33:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa2037fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d337a0 0x560db0d33260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa2037fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d337a0 0x560db0d33260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33854/0 (socket says 172.21.15.45:33854) 2024-06-06T04:33:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa2037fe700 1 -- 172.21.15.45:0/3601396435 learned_addr learned my addr 172.21.15.45:0/3601396435 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa203fff700 1 --2- 172.21.15.45:0/3601396435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d39880 0x560db0d3a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa202ffd700 1 --2- 172.21.15.45:0/3601396435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d38f60 0x560db0d3ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa2037fe700 1 -- 172.21.15.45:0/3601396435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d38f60 msgr2=0x560db0d3ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa2037fe700 1 --2- 172.21.15.45:0/3601396435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d38f60 0x560db0d3ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa2037fe700 1 -- 172.21.15.45:0/3601396435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d39880 msgr2=0x560db0d3a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:44.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa2037fe700 1 --2- 172.21.15.45:0/3601396435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d39880 0x560db0d3a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa2037fe700 1 -- 172.21.15.45:0/3601396435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560db0d3a8b0 con 0x560db0d337a0 2024-06-06T04:33:44.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa203fff700 1 --2- 172.21.15.45:0/3601396435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d39880 0x560db0d3a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:44.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.894+0000 7fa2037fe700 1 --2- 172.21.15.45:0/3601396435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d337a0 0x560db0d33260 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40bb010 tx=0x7fa1f40bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bf2b49aa6a792b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:44.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.895+0000 7fa1e97fa700 1 -- 172.21.15.45:0/3601396435 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4020040 con 0x560db0d337a0 2024-06-06T04:33:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.895+0000 7fa1e97fa700 1 -- 172.21.15.45:0/3601396435 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1f409ba60 con 0x560db0d337a0 2024-06-06T04:33:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.895+0000 7fa1e97fa700 1 -- 172.21.15.45:0/3601396435 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4099660 con 0x560db0d337a0 2024-06-06T04:33:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.895+0000 7fa214aa3b80 1 -- 172.21.15.45:0/3601396435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d337a0 msgr2=0x560db0d33260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.895+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/3601396435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d337a0 0x560db0d33260 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40bb010 tx=0x7fa1f40bb040 comp rx=0 tx=0).stop 2024-06-06T04:33:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.895+0000 7fa214aa3b80 1 -- 172.21.15.45:0/3601396435 shutdown_connections 2024-06-06T04:33:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.895+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/3601396435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d39880 0x560db0d3a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.895+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/3601396435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d337a0 0x560db0d33260 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.895+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/3601396435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d38f60 0x560db0d3ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.895+0000 7fa214aa3b80 1 -- 172.21.15.45:0/3601396435 >> 172.21.15.45:0/3601396435 conn(0x560db0c60080 msgr2=0x560db0d39c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.896+0000 7fa214aa3b80 1 -- 172.21.15.45:0/3601396435 shutdown_connections 2024-06-06T04:33:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.896+0000 7fa214aa3b80 1 -- 172.21.15.45:0/3601396435 wait complete. 2024-06-06T04:33:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.896+0000 7fa214aa3b80 1 -- start start 2024-06-06T04:33:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.896+0000 7fa214aa3b80 1 Processor -- start 2024-06-06T04:33:44.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.896+0000 7fa214aa3b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d337a0 0x560db0d552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.896+0000 7fa214aa3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d38f60 0x560db0d45520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa214aa3b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d39880 0x560db0d45a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa214aa3b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041aebd0 con 0x560db0d39880 2024-06-06T04:33:44.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa214aa3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041b0040 con 0x560db0d38f60 2024-06-06T04:33:44.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa214aa3b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041af2a0 con 0x560db0d337a0 2024-06-06T04:33:44.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa214aa3b80 10 client.0 _ll_register_callbacks cb 0x560db0c1a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:44.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa203fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d39880 0x560db0d45a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa202ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d38f60 0x560db0d45520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa203fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d39880 0x560db0d45a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33870/0 (socket says 172.21.15.45:33870) 2024-06-06T04:33:44.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa203fff700 1 -- 172.21.15.45:0/710777977 learned_addr learned my addr 172.21.15.45:0/710777977 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:44.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa2037fe700 1 --2- 172.21.15.45:0/710777977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d337a0 0x560db0d552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa203fff700 1 -- 172.21.15.45:0/710777977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d337a0 msgr2=0x560db0d552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:44.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa203fff700 1 --2- 172.21.15.45:0/710777977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d337a0 0x560db0d552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa203fff700 1 -- 172.21.15.45:0/710777977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d38f60 msgr2=0x560db0d45520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:44.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa203fff700 1 --2- 172.21.15.45:0/710777977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d38f60 0x560db0d45520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa203fff700 1 -- 172.21.15.45:0/710777977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560db0d53b90 con 0x560db0d39880 2024-06-06T04:33:44.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa202ffd700 1 --2- 172.21.15.45:0/710777977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d38f60 0x560db0d45520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:44.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa203fff700 1 --2- 172.21.15.45:0/710777977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d39880 0x560db0d45a60 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc019010 tx=0x7fa1fc0175f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91450bf4533ced5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:44.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa1e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:33:44.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa1e2ffd700 1 -- 172.21.15.45:0/710777977 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc042020 con 0x560db0d39880 2024-06-06T04:33:44.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa1e2ffd700 1 -- 172.21.15.45:0/710777977 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fa1fc012020 con 0x560db0d39880 2024-06-06T04:33:44.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.897+0000 7fa214aa3b80 1 -- 172.21.15.45:0/710777977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f409ba60 con 0x560db0d39880 2024-06-06T04:33:44.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.898+0000 7fa1e2ffd700 1 -- 172.21.15.45:0/710777977 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa1fc02f040 con 0x560db0d39880 2024-06-06T04:33:44.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.898+0000 7fa214aa3b80 10 client.15768 fetch_fsmap learned FSMap version 30 2024-06-06T04:33:44.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.898+0000 7fa214aa3b80 1 -- 172.21.15.45:0/710777977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560db0db6c60 con 0x560db0d39880 2024-06-06T04:33:44.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.898+0000 7fa1e2ffd700 1 -- 172.21.15.45:0/710777977 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1f409ba60 con 0x560db0d39880 2024-06-06T04:33:44.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.898+0000 7fa1e2ffd700 1 -- 172.21.15.45:0/710777977 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fa1fc02b5b0 con 0x560db0d39880 2024-06-06T04:33:44.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.898+0000 7fa214aa3b80 10 client.15768 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:33:44.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.898+0000 7fa214aa3b80 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:33:44.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.898+0000 7fa214aa3b80 20 client.15768 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:44.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.898+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/710777977 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560db0db7420 0x560db0db9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:44.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.898+0000 7fa214aa3b80 4 client.15768 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T04:33:44.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.898+0000 7fa214aa3b80 1 -- 172.21.15.45:0/710777977 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560db0c8c2d0 con 0x560db0db7420 2024-06-06T04:33:44.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.898+0000 7fa2037fe700 1 --2- 172.21.15.45:0/710777977 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560db0db7420 0x560db0db9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:44.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.899+0000 7fa2037fe700 1 --2- 172.21.15.45:0/710777977 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560db0db7420 0x560db0db9840 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=231e2bd1172544bc server_cookie=c5ea8f3f3d1e00b6 in_seq=0 out_seq=0 2024-06-06T04:33:44.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.899+0000 7fa1e2ffd700 10 client.15768 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:44.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.899+0000 7fa1e2ffd700 1 -- 172.21.15.45:0/710777977 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560db0c8c2d0 con 0x560db0db7420 2024-06-06T04:33:44.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.900+0000 7fa1e2ffd700 10 client.15768 handle_command_reply: tid=0 2024-06-06T04:33:44.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.900+0000 7fa214aa3b80 1 client.15768 shutdown 2024-06-06T04:33:44.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.900+0000 7fa214aa3b80 1 -- 172.21.15.45:0/710777977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d39880 msgr2=0x560db0d45a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:44.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.900+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/710777977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d39880 0x560db0d45a60 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc019010 tx=0x7fa1fc0175f0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.900+0000 7fa214aa3b80 1 -- 172.21.15.45:0/710777977 shutdown_connections 2024-06-06T04:33:44.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.900+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/710777977 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x560db0db7420 0x560db0db9840 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.900+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/710777977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560db0d39880 0x560db0d45a60 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.900+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/710777977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560db0d337a0 0x560db0d552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.900+0000 7fa214aa3b80 1 --2- 172.21.15.45:0/710777977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560db0d38f60 0x560db0d45520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.900+0000 7fa214aa3b80 1 -- 172.21.15.45:0/710777977 >> 172.21.15.45:0/710777977 conn(0x560db0c60080 msgr2=0x560db0d3e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:44.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.900+0000 7fa1e2ffd700 0 client.15768 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:44.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.900+0000 7fa214aa3b80 1 -- 172.21.15.45:0/710777977 shutdown_connections 2024-06-06T04:33:44.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.900+0000 7fa214aa3b80 1 -- 172.21.15.45:0/710777977 wait complete. 2024-06-06T04:33:44.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.900+0000 7fa214aa3b80 20 client.15768 trim_cache size 0 max 16384 2024-06-06T04:33:44.905 INFO:tasks.workunit.client.0.smithi045.stdout:checking uuid.h usability... no 2024-06-06T04:33:44.905 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-06-06T04:33:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.901+0000 7fa20a220700 1 -- 172.21.15.45:0/719627472 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa1ec0a2c20 msgr2=0x7fa1ec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.901+0000 7fa20a220700 1 --2- 172.21.15.45:0/719627472 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa1ec0a2c20 0x7fa1ec0a50e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc021000 tx=0x7fa1fc01f000 comp rx=0 tx=0).stop 2024-06-06T04:33:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.901+0000 7fa20a220700 1 -- 172.21.15.45:0/719627472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa20406ae60 msgr2=0x7fa2041014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.901+0000 7fa20a220700 1 --2- 172.21.15.45:0/719627472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa20406ae60 0x7fa2041014f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa1f400d950 tx=0x7fa1f400dc60 comp rx=0 tx=0).stop 2024-06-06T04:33:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.901+0000 7fa20a220700 1 -- 172.21.15.45:0/719627472 shutdown_connections 2024-06-06T04:33:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.901+0000 7fa20a220700 1 --2- 172.21.15.45:0/719627472 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa1ec0a2c20 0x7fa1ec0a50e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.901+0000 7fa20a220700 1 --2- 172.21.15.45:0/719627472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2041077b0 0x7fa204101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.901+0000 7fa20a220700 1 --2- 172.21.15.45:0/719627472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa20406ae60 0x7fa2041014f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.901+0000 7fa20a220700 1 --2- 172.21.15.45:0/719627472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa20406b810 0x7fa204101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.901+0000 7fa20a220700 1 -- 172.21.15.45:0/719627472 >> 172.21.15.45:0/719627472 conn(0x7fa2040773e0 msgr2=0x7fa204106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:44.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.901+0000 7fa20a220700 1 -- 172.21.15.45:0/719627472 shutdown_connections 2024-06-06T04:33:44.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:44.901+0000 7fa20a220700 1 -- 172.21.15.45:0/719627472 wait complete. 2024-06-06T04:33:44.961 INFO:tasks.workunit.client.0.smithi045.stdout:checking uuid.h presence... no 2024-06-06T04:33:44.961 INFO:tasks.workunit.client.0.smithi045.stdout:checking for uuid.h... no 2024-06-06T04:33:45.035 INFO:tasks.workunit.client.0.smithi045.stdout:checking uuid/uuid.h usability... yes 2024-06-06T04:33:45.062 INFO:tasks.workunit.client.0.smithi045.stdout:checking uuid/uuid.h presence... yes 2024-06-06T04:33:45.062 INFO:tasks.workunit.client.0.smithi045.stdout:checking for uuid/uuid.h... yes 2024-06-06T04:33:45.147 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/uuid.h usability... no 2024-06-06T04:33:45.191 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/uuid.h presence... no 2024-06-06T04:33:45.192 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/uuid.h... no 2024-06-06T04:33:45.288 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/file.h usability... yes 2024-06-06T04:33:45.314 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/file.h presence... yes 2024-06-06T04:33:45.314 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/file.h... yes 2024-06-06T04:33:45.424 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/fcntl.h usability... yes 2024-06-06T04:33:45.453 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/fcntl.h presence... yes 2024-06-06T04:33:45.454 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/fcntl.h... yes 2024-06-06T04:33:45.540 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/syssgi.h usability... no 2024-06-06T04:33:45.583 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/syssgi.h presence... no 2024-06-06T04:33:45.583 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/syssgi.h... no 2024-06-06T04:33:45.861 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/param.h usability... yes 2024-06-06T04:33:45.888 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/param.h presence... yes 2024-06-06T04:33:45.889 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/param.h... yes 2024-06-06T04:33:45.914 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/stat.h... (cached) yes 2024-06-06T04:33:46.001 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/statvfs.h usability... yes 2024-06-06T04:33:46.026 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/statvfs.h presence... yes 2024-06-06T04:33:46.026 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/statvfs.h... yes 2024-06-06T04:33:46.101 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/time.h usability... yes 2024-06-06T04:33:46.129 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/time.h presence... yes 2024-06-06T04:33:46.129 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/time.h... yes 2024-06-06T04:33:46.203 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/ioctl.h usability... yes 2024-06-06T04:33:46.227 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/ioctl.h presence... yes 2024-06-06T04:33:46.227 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/ioctl.h... yes 2024-06-06T04:33:46.315 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/wait.h usability... yes 2024-06-06T04:33:46.343 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/wait.h presence... yes 2024-06-06T04:33:46.343 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/wait.h... yes 2024-06-06T04:33:46.369 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/types.h... (cached) yes 2024-06-06T04:33:46.391 INFO:tasks.workunit.client.0.smithi045.stdout:checking for strings.h... (cached) yes 2024-06-06T04:33:46.459 INFO:tasks.workunit.client.0.smithi045.stdout:checking err.h usability... yes 2024-06-06T04:33:46.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:46 smithi045 ceph-mon[32299]: pgmap v500: 129 pgs: 129 active+clean; 29 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 1.1 MiB/s wr, 741 op/s 2024-06-06T04:33:46.484 INFO:tasks.workunit.client.0.smithi045.stdout:checking err.h presence... yes 2024-06-06T04:33:46.484 INFO:tasks.workunit.client.0.smithi045.stdout:checking for err.h... yes 2024-06-06T04:33:46.557 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/falloc.h usability... yes 2024-06-06T04:33:46.583 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/falloc.h presence... yes 2024-06-06T04:33:46.584 INFO:tasks.workunit.client.0.smithi045.stdout:checking for linux/falloc.h... yes 2024-06-06T04:33:46.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:46 smithi086 ceph-mon[36053]: pgmap v500: 129 pgs: 129 active+clean; 29 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 1.1 MiB/s wr, 741 op/s 2024-06-06T04:33:46.671 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-06-06T04:33:46.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:46 smithi154 ceph-mon[36428]: pgmap v500: 129 pgs: 129 active+clean; 29 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 1.1 MiB/s wr, 741 op/s 2024-06-06T04:33:46.722 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-06-06T04:33:46.722 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/fs/xfs_fsops.h... no 2024-06-06T04:33:46.792 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/fs/xfs_itable.h usability... no 2024-06-06T04:33:46.835 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/fs/xfs_itable.h presence... no 2024-06-06T04:33:46.835 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/fs/xfs_itable.h... no 2024-06-06T04:33:46.860 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-06T04:33:46.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-06T04:33:46.905 INFO:tasks.workunit.client.0.smithi045.stdout:checking xfs/platform_defs.h usability... no 2024-06-06T04:33:46.985 INFO:tasks.workunit.client.0.smithi045.stdout:checking xfs/platform_defs.h presence... no 2024-06-06T04:33:46.986 INFO:tasks.workunit.client.0.smithi045.stdout:checking for xfs/platform_defs.h... no 2024-06-06T04:33:47.064 INFO:tasks.workunit.client.0.smithi045.stdout:checking btrfs/ioctl.h usability... no 2024-06-06T04:33:47.115 INFO:tasks.workunit.client.0.smithi045.stdout:checking btrfs/ioctl.h presence... no 2024-06-06T04:33:47.115 INFO:tasks.workunit.client.0.smithi045.stdout:checking for btrfs/ioctl.h... no 2024-06-06T04:33:47.149 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:33:47.192 INFO:tasks.workunit.client.0.smithi045.stdout:checking cifs/ioctl.h usability... no 2024-06-06T04:33:47.233 INFO:tasks.workunit.client.0.smithi045.stdout:checking cifs/ioctl.h presence... no 2024-06-06T04:33:47.234 INFO:tasks.workunit.client.0.smithi045.stdout:checking for cifs/ioctl.h... no 2024-06-06T04:33:47.325 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/mman.h usability... yes 2024-06-06T04:33:47.349 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/mman.h presence... yes 2024-06-06T04:33:47.349 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/mman.h... yes 2024-06-06T04:33:47.424 INFO:tasks.workunit.client.0.smithi045.stdout:checking for xfs/xfs_log_format.h... no 2024-06-06T04:33:47.516 INFO:tasks.workunit.client.0.smithi045.stdout:checking for uuid_compare... no 2024-06-06T04:33:47.583 INFO:tasks.workunit.client.0.smithi045.stdout:checking for uuid_compare in -luuid... yes 2024-06-06T04:33:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.678+0000 7f3ca6377640 1 -- 172.21.15.45:0/2284918590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca01035c0 msgr2=0x7f3ca0109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.678+0000 7f3ca6377640 1 --2- 172.21.15.45:0/2284918590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca01035c0 0x7f3ca0109e50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009ca0 tx=0x7f3c9802a8c0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.678+0000 7f3ca6377640 1 -- 172.21.15.45:0/2284918590 shutdown_connections 2024-06-06T04:33:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.678+0000 7f3ca6377640 1 --2- 172.21.15.45:0/2284918590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca01035c0 0x7f3ca0109e50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.678+0000 7f3ca6377640 1 --2- 172.21.15.45:0/2284918590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ca0102c00 0x7f3ca0103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.678+0000 7f3ca6377640 1 --2- 172.21.15.45:0/2284918590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ca0101a10 0x7f3ca0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.678+0000 7f3ca6377640 1 -- 172.21.15.45:0/2284918590 >> 172.21.15.45:0/2284918590 conn(0x7f3ca00fcf80 msgr2=0x7f3ca00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.679+0000 7f3ca6377640 1 -- 172.21.15.45:0/2284918590 shutdown_connections 2024-06-06T04:33:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.679+0000 7f3ca6377640 1 -- 172.21.15.45:0/2284918590 wait complete. 2024-06-06T04:33:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.679+0000 7f3ca6377640 1 Processor -- start 2024-06-06T04:33:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.679+0000 7f3ca6377640 1 -- start start 2024-06-06T04:33:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.679+0000 7f3ca6377640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ca0101a10 0x7f3ca0197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3ca6377640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ca0102c00 0x7f3ca0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3ca6377640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca01035c0 0x7f3ca019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3ca6377640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca010d520 con 0x7f3ca01035c0 2024-06-06T04:33:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3ca6377640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca010d3a0 con 0x7f3ca0101a10 2024-06-06T04:33:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3ca6377640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca010d6a0 con 0x7f3ca0102c00 2024-06-06T04:33:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3ca4914640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca01035c0 0x7f3ca019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3c9ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ca0101a10 0x7f3ca0197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3c9ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ca0101a10 0x7f3ca0197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:50322/0 (socket says 172.21.15.45:50322) 2024-06-06T04:33:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3c9ffff640 1 -- 172.21.15.45:0/4026876259 learned_addr learned my addr 172.21.15.45:0/4026876259 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3ca4914640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca01035c0 0x7f3ca019efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33892/0 (socket says 172.21.15.45:33892) 2024-06-06T04:33:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3c9f7fe640 1 --2- 172.21.15.45:0/4026876259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ca0102c00 0x7f3ca0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3ca4914640 1 -- 172.21.15.45:0/4026876259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ca0102c00 msgr2=0x7f3ca0197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3ca4914640 1 --2- 172.21.15.45:0/4026876259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ca0102c00 0x7f3ca0197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3ca4914640 1 -- 172.21.15.45:0/4026876259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ca0101a10 msgr2=0x7f3ca0197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3ca4914640 1 --2- 172.21.15.45:0/4026876259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ca0101a10 0x7f3ca0197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3ca4914640 1 -- 172.21.15.45:0/4026876259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca019f630 con 0x7f3ca01035c0 2024-06-06T04:33:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3c9ffff640 1 --2- 172.21.15.45:0/4026876259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ca0101a10 0x7f3ca0197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3ca4914640 1 --2- 172.21.15.45:0/4026876259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca01035c0 0x7f3ca019efc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3c98031970 tx=0x7f3c98033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3c9d7fa640 1 -- 172.21.15.45:0/4026876259 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98004040 con 0x7f3ca01035c0 2024-06-06T04:33:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3c9d7fa640 1 -- 172.21.15.45:0/4026876259 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c98004690 con 0x7f3ca01035c0 2024-06-06T04:33:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3ca6377640 1 -- 172.21.15.45:0/4026876259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca019f8c0 con 0x7f3ca01035c0 2024-06-06T04:33:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3c9d7fa640 1 -- 172.21.15.45:0/4026876259 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9802f4c0 con 0x7f3ca01035c0 2024-06-06T04:33:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.680+0000 7f3ca6377640 1 -- 172.21.15.45:0/4026876259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca019fe10 con 0x7f3ca01035c0 2024-06-06T04:33:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.683+0000 7f3c9d7fa640 1 -- 172.21.15.45:0/4026876259 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3c980305c0 con 0x7f3ca01035c0 2024-06-06T04:33:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.683+0000 7f3ca9163740 1 Processor -- start 2024-06-06T04:33:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.683+0000 7f3ca9163740 1 -- start start 2024-06-06T04:33:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.683+0000 7f3ca9163740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba4577430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.683+0000 7f3ca9163740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba4577a00 0x555ba457c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.683+0000 7f3ca9163740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba457cb70 0x555ba457f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.683+0000 7f3ca9163740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555ba44da4f0 con 0x555ba4577010 2024-06-06T04:33:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.683+0000 7f3ca9163740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555ba4478870 con 0x555ba457cb70 2024-06-06T04:33:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.683+0000 7f3ca9163740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555ba44d64b0 con 0x555ba4577a00 2024-06-06T04:33:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.683+0000 7f3c9d7fa640 1 --2- 172.21.15.45:0/4026876259 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3c880a2c30 0x7f3c880a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.683+0000 7f3c9f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba4577430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.683+0000 7f3ca4914640 1 --2- 172.21.15.45:0/4026876259 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3c880a2c30 0x7f3c880a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.683+0000 7f3c9f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba4577430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33900/0 (socket says 172.21.15.45:33900) 2024-06-06T04:33:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.683+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/928917989 learned_addr learned my addr 172.21.15.45:0/928917989 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.684+0000 7f3c9ffff640 1 --2- 172.21.15.45:0/928917989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba4577a00 0x555ba457c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.684+0000 7f3c9d7fa640 1 -- 172.21.15.45:0/4026876259 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f3c980f0b70 con 0x7f3ca01035c0 2024-06-06T04:33:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.684+0000 7f3c9f7fe640 1 --2- 172.21.15.45:0/928917989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba457cb70 0x555ba457f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.684+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/928917989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba4577a00 msgr2=0x555ba457c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.684+0000 7f3c9f7fe640 1 --2- 172.21.15.45:0/928917989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba4577a00 0x555ba457c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.684+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/928917989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba457cb70 msgr2=0x555ba457f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.684+0000 7f3c9f7fe640 1 --2- 172.21.15.45:0/928917989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba457cb70 0x555ba457f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.684+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/928917989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ba44949b0 con 0x555ba4577010 2024-06-06T04:33:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.684+0000 7f3c9f7fe640 1 --2- 172.21.15.45:0/928917989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba4577430 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400cfa0 tx=0x7f3c9400c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5cda6eb4a07082 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.684+0000 7f3ca4914640 1 --2- 172.21.15.45:0/4026876259 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3c880a2c30 0x7f3c880a50f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3c980c56f0 tx=0x7f3c9808ce60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.684+0000 7f3c867fc640 1 -- 172.21.15.45:0/928917989 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94018030 con 0x555ba4577010 2024-06-06T04:33:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.684+0000 7f3c867fc640 1 -- 172.21.15.45:0/928917989 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c940054d0 con 0x555ba4577010 2024-06-06T04:33:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.684+0000 7f3c867fc640 1 -- 172.21.15.45:0/928917989 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94010880 con 0x555ba4577010 2024-06-06T04:33:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.685+0000 7f3ca9163740 1 -- 172.21.15.45:0/928917989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 msgr2=0x555ba4577430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.685+0000 7f3ca9163740 1 --2- 172.21.15.45:0/928917989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba4577430 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400cfa0 tx=0x7f3c9400c620 comp rx=0 tx=0).stop 2024-06-06T04:33:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.685+0000 7f3ca9163740 1 -- 172.21.15.45:0/928917989 shutdown_connections 2024-06-06T04:33:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.685+0000 7f3ca9163740 1 --2- 172.21.15.45:0/928917989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba457cb70 0x555ba457f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.685+0000 7f3ca9163740 1 --2- 172.21.15.45:0/928917989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba4577a00 0x555ba457c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.685+0000 7f3ca9163740 1 --2- 172.21.15.45:0/928917989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba4577430 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.685+0000 7f3ca9163740 1 -- 172.21.15.45:0/928917989 >> 172.21.15.45:0/928917989 conn(0x555ba4572680 msgr2=0x555ba4572a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.685+0000 7f3ca9163740 1 -- 172.21.15.45:0/928917989 shutdown_connections 2024-06-06T04:33:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.685+0000 7f3ca9163740 1 -- 172.21.15.45:0/928917989 wait complete. 2024-06-06T04:33:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.685+0000 7f3ca9163740 1 -- start start 2024-06-06T04:33:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3ca9163740 1 Processor -- start 2024-06-06T04:33:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3ca9163740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba45856e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3ca9163740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba4577a00 0x555ba4585c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3ca9163740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba457cb70 0x555ba458cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3ca9163740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555ba457fb20 con 0x555ba4577010 2024-06-06T04:33:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3ca9163740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555ba4463ae0 con 0x555ba457cb70 2024-06-06T04:33:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3ca9163740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555ba457fca0 con 0x555ba4577a00 2024-06-06T04:33:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3ca9163740 10 client.0 _ll_register_callbacks cb 0x555ba449c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3c9f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba45856e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3c9f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba45856e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33904/0 (socket says 172.21.15.45:33904) 2024-06-06T04:33:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/1321780505 learned_addr learned my addr 172.21.15.45:0/1321780505 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3c9f7fe640 1 --2- 172.21.15.45:0/1321780505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba457cb70 0x555ba458cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3c9ffff640 1 --2- 172.21.15.45:0/1321780505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba4577a00 0x555ba4585c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/1321780505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba4577a00 msgr2=0x555ba4585c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3c9f7fe640 1 --2- 172.21.15.45:0/1321780505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba4577a00 0x555ba4585c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/1321780505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba457cb70 msgr2=0x555ba458cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3c9f7fe640 1 --2- 172.21.15.45:0/1321780505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba457cb70 0x555ba458cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/1321780505 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ba41ee210 con 0x555ba4577010 2024-06-06T04:33:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3c9f7fe640 1 --2- 172.21.15.45:0/1321780505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba45856e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3c94005750 tx=0x7f3c94005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84770a99350bbd5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.686+0000 7f3c7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:33:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.687+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/1321780505 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94018070 con 0x555ba4577010 2024-06-06T04:33:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.687+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/1321780505 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f3c94004260 con 0x555ba4577010 2024-06-06T04:33:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.687+0000 7f3ca9163740 1 -- 172.21.15.45:0/1321780505 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c940054d0 con 0x555ba4577010 2024-06-06T04:33:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.687+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/1321780505 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c94032050 con 0x555ba4577010 2024-06-06T04:33:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.687+0000 7f3ca9163740 10 client.15780 fetch_fsmap learned FSMap version 30 2024-06-06T04:33:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.687+0000 7f3ca9163740 1 -- 172.21.15.45:0/1321780505 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555ba42ba620 con 0x555ba4577010 2024-06-06T04:33:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.687+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/1321780505 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c940054d0 con 0x555ba4577010 2024-06-06T04:33:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.687+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/1321780505 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f3c9402b060 con 0x555ba4577010 2024-06-06T04:33:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.687+0000 7f3ca9163740 10 client.15780 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:33:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.687+0000 7f3ca9163740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:33:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.687+0000 7f3ca9163740 20 client.15780 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.687+0000 7f3ca9163740 1 --2- 172.21.15.45:0/1321780505 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x555ba4591330 0x555ba4593720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.687+0000 7f3ca9163740 4 client.15780 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:33:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.687+0000 7f3ca9163740 1 -- 172.21.15.45:0/1321780505 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ba44edf10 con 0x555ba4591330 2024-06-06T04:33:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.688+0000 7f3c9ffff640 1 --2- 172.21.15.45:0/1321780505 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x555ba4591330 0x555ba4593720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.688+0000 7f3c9ffff640 1 --2- 172.21.15.45:0/1321780505 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x555ba4591330 0x555ba4593720 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a730ce264e2e71 server_cookie=5b3d0cb09d25a25a in_seq=0 out_seq=0 2024-06-06T04:33:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.688+0000 7f3c7f7fe640 10 client.15780 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.688+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/1321780505 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555ba44edf10 con 0x555ba4591330 2024-06-06T04:33:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.688+0000 7f3c7f7fe640 10 client.15780 handle_command_reply: tid=0 2024-06-06T04:33:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.689+0000 7f3ca9163740 1 client.15780 shutdown 2024-06-06T04:33:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.689+0000 7f3ca9163740 1 -- 172.21.15.45:0/1321780505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 msgr2=0x555ba45856e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.689+0000 7f3ca9163740 1 --2- 172.21.15.45:0/1321780505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba45856e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3c94005750 tx=0x7f3c94005d60 comp rx=0 tx=0).stop 2024-06-06T04:33:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.689+0000 7f3ca9163740 1 -- 172.21.15.45:0/1321780505 shutdown_connections 2024-06-06T04:33:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.689+0000 7f3ca9163740 1 --2- 172.21.15.45:0/1321780505 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x555ba4591330 0x555ba4593720 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.689+0000 7f3ca9163740 1 --2- 172.21.15.45:0/1321780505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba457cb70 0x555ba458cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.689+0000 7f3ca9163740 1 --2- 172.21.15.45:0/1321780505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba4577a00 0x555ba4585c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.689+0000 7f3c7f7fe640 0 client.15780 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.689+0000 7f3ca9163740 1 --2- 172.21.15.45:0/1321780505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba45856e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.689+0000 7f3ca9163740 1 -- 172.21.15.45:0/1321780505 >> 172.21.15.45:0/1321780505 conn(0x555ba4572680 msgr2=0x555ba45717e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.689+0000 7f3ca9163740 1 -- 172.21.15.45:0/1321780505 shutdown_connections 2024-06-06T04:33:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.689+0000 7f3ca9163740 1 -- 172.21.15.45:0/1321780505 wait complete. 2024-06-06T04:33:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.689+0000 7f3ca9163740 20 client.15780 trim_cache size 0 max 16384 2024-06-06T04:33:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.704+0000 7f3ca9163740 1 Processor -- start 2024-06-06T04:33:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.704+0000 7f3ca9163740 1 -- start start 2024-06-06T04:33:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.704+0000 7f3ca9163740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba45852f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.704+0000 7f3ca9163740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba4577a00 0x555ba458cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.704+0000 7f3ca9163740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba457cb70 0x555ba4585830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.704+0000 7f3ca9163740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca01aaae0 con 0x555ba4577010 2024-06-06T04:33:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.704+0000 7f3ca9163740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca01a8380 con 0x555ba457cb70 2024-06-06T04:33:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.704+0000 7f3ca9163740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca01a61d0 con 0x555ba4577a00 2024-06-06T04:33:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.704+0000 7f3c9f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba45852f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.704+0000 7f3c9f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba45852f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33918/0 (socket says 172.21.15.45:33918) 2024-06-06T04:33:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.704+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/1756656216 learned_addr learned my addr 172.21.15.45:0/1756656216 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.705+0000 7f3c9ffff640 1 --2- 172.21.15.45:0/1756656216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba4577a00 0x555ba458cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.705+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/1756656216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba4577a00 msgr2=0x555ba458cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.705+0000 7f3c9f7fe640 1 --2- 172.21.15.45:0/1756656216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba4577a00 0x555ba458cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.705+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/1756656216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba457cb70 msgr2=0x555ba4585830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.705+0000 7f3c9f7fe640 1 --2- 172.21.15.45:0/1756656216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba457cb70 0x555ba4585830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.705+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/1756656216 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ba4593d80 con 0x555ba4577010 2024-06-06T04:33:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.705+0000 7f3c9ffff640 1 --2- 172.21.15.45:0/1756656216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba4577a00 0x555ba458cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:33:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.705+0000 7f3c9f7fe640 1 --2- 172.21.15.45:0/1756656216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba45852f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400c520 tx=0x7f3c94003610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3736d6fe05dedb5b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.705+0000 7f3c85ffb640 1 -- 172.21.15.45:0/1756656216 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94021030 con 0x555ba4577010 2024-06-06T04:33:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.705+0000 7f3c85ffb640 1 -- 172.21.15.45:0/1756656216 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c94019040 con 0x555ba4577010 2024-06-06T04:33:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.705+0000 7f3c85ffb640 1 -- 172.21.15.45:0/1756656216 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94018070 con 0x555ba4577010 2024-06-06T04:33:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 -- 172.21.15.45:0/1756656216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 msgr2=0x555ba45852f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 --2- 172.21.15.45:0/1756656216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba45852f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400c520 tx=0x7f3c94003610 comp rx=0 tx=0).stop 2024-06-06T04:33:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 -- 172.21.15.45:0/1756656216 shutdown_connections 2024-06-06T04:33:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 --2- 172.21.15.45:0/1756656216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba457cb70 0x555ba4585830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 --2- 172.21.15.45:0/1756656216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba4577a00 0x555ba458cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 --2- 172.21.15.45:0/1756656216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba45852f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 -- 172.21.15.45:0/1756656216 >> 172.21.15.45:0/1756656216 conn(0x555ba4572680 msgr2=0x555ba4580700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 -- 172.21.15.45:0/1756656216 shutdown_connections 2024-06-06T04:33:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 -- 172.21.15.45:0/1756656216 wait complete. 2024-06-06T04:33:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 -- start start 2024-06-06T04:33:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 Processor -- start 2024-06-06T04:33:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba458cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba4577a00 0x555ba45897c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba457cb70 0x555ba4589df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca01a6840 con 0x555ba4577010 2024-06-06T04:33:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca01a7cb0 con 0x555ba4577a00 2024-06-06T04:33:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.706+0000 7f3ca9163740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca01a6f10 con 0x555ba457cb70 2024-06-06T04:33:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3ca9163740 10 client.0 _ll_register_callbacks cb 0x7f3c9000aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3c9f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba458cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3c9ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba4577a00 0x555ba45897c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3c9f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba458cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33920/0 (socket says 172.21.15.45:33920) 2024-06-06T04:33:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3c9f7fe640 1 -- 172.21.15.45:0/3361622028 learned_addr learned my addr 172.21.15.45:0/3361622028 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3c9ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba4577a00 0x555ba45897c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:50364/0 (socket says 172.21.15.45:50364) 2024-06-06T04:33:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3c9ffff640 1 -- 172.21.15.45:0/3361622028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba457cb70 msgr2=0x555ba4589df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3c9ffff640 1 --2- 172.21.15.45:0/3361622028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba457cb70 0x555ba4589df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3c9ffff640 1 -- 172.21.15.45:0/3361622028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 msgr2=0x555ba458cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3c9ffff640 1 --2- 172.21.15.45:0/3361622028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba458cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3c9ffff640 1 -- 172.21.15.45:0/3361622028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ba45fa080 con 0x555ba4577a00 2024-06-06T04:33:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3c9f7fe640 1 --2- 172.21.15.45:0/3361622028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba458cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3c9ffff640 1 --2- 172.21.15.45:0/3361622028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba4577a00 0x555ba45897c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x555ba4593fb0 tx=0x7f3c90010bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=130ef0e12795193d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3c7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-06-06T04:33:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/3361622028 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c90003cc0 con 0x555ba4577a00 2024-06-06T04:33:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3ca9163740 1 -- 172.21.15.45:0/3361622028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c94019040 con 0x555ba4577a00 2024-06-06T04:33:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/3361622028 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f3c9001f420 con 0x555ba4577a00 2024-06-06T04:33:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/3361622028 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c90003e60 con 0x555ba4577a00 2024-06-06T04:33:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.707+0000 7f3ca9163740 10 client.25415 fetch_fsmap learned FSMap version 30 2024-06-06T04:33:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.708+0000 7f3ca9163740 1 -- 172.21.15.45:0/3361622028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555ba45fb370 con 0x555ba4577a00 2024-06-06T04:33:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.708+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/3361622028 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c94019040 con 0x555ba4577a00 2024-06-06T04:33:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.708+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/3361622028 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f3c90024070 con 0x555ba4577a00 2024-06-06T04:33:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.708+0000 7f3ca9163740 10 client.25415 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:33:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.708+0000 7f3ca9163740 10 client.25415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-06T04:33:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.708+0000 7f3ca9163740 20 client.25415 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.708+0000 7f3ca9163740 1 --2- 172.21.15.45:0/3361622028 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x555ba45fbdd0 0x555ba45fe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.708+0000 7f3ca9163740 4 client.25415 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T04:33:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.708+0000 7f3ca9163740 1 -- 172.21.15.45:0/3361622028 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ba44e7c20 con 0x555ba45fbdd0 2024-06-06T04:33:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.708+0000 7f3c9f7fe640 1 --2- 172.21.15.45:0/3361622028 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x555ba45fbdd0 0x555ba45fe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.708+0000 7f3c9f7fe640 1 --2- 172.21.15.45:0/3361622028 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x555ba45fbdd0 0x555ba45fe1f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d78eecdaf287f5e7 server_cookie=542dbdbbbd51694a in_seq=0 out_seq=0 2024-06-06T04:33:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.708+0000 7f3c7f7fe640 10 client.25415 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.785+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/3361622028 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555ba44e7c20 con 0x555ba45fbdd0 2024-06-06T04:33:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.785+0000 7f3c7f7fe640 10 client.25415 handle_command_reply: tid=0 2024-06-06T04:33:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.785+0000 7f3ca9163740 1 client.25415 shutdown 2024-06-06T04:33:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.786+0000 7f3ca9163740 1 -- 172.21.15.45:0/3361622028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba4577a00 msgr2=0x555ba45897c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.786+0000 7f3ca9163740 1 --2- 172.21.15.45:0/3361622028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba4577a00 0x555ba45897c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x555ba4593fb0 tx=0x7f3c90010bc0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.786+0000 7f3ca9163740 1 -- 172.21.15.45:0/3361622028 shutdown_connections 2024-06-06T04:33:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.786+0000 7f3ca9163740 1 --2- 172.21.15.45:0/3361622028 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x555ba45fbdd0 0x555ba45fe1f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.786+0000 7f3c7f7fe640 0 client.25415 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.786+0000 7f3ca9163740 1 --2- 172.21.15.45:0/3361622028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ba4577a00 0x555ba45897c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.786+0000 7f3ca9163740 1 --2- 172.21.15.45:0/3361622028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555ba457cb70 0x555ba4589df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.786+0000 7f3ca9163740 1 --2- 172.21.15.45:0/3361622028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ba4577010 0x555ba458cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.786+0000 7f3ca9163740 1 -- 172.21.15.45:0/3361622028 >> 172.21.15.45:0/3361622028 conn(0x555ba4572680 msgr2=0x555ba45852f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.786+0000 7f3ca9163740 1 -- 172.21.15.45:0/3361622028 shutdown_connections 2024-06-06T04:33:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.786+0000 7f3ca9163740 1 -- 172.21.15.45:0/3361622028 wait complete. 2024-06-06T04:33:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.786+0000 7f3ca9163740 20 client.25415 trim_cache size 0 max 16384 2024-06-06T04:33:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-06-06T04:33:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-06-06T04:33:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-06-06T04:33:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.787+0000 7f3ca6377640 1 -- 172.21.15.45:0/4026876259 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3c880a2c30 msgr2=0x7f3c880a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.787+0000 7f3ca6377640 1 --2- 172.21.15.45:0/4026876259 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3c880a2c30 0x7f3c880a50f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3c980c56f0 tx=0x7f3c9808ce60 comp rx=0 tx=0).stop 2024-06-06T04:33:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.787+0000 7f3ca6377640 1 -- 172.21.15.45:0/4026876259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca01035c0 msgr2=0x7f3ca019efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.787+0000 7f3ca6377640 1 --2- 172.21.15.45:0/4026876259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca01035c0 0x7f3ca019efc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3c98031970 tx=0x7f3c98033000 comp rx=0 tx=0).stop 2024-06-06T04:33:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.787+0000 7f3ca6377640 1 -- 172.21.15.45:0/4026876259 shutdown_connections 2024-06-06T04:33:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.787+0000 7f3ca6377640 1 --2- 172.21.15.45:0/4026876259 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3c880a2c30 0x7f3c880a50f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.787+0000 7f3ca6377640 1 --2- 172.21.15.45:0/4026876259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ca01035c0 0x7f3ca019efc0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.787+0000 7f3ca6377640 1 --2- 172.21.15.45:0/4026876259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ca0102c00 0x7f3ca0197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.787+0000 7f3ca6377640 1 --2- 172.21.15.45:0/4026876259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ca0101a10 0x7f3ca0197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.787+0000 7f3ca6377640 1 -- 172.21.15.45:0/4026876259 >> 172.21.15.45:0/4026876259 conn(0x7f3ca00fcf80 msgr2=0x7f3ca0105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.787+0000 7f3ca6377640 1 -- 172.21.15.45:0/4026876259 shutdown_connections 2024-06-06T04:33:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-06-06T04:33:47.787+0000 7f3ca6377640 1 -- 172.21.15.45:0/4026876259 wait complete. 2024-06-06T04:33:47.794 INFO:tasks.workunit.client.0.smithi045.stdout:checking for xfs/xfs.h... yes 2024-06-06T04:33:47.847 INFO:tasks.workunit.client.0.smithi045.stdout:checking for xfs/libxfs.h... no 2024-06-06T04:33:47.906 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether xlog_assign_lsn is declared... no 2024-06-06T04:33:47.972 INFO:tasks.workunit.client.0.smithi045.stdout:checking for xfs/xqm.h... yes 2024-06-06T04:33:48.228 INFO:tasks.workunit.client.0.smithi045.stdout:checking xfsctl from xfs/xfs.h... ok 2024-06-06T04:33:48.310 INFO:tasks.workunit.client.0.smithi045.stdout:checking xfs/handle.h usability... yes 2024-06-06T04:33:48.336 INFO:tasks.workunit.client.0.smithi045.stdout:checking xfs/handle.h presence... yes 2024-06-06T04:33:48.336 INFO:tasks.workunit.client.0.smithi045.stdout:checking for xfs/handle.h... yes 2024-06-06T04:33:48.442 INFO:tasks.workunit.client.0.smithi045.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-06-06T04:33:48.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:48 smithi045 ceph-mon[32299]: pgmap v501: 129 pgs: 129 active+clean; 29 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 544 KiB/s wr, 614 op/s 2024-06-06T04:33:48.527 INFO:tasks.workunit.client.0.smithi045.stdout:checking attr/attributes.h usability... yes 2024-06-06T04:33:48.553 INFO:tasks.workunit.client.0.smithi045.stdout:checking attr/attributes.h presence... yes 2024-06-06T04:33:48.553 INFO:tasks.workunit.client.0.smithi045.stdout:checking for attr/attributes.h... yes 2024-06-06T04:33:48.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:48 smithi086 ceph-mon[36053]: pgmap v501: 129 pgs: 129 active+clean; 29 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 544 KiB/s wr, 614 op/s 2024-06-06T04:33:48.659 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/attributes.h usability... no 2024-06-06T04:33:48.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:48 smithi154 ceph-mon[36428]: pgmap v501: 129 pgs: 129 active+clean; 29 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 544 KiB/s wr, 614 op/s 2024-06-06T04:33:48.703 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/attributes.h presence... no 2024-06-06T04:33:48.703 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/attributes.h... no 2024-06-06T04:33:48.771 INFO:tasks.workunit.client.0.smithi045.stdout:checking for attr_list in -lattr... yes 2024-06-06T04:33:48.841 INFO:tasks.workunit.client.0.smithi045.stdout:checking for attr_set in -lattr... yes 2024-06-06T04:33:48.906 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/acl.h usability... yes 2024-06-06T04:33:48.940 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/acl.h presence... yes 2024-06-06T04:33:48.941 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/acl.h... yes 2024-06-06T04:33:49.023 INFO:tasks.workunit.client.0.smithi045.stdout:checking acl/libacl.h usability... yes 2024-06-06T04:33:49.048 INFO:tasks.workunit.client.0.smithi045.stdout:checking acl/libacl.h presence... yes 2024-06-06T04:33:49.048 INFO:tasks.workunit.client.0.smithi045.stdout:checking for acl/libacl.h... yes 2024-06-06T04:33:49.120 INFO:tasks.workunit.client.0.smithi045.stdout:checking for acl_init in -lacl... yes 2024-06-06T04:33:49.181 INFO:tasks.workunit.client.0.smithi045.stdout:checking gdbm-ndbm.h usability... no 2024-06-06T04:33:49.225 INFO:tasks.workunit.client.0.smithi045.stdout:checking gdbm-ndbm.h presence... no 2024-06-06T04:33:49.226 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gdbm-ndbm.h... no 2024-06-06T04:33:49.286 INFO:tasks.workunit.client.0.smithi045.stdout:checking gdbm/ndbm.h usability... no 2024-06-06T04:33:49.335 INFO:tasks.workunit.client.0.smithi045.stdout:checking gdbm/ndbm.h presence... no 2024-06-06T04:33:49.335 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gdbm/ndbm.h... no 2024-06-06T04:33:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.361+0000 7f239453d700 1 -- 172.21.15.45:0/1353418087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f238c102380 msgr2=0x7f238c105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.361+0000 7f239453d700 1 --2- 172.21.15.45:0/1353418087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f238c102380 0x7f238c105ff0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f23800099c0 tx=0x7f2380009cd0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.361+0000 7f239453d700 1 -- 172.21.15.45:0/1353418087 shutdown_connections 2024-06-06T04:33:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.361+0000 7f239453d700 1 --2- 172.21.15.45:0/1353418087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238c106720 0x7f238c10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.361+0000 7f239453d700 1 --2- 172.21.15.45:0/1353418087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f238c1019d0 0x7f238c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.361+0000 7f239453d700 1 --2- 172.21.15.45:0/1353418087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f238c102380 0x7f238c105ff0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.361+0000 7f239453d700 1 -- 172.21.15.45:0/1353418087 >> 172.21.15.45:0/1353418087 conn(0x7f238c078340 msgr2=0x7f238c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.361+0000 7f239453d700 1 -- 172.21.15.45:0/1353418087 shutdown_connections 2024-06-06T04:33:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.361+0000 7f239453d700 1 -- 172.21.15.45:0/1353418087 wait complete. 2024-06-06T04:33:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.362+0000 7f239453d700 1 Processor -- start 2024-06-06T04:33:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.362+0000 7f239453d700 1 -- start start 2024-06-06T04:33:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.362+0000 7f239453d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238c1019d0 0x7f238c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.362+0000 7f239453d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f238c102380 0x7f238c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f239453d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f238c106720 0x7f238c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f239453d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f238c111540 con 0x7f238c102380 2024-06-06T04:33:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f239453d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f238c1113c0 con 0x7f238c1019d0 2024-06-06T04:33:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f239453d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f238c1116c0 con 0x7f238c106720 2024-06-06T04:33:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f23922d9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238c1019d0 0x7f238c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f23922d9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238c1019d0 0x7f238c19ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:59892/0 (socket says 172.21.15.45:59892) 2024-06-06T04:33:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f23922d9700 1 -- 172.21.15.45:0/2391751337 learned_addr learned my addr 172.21.15.45:0/2391751337 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f2391ad8700 1 --2- 172.21.15.45:0/2391751337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f238c102380 0x7f238c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f2392ada700 1 --2- 172.21.15.45:0/2391751337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f238c106720 0x7f238c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f23922d9700 1 -- 172.21.15.45:0/2391751337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f238c106720 msgr2=0x7f238c19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f23922d9700 1 --2- 172.21.15.45:0/2391751337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f238c106720 0x7f238c19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f23922d9700 1 -- 172.21.15.45:0/2391751337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f238c102380 msgr2=0x7f238c19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f23922d9700 1 --2- 172.21.15.45:0/2391751337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f238c102380 0x7f238c19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f23922d9700 1 -- 172.21.15.45:0/2391751337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238c19fdb0 con 0x7f238c1019d0 2024-06-06T04:33:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f23922d9700 1 --2- 172.21.15.45:0/2391751337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238c1019d0 0x7f238c19ada0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f237c007f10 tx=0x7f237c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:49.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f238b7fe700 1 -- 172.21.15.45:0/2391751337 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c004db0 con 0x7f238c1019d0 2024-06-06T04:33:49.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.363+0000 7f238b7fe700 1 -- 172.21.15.45:0/2391751337 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f237c003a00 con 0x7f238c1019d0 2024-06-06T04:33:49.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.364+0000 7f239453d700 1 -- 172.21.15.45:0/2391751337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238c1a00a0 con 0x7f238c1019d0 2024-06-06T04:33:49.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.364+0000 7f238b7fe700 1 -- 172.21.15.45:0/2391751337 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c0045e0 con 0x7f238c1019d0 2024-06-06T04:33:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.364+0000 7f239453d700 1 -- 172.21.15.45:0/2391751337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238c1a7920 con 0x7f238c1019d0 2024-06-06T04:33:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.367+0000 7f238b7fe700 1 -- 172.21.15.45:0/2391751337 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f237c01c020 con 0x7f238c1019d0 2024-06-06T04:33:49.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.367+0000 7f238b7fe700 1 --2- 172.21.15.45:0/2391751337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f23740a2c20 0x7f23740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.368+0000 7f238b7fe700 1 -- 172.21.15.45:0/2391751337 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f237c0cec20 con 0x7f238c1019d0 2024-06-06T04:33:49.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.368+0000 7f2391ad8700 1 --2- 172.21.15.45:0/2391751337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f23740a2c20 0x7f23740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.368+0000 7f2391ad8700 1 --2- 172.21.15.45:0/2391751337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f23740a2c20 0x7f23740a50e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2380009990 tx=0x7f2380005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:49.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.368+0000 7f239edc0b80 1 Processor -- start 2024-06-06T04:33:49.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f239edc0b80 1 -- start start 2024-06-06T04:33:49.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f239edc0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9afe0 0x55fb2aa9b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f239edc0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa9b900 0x55fb2aa952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f239edc0b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa95820 0x55fb2aa95c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f239edc0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fb2a987fe0 con 0x55fb2aa9afe0 2024-06-06T04:33:49.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f239edc0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fb2a9e68d0 con 0x55fb2aa9b900 2024-06-06T04:33:49.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f239edc0b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fb2a94e4e0 con 0x55fb2aa95820 2024-06-06T04:33:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f23922d9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9afe0 0x55fb2aa9b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f2391ad8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa9b900 0x55fb2aa952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f23922d9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9afe0 0x55fb2aa9b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43082/0 (socket says 172.21.15.45:43082) 2024-06-06T04:33:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f23922d9700 1 -- 172.21.15.45:0/1668274441 learned_addr learned my addr 172.21.15.45:0/1668274441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f2392ada700 1 --2- 172.21.15.45:0/1668274441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa95820 0x55fb2aa95c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f23922d9700 1 -- 172.21.15.45:0/1668274441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa95820 msgr2=0x55fb2aa95c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f23922d9700 1 --2- 172.21.15.45:0/1668274441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa95820 0x55fb2aa95c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f23922d9700 1 -- 172.21.15.45:0/1668274441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa9b900 msgr2=0x55fb2aa952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f23922d9700 1 --2- 172.21.15.45:0/1668274441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa9b900 0x55fb2aa952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f23922d9700 1 -- 172.21.15.45:0/1668274441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb2a935950 con 0x55fb2aa9afe0 2024-06-06T04:33:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.369+0000 7f2391ad8700 1 --2- 172.21.15.45:0/1668274441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa9b900 0x55fb2aa952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.370+0000 7f23922d9700 1 --2- 172.21.15.45:0/1668274441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9afe0 0x55fb2aa9b3c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f237c097520 tx=0x7f237c0094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c9d78477a3de62 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.370+0000 7f2388ff9700 1 -- 172.21.15.45:0/1668274441 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c019030 con 0x55fb2aa9afe0 2024-06-06T04:33:49.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.370+0000 7f2388ff9700 1 -- 172.21.15.45:0/1668274441 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f237c09f470 con 0x55fb2aa9afe0 2024-06-06T04:33:49.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.370+0000 7f2388ff9700 1 -- 172.21.15.45:0/1668274441 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c09f120 con 0x55fb2aa9afe0 2024-06-06T04:33:49.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.371+0000 7f239edc0b80 1 -- 172.21.15.45:0/1668274441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9afe0 msgr2=0x55fb2aa9b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:49.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.371+0000 7f239edc0b80 1 --2- 172.21.15.45:0/1668274441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9afe0 0x55fb2aa9b3c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f237c097520 tx=0x7f237c0094f0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.371+0000 7f239edc0b80 1 -- 172.21.15.45:0/1668274441 shutdown_connections 2024-06-06T04:33:49.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.371+0000 7f239edc0b80 1 --2- 172.21.15.45:0/1668274441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa95820 0x55fb2aa95c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.371+0000 7f239edc0b80 1 --2- 172.21.15.45:0/1668274441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9afe0 0x55fb2aa9b3c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.371+0000 7f239edc0b80 1 --2- 172.21.15.45:0/1668274441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa9b900 0x55fb2aa952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.371+0000 7f239edc0b80 1 -- 172.21.15.45:0/1668274441 >> 172.21.15.45:0/1668274441 conn(0x55fb2a9c2100 msgr2=0x55fb2aa902a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.371+0000 7f239edc0b80 1 -- 172.21.15.45:0/1668274441 shutdown_connections 2024-06-06T04:33:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.371+0000 7f239edc0b80 1 -- 172.21.15.45:0/1668274441 wait complete. 2024-06-06T04:33:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.371+0000 7f239edc0b80 1 -- start start 2024-06-06T04:33:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.372+0000 7f239edc0b80 1 Processor -- start 2024-06-06T04:33:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.372+0000 7f239edc0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa95820 0x55fb2aaa1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.372+0000 7f239edc0b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa9afe0 0x55fb2aa9ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.372+0000 7f239edc0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9b900 0x55fb2aa9ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.372+0000 7f239edc0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fb2a987040 con 0x55fb2aa9b900 2024-06-06T04:33:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.372+0000 7f239edc0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fb2a9e2890 con 0x55fb2aa95820 2024-06-06T04:33:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.372+0000 7f239edc0b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fb2aaa22e0 con 0x55fb2aa9afe0 2024-06-06T04:33:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.372+0000 7f239edc0b80 10 client.0 _ll_register_callbacks cb 0x55fb2a97c630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.372+0000 7f23922d9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa95820 0x55fb2aaa1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.372+0000 7f2392ada700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9b900 0x55fb2aa9ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.372+0000 7f23922d9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa95820 0x55fb2aaa1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:59912/0 (socket says 172.21.15.45:59912) 2024-06-06T04:33:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.372+0000 7f23922d9700 1 -- 172.21.15.45:0/2126724593 learned_addr learned my addr 172.21.15.45:0/2126724593 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.372+0000 7f2391ad8700 1 --2- 172.21.15.45:0/2126724593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa9afe0 0x55fb2aa9ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.372+0000 7f2391ad8700 1 -- 172.21.15.45:0/2126724593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa95820 msgr2=0x55fb2aaa1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.372+0000 7f2391ad8700 1 --2- 172.21.15.45:0/2126724593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa95820 0x55fb2aaa1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.373+0000 7f2391ad8700 1 -- 172.21.15.45:0/2126724593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9b900 msgr2=0x55fb2aa9ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.373+0000 7f2391ad8700 1 --2- 172.21.15.45:0/2126724593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9b900 0x55fb2aa9ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.373+0000 7f2391ad8700 1 -- 172.21.15.45:0/2126724593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb2a9ac7d0 con 0x55fb2aa9afe0 2024-06-06T04:33:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.373+0000 7f2392ada700 1 --2- 172.21.15.45:0/2126724593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9b900 0x55fb2aa9ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.373+0000 7f2391ad8700 1 --2- 172.21.15.45:0/2126724593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa9afe0 0x55fb2aa9ea30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f238000b580 tx=0x7f2380009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a89a4b85b15ccc5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.373+0000 7f2371ffb700 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-06-06T04:33:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.373+0000 7f2371ffb700 1 -- 172.21.15.45:0/2126724593 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238000be00 con 0x55fb2aa9afe0 2024-06-06T04:33:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.373+0000 7f239edc0b80 1 -- 172.21.15.45:0/2126724593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f237c09f470 con 0x55fb2aa9afe0 2024-06-06T04:33:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.373+0000 7f2371ffb700 1 -- 172.21.15.45:0/2126724593 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f2380023460 con 0x55fb2aa9afe0 2024-06-06T04:33:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.374+0000 7f2371ffb700 1 -- 172.21.15.45:0/2126724593 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f238003d050 con 0x55fb2aa9afe0 2024-06-06T04:33:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.374+0000 7f239edc0b80 10 client.25384 fetch_fsmap learned FSMap version 30 2024-06-06T04:33:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.374+0000 7f239edc0b80 1 -- 172.21.15.45:0/2126724593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fb2aaacb60 con 0x55fb2aa9afe0 2024-06-06T04:33:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.374+0000 7f2371ffb700 1 -- 172.21.15.45:0/2126724593 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f237c09f470 con 0x55fb2aa9afe0 2024-06-06T04:33:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.374+0000 7f2371ffb700 1 -- 172.21.15.45:0/2126724593 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f2380010600 con 0x55fb2aa9afe0 2024-06-06T04:33:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.374+0000 7f239edc0b80 10 client.25384 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:33:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.374+0000 7f239edc0b80 10 client.25384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:33:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.375+0000 7f239edc0b80 20 client.25384 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.375+0000 7f239edc0b80 1 --2- 172.21.15.45:0/2126724593 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55fb2aaaf190 0x55fb2aab1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.375+0000 7f239edc0b80 4 client.25384 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:33:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.375+0000 7f239edc0b80 1 -- 172.21.15.45:0/2126724593 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb2a9f3dd0 con 0x55fb2aaaf190 2024-06-06T04:33:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.375+0000 7f23922d9700 1 --2- 172.21.15.45:0/2126724593 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55fb2aaaf190 0x55fb2aab1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.375+0000 7f2371ffb700 10 client.25384 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.375+0000 7f23922d9700 1 --2- 172.21.15.45:0/2126724593 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55fb2aaaf190 0x55fb2aab1580 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7392d8590d29d385 server_cookie=b8ae9768b9f549fb in_seq=0 out_seq=0 2024-06-06T04:33:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.376+0000 7f2371ffb700 1 -- 172.21.15.45:0/2126724593 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fb2a9f3dd0 con 0x55fb2aaaf190 2024-06-06T04:33:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.376+0000 7f2371ffb700 10 client.25384 handle_command_reply: tid=0 2024-06-06T04:33:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.376+0000 7f239edc0b80 1 client.25384 shutdown 2024-06-06T04:33:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.376+0000 7f239edc0b80 1 -- 172.21.15.45:0/2126724593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa9afe0 msgr2=0x55fb2aa9ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.376+0000 7f239edc0b80 1 --2- 172.21.15.45:0/2126724593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa9afe0 0x55fb2aa9ea30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f238000b580 tx=0x7f2380009cd0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.376+0000 7f239edc0b80 1 -- 172.21.15.45:0/2126724593 shutdown_connections 2024-06-06T04:33:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.376+0000 7f239edc0b80 1 --2- 172.21.15.45:0/2126724593 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55fb2aaaf190 0x55fb2aab1580 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.376+0000 7f239edc0b80 1 --2- 172.21.15.45:0/2126724593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9b900 0x55fb2aa9ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.376+0000 7f239edc0b80 1 --2- 172.21.15.45:0/2126724593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa95820 0x55fb2aaa1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.376+0000 7f239edc0b80 1 --2- 172.21.15.45:0/2126724593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa9afe0 0x55fb2aa9ea30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.376+0000 7f2371ffb700 0 client.25384 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.376+0000 7f239edc0b80 1 -- 172.21.15.45:0/2126724593 >> 172.21.15.45:0/2126724593 conn(0x55fb2a9c2100 msgr2=0x55fb2aa8f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.376+0000 7f239edc0b80 1 -- 172.21.15.45:0/2126724593 shutdown_connections 2024-06-06T04:33:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.376+0000 7f239edc0b80 1 -- 172.21.15.45:0/2126724593 wait complete. 2024-06-06T04:33:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.376+0000 7f239edc0b80 20 client.25384 trim_cache size 0 max 16384 2024-06-06T04:33:49.393 INFO:tasks.workunit.client.0.smithi045.stdout:checking ndbm.h usability... no 2024-06-06T04:33:49.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.400+0000 7f239edc0b80 1 Processor -- start 2024-06-06T04:33:49.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.400+0000 7f239edc0b80 1 -- start start 2024-06-06T04:33:49.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.400+0000 7f239edc0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa95820 0x55fb2aaab400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.400+0000 7f239edc0b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa9afe0 0x55fb2aa9edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.400+0000 7f239edc0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa9b900 0x55fb2aaa1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.400+0000 7f239edc0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f238c1b2ec0 con 0x55fb2aa95820 2024-06-06T04:33:49.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.400+0000 7f239edc0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f238c1b0720 con 0x55fb2aa9b900 2024-06-06T04:33:49.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.400+0000 7f239edc0b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f238c1ae570 con 0x55fb2aa9afe0 2024-06-06T04:33:49.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.401+0000 7f23922d9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa95820 0x55fb2aaab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.401+0000 7f23922d9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa95820 0x55fb2aaab400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43096/0 (socket says 172.21.15.45:43096) 2024-06-06T04:33:49.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.401+0000 7f23922d9700 1 -- 172.21.15.45:0/3726724652 learned_addr learned my addr 172.21.15.45:0/3726724652 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:49.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.401+0000 7f2392ada700 1 --2- 172.21.15.45:0/3726724652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa9b900 0x55fb2aaa1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.401+0000 7f2391ad8700 1 --2- 172.21.15.45:0/3726724652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa9afe0 0x55fb2aa9edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.401+0000 7f23922d9700 1 -- 172.21.15.45:0/3726724652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa9afe0 msgr2=0x55fb2aa9edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.401+0000 7f23922d9700 1 --2- 172.21.15.45:0/3726724652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa9afe0 0x55fb2aa9edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.401+0000 7f23922d9700 1 -- 172.21.15.45:0/3726724652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa9b900 msgr2=0x55fb2aaa1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.401+0000 7f23922d9700 1 --2- 172.21.15.45:0/3726724652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa9b900 0x55fb2aaa1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.401+0000 7f23922d9700 1 -- 172.21.15.45:0/3726724652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb2aa8c890 con 0x55fb2aa95820 2024-06-06T04:33:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.401+0000 7f23922d9700 1 --2- 172.21.15.45:0/3726724652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa95820 0x55fb2aaab400 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f237c09adc0 tx=0x7f237c0ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9cd95e9494ed03 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.401+0000 7f2373fff700 1 -- 172.21.15.45:0/3726724652 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c019040 con 0x55fb2aa95820 2024-06-06T04:33:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.401+0000 7f2373fff700 1 -- 172.21.15.45:0/3726724652 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f237c004780 con 0x55fb2aa95820 2024-06-06T04:33:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.401+0000 7f2373fff700 1 -- 172.21.15.45:0/3726724652 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c003bb0 con 0x55fb2aa95820 2024-06-06T04:33:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.402+0000 7f239edc0b80 1 -- 172.21.15.45:0/3726724652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa95820 msgr2=0x55fb2aaab400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.402+0000 7f239edc0b80 1 --2- 172.21.15.45:0/3726724652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa95820 0x55fb2aaab400 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f237c09adc0 tx=0x7f237c0ce040 comp rx=0 tx=0).stop 2024-06-06T04:33:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.402+0000 7f239edc0b80 1 -- 172.21.15.45:0/3726724652 shutdown_connections 2024-06-06T04:33:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.402+0000 7f239edc0b80 1 --2- 172.21.15.45:0/3726724652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa9b900 0x55fb2aaa1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.402+0000 7f239edc0b80 1 --2- 172.21.15.45:0/3726724652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa95820 0x55fb2aaab400 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.402+0000 7f239edc0b80 1 --2- 172.21.15.45:0/3726724652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa9afe0 0x55fb2aa9edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.402+0000 7f239edc0b80 1 -- 172.21.15.45:0/3726724652 >> 172.21.15.45:0/3726724652 conn(0x55fb2a9c2100 msgr2=0x55fb2aa9a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.402+0000 7f239edc0b80 1 -- 172.21.15.45:0/3726724652 shutdown_connections 2024-06-06T04:33:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.402+0000 7f239edc0b80 1 -- 172.21.15.45:0/3726724652 wait complete. 2024-06-06T04:33:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.403+0000 7f239edc0b80 1 -- start start 2024-06-06T04:33:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.403+0000 7f239edc0b80 1 Processor -- start 2024-06-06T04:33:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.403+0000 7f239edc0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa95820 0x55fb2aaa7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.403+0000 7f239edc0b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa9afe0 0x55fb2aaa4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.403+0000 7f239edc0b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9b900 0x55fb2aaa4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.403+0000 7f239edc0b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f238c1aebe0 con 0x55fb2aa9b900 2024-06-06T04:33:49.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.403+0000 7f239edc0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f238c1b0050 con 0x55fb2aa95820 2024-06-06T04:33:49.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.403+0000 7f239edc0b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f238c1af2b0 con 0x55fb2aa9afe0 2024-06-06T04:33:49.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.403+0000 7f239edc0b80 10 client.0 _ll_register_callbacks cb 0x55fb2a97c630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:49.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.403+0000 7f2392ada700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9b900 0x55fb2aaa4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.403+0000 7f2392ada700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9b900 0x55fb2aaa4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43106/0 (socket says 172.21.15.45:43106) 2024-06-06T04:33:49.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.403+0000 7f2392ada700 1 -- 172.21.15.45:0/776939192 learned_addr learned my addr 172.21.15.45:0/776939192 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:49.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.403+0000 7f23922d9700 1 --2- 172.21.15.45:0/776939192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa95820 0x55fb2aaa7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.403+0000 7f2391ad8700 1 --2- 172.21.15.45:0/776939192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa9afe0 0x55fb2aaa4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.404+0000 7f2392ada700 1 -- 172.21.15.45:0/776939192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa9afe0 msgr2=0x55fb2aaa4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.404+0000 7f2392ada700 1 --2- 172.21.15.45:0/776939192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa9afe0 0x55fb2aaa4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.404+0000 7f2392ada700 1 -- 172.21.15.45:0/776939192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa95820 msgr2=0x55fb2aaa7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.404+0000 7f2392ada700 1 --2- 172.21.15.45:0/776939192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa95820 0x55fb2aaa7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.404+0000 7f2392ada700 1 -- 172.21.15.45:0/776939192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb2aaa5120 con 0x55fb2aa9b900 2024-06-06T04:33:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.404+0000 7f2392ada700 1 --2- 172.21.15.45:0/776939192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9b900 0x55fb2aaa4a80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2384011ca0 tx=0x7f2384009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c708ee911d0391 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.404+0000 7f2371ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:33:49.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.404+0000 7f2371ffb700 1 -- 172.21.15.45:0/776939192 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384021020 con 0x55fb2aa9b900 2024-06-06T04:33:49.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.404+0000 7f2371ffb700 1 -- 172.21.15.45:0/776939192 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f238400fd00 con 0x55fb2aa9b900 2024-06-06T04:33:49.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.404+0000 7f239edc0b80 1 -- 172.21.15.45:0/776939192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f237c004780 con 0x55fb2aa9b900 2024-06-06T04:33:49.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.404+0000 7f2371ffb700 1 -- 172.21.15.45:0/776939192 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f238402c050 con 0x55fb2aa9b900 2024-06-06T04:33:49.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.404+0000 7f239edc0b80 10 client.15804 fetch_fsmap learned FSMap version 30 2024-06-06T04:33:49.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.404+0000 7f239edc0b80 1 -- 172.21.15.45:0/776939192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fb2ab1ad30 con 0x55fb2aa9b900 2024-06-06T04:33:49.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.405+0000 7f2371ffb700 1 -- 172.21.15.45:0/776939192 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f237c004780 con 0x55fb2aa9b900 2024-06-06T04:33:49.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.405+0000 7f2371ffb700 1 -- 172.21.15.45:0/776939192 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f238401c4d0 con 0x55fb2aa9b900 2024-06-06T04:33:49.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.405+0000 7f239edc0b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:33:49.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.405+0000 7f239edc0b80 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:33:49.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.405+0000 7f239edc0b80 20 client.15804 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:49.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.405+0000 7f239edc0b80 1 --2- 172.21.15.45:0/776939192 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55fb2ab1b4d0 0x55fb2ab1d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:49.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.405+0000 7f239edc0b80 4 client.15804 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-06T04:33:49.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.405+0000 7f239edc0b80 1 -- 172.21.15.45:0/776939192 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fb2aa9dda0 con 0x55fb2ab1b4d0 2024-06-06T04:33:49.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.405+0000 7f23922d9700 1 --2- 172.21.15.45:0/776939192 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55fb2ab1b4d0 0x55fb2ab1d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:49.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.406+0000 7f23922d9700 1 --2- 172.21.15.45:0/776939192 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55fb2ab1b4d0 0x55fb2ab1d8f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f5228749875ffca server_cookie=c4328bf1943c620c in_seq=0 out_seq=0 2024-06-06T04:33:49.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.406+0000 7f2371ffb700 10 client.15804 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:49.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.406+0000 7f2371ffb700 1 -- 172.21.15.45:0/776939192 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fb2aa9dda0 con 0x55fb2ab1b4d0 2024-06-06T04:33:49.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.406+0000 7f2371ffb700 10 client.15804 handle_command_reply: tid=0 2024-06-06T04:33:49.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.407+0000 7f239edc0b80 1 client.15804 shutdown 2024-06-06T04:33:49.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.407+0000 7f239edc0b80 1 -- 172.21.15.45:0/776939192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9b900 msgr2=0x55fb2aaa4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:49.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.407+0000 7f239edc0b80 1 --2- 172.21.15.45:0/776939192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9b900 0x55fb2aaa4a80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2384011ca0 tx=0x7f2384009ed0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.407+0000 7f239edc0b80 1 -- 172.21.15.45:0/776939192 shutdown_connections 2024-06-06T04:33:49.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.407+0000 7f239edc0b80 1 --2- 172.21.15.45:0/776939192 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55fb2ab1b4d0 0x55fb2ab1d8f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.407+0000 7f239edc0b80 1 --2- 172.21.15.45:0/776939192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb2aa9b900 0x55fb2aaa4a80 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.407+0000 7f2371ffb700 0 client.15804 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:49.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.407+0000 7f239edc0b80 1 --2- 172.21.15.45:0/776939192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb2aa95820 0x55fb2aaa7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.407+0000 7f239edc0b80 1 --2- 172.21.15.45:0/776939192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb2aa9afe0 0x55fb2aaa4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.407+0000 7f239edc0b80 1 -- 172.21.15.45:0/776939192 >> 172.21.15.45:0/776939192 conn(0x55fb2a9c2100 msgr2=0x55fb2aa942d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:49.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.407+0000 7f239edc0b80 1 -- 172.21.15.45:0/776939192 shutdown_connections 2024-06-06T04:33:49.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.407+0000 7f239edc0b80 1 -- 172.21.15.45:0/776939192 wait complete. 2024-06-06T04:33:49.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.407+0000 7f239edc0b80 20 client.15804 trim_cache size 0 max 16384 2024-06-06T04:33:49.412 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:33:49.412 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-06-06T04:33:49.412 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "1b435196-96f7-41c3-b1bb-d366533cfe0c", 2024-06-06T04:33:49.413 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-06-06T04:33:49.413 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:33:49.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.408+0000 7f239453d700 1 -- 172.21.15.45:0/2391751337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f23740a2c20 msgr2=0x7f23740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:49.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.408+0000 7f239453d700 1 --2- 172.21.15.45:0/2391751337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f23740a2c20 0x7f23740a50e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2380009990 tx=0x7f2380005fb0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.408+0000 7f239453d700 1 -- 172.21.15.45:0/2391751337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238c1019d0 msgr2=0x7f238c19ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:49.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.408+0000 7f239453d700 1 --2- 172.21.15.45:0/2391751337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238c1019d0 0x7f238c19ada0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f237c007f10 tx=0x7f237c00d3b0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.408+0000 7f239453d700 1 -- 172.21.15.45:0/2391751337 shutdown_connections 2024-06-06T04:33:49.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.408+0000 7f239453d700 1 --2- 172.21.15.45:0/2391751337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f238c106720 0x7f238c19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.408+0000 7f239453d700 1 --2- 172.21.15.45:0/2391751337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f23740a2c20 0x7f23740a50e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.408+0000 7f239453d700 1 --2- 172.21.15.45:0/2391751337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238c1019d0 0x7f238c19ada0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.408+0000 7f239453d700 1 --2- 172.21.15.45:0/2391751337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f238c102380 0x7f238c19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:49.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.408+0000 7f239453d700 1 -- 172.21.15.45:0/2391751337 >> 172.21.15.45:0/2391751337 conn(0x7f238c078340 msgr2=0x7f238c06b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:49.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.408+0000 7f239453d700 1 -- 172.21.15.45:0/2391751337 shutdown_connections 2024-06-06T04:33:49.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:49.408+0000 7f239453d700 1 -- 172.21.15.45:0/2391751337 wait complete. 2024-06-06T04:33:49.445 INFO:tasks.workunit.client.0.smithi045.stdout:checking ndbm.h presence... no 2024-06-06T04:33:49.445 INFO:tasks.workunit.client.0.smithi045.stdout:checking for ndbm.h... no 2024-06-06T04:33:49.513 INFO:tasks.workunit.client.0.smithi045.stdout:checking libaio.h usability... yes 2024-06-06T04:33:49.542 INFO:tasks.workunit.client.0.smithi045.stdout:checking libaio.h presence... yes 2024-06-06T04:33:49.542 INFO:tasks.workunit.client.0.smithi045.stdout:checking for libaio.h... yes 2024-06-06T04:33:49.796 INFO:tasks.workunit.client.0.smithi045.stdout:checking for dm_make_handle in -ldm... no 2024-06-06T04:33:49.797 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-06-06T04:33:49.797 INFO:tasks.workunit.client.0.smithi045.stdout:WARNING: could not find a valid DMAPI base library. 2024-06-06T04:33:49.797 INFO:tasks.workunit.client.0.smithi045.stdout:If you want DMAPI support please install the data migration 2024-06-06T04:33:49.797 INFO:tasks.workunit.client.0.smithi045.stdout:API (dmapi) library package. Alternatively, run "make install" 2024-06-06T04:33:49.797 INFO:tasks.workunit.client.0.smithi045.stdout:from the dmapi source. 2024-06-06T04:33:49.797 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-06-06T04:33:49.877 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/fiemap.h usability... yes 2024-06-06T04:33:49.901 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/fiemap.h presence... yes 2024-06-06T04:33:49.901 INFO:tasks.workunit.client.0.smithi045.stdout:checking for linux/fiemap.h... yes 2024-06-06T04:33:50.042 INFO:tasks.workunit.client.0.smithi045.stdout:checking for fallocate... yes 2024-06-06T04:33:50.119 INFO:tasks.workunit.client.0.smithi045.stdout:checking for open_by_handle_at... yes 2024-06-06T04:33:50.185 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/prctl.h usability... yes 2024-06-06T04:33:50.212 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/prctl.h presence... yes 2024-06-06T04:33:50.212 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/prctl.h... yes 2024-06-06T04:33:50.274 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/fs.h usability... yes 2024-06-06T04:33:50.297 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/fs.h presence... yes 2024-06-06T04:33:50.298 INFO:tasks.workunit.client.0.smithi045.stdout:checking for linux/fs.h... yes 2024-06-06T04:33:50.365 INFO:tasks.workunit.client.0.smithi045.stdout:checking btrfsutil.h usability... no 2024-06-06T04:33:50.407 INFO:tasks.workunit.client.0.smithi045.stdout:checking btrfsutil.h presence... no 2024-06-06T04:33:50.408 INFO:tasks.workunit.client.0.smithi045.stdout:checking for btrfsutil.h... no 2024-06-06T04:33:50.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:50 smithi045 ceph-mon[32299]: pgmap v502: 129 pgs: 129 active+clean; 31 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 700 KiB/s wr, 579 op/s 2024-06-06T04:33:50.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:50 smithi045 ceph-mon[32299]: scrub summary: active paths [/] 2024-06-06T04:33:50.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:50 smithi045 ceph-mon[32299]: scrub summary: idle+waiting paths [/] 2024-06-06T04:33:50.537 INFO:tasks.workunit.client.0.smithi045.stdout:checking for copy_file_range... yes 2024-06-06T04:33:50.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:50 smithi086 ceph-mon[36053]: pgmap v502: 129 pgs: 129 active+clean; 31 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 700 KiB/s wr, 579 op/s 2024-06-06T04:33:50.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:50 smithi086 ceph-mon[36053]: scrub summary: active paths [/] 2024-06-06T04:33:50.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:50 smithi086 ceph-mon[36053]: scrub summary: idle+waiting paths [/] 2024-06-06T04:33:50.613 INFO:tasks.workunit.client.0.smithi045.stdout:checking for renameat2... yes 2024-06-06T04:33:50.644 INFO:tasks.workunit.client.0.smithi045.stdout:configure: creating ./config.status 2024-06-06T04:33:50.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:50 smithi154 ceph-mon[36428]: pgmap v502: 129 pgs: 129 active+clean; 31 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 700 KiB/s wr, 579 op/s 2024-06-06T04:33:50.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:50 smithi154 ceph-mon[36428]: scrub summary: active paths [/] 2024-06-06T04:33:50.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:50 smithi154 ceph-mon[36428]: scrub summary: idle+waiting paths [/] 2024-06-06T04:33:50.735 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T04:33:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:33:51.111 INFO:tasks.workunit.client.0.smithi045.stdout:config.status: creating include/builddefs 2024-06-06T04:33:51.135 INFO:tasks.workunit.client.0.smithi045.stdout:config.status: creating include/config.h 2024-06-06T04:33:51.155 INFO:tasks.workunit.client.0.smithi045.stdout:config.status: executing libtool commands 2024-06-06T04:33:51.346 INFO:tasks.workunit.client.0.smithi045.stdout:Building lib 2024-06-06T04:33:51.347 INFO:tasks.workunit.client.0.smithi045.stdout:Building include 2024-06-06T04:33:51.347 INFO:tasks.workunit.client.0.smithi045.stdout:Building tests 2024-06-06T04:33:51.370 INFO:tasks.workunit.client.0.smithi045.stdout: [LTDEP] 2024-06-06T04:33:51.451 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:51 smithi045 ceph-mon[32299]: scrub queued for path: / 2024-06-06T04:33:51.451 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:51 smithi045 ceph-mon[32299]: scrub summary: idle+waiting paths [/] 2024-06-06T04:33:51.452 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:51 smithi045 ceph-mon[32299]: scrub summary: active paths [/] 2024-06-06T04:33:51.452 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:51 smithi045 ceph-mon[32299]: scrub summary: active paths [/] 2024-06-06T04:33:51.452 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:51 smithi045 ceph-mon[32299]: scrub summary: idle+waiting paths [/] 2024-06-06T04:33:51.452 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:51 smithi045 ceph-mon[32299]: scrub summary: idle+waiting paths [/] 2024-06-06T04:33:51.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:51 smithi086 ceph-mon[36053]: scrub queued for path: / 2024-06-06T04:33:51.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:51 smithi086 ceph-mon[36053]: scrub summary: idle+waiting paths [/] 2024-06-06T04:33:51.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:51 smithi086 ceph-mon[36053]: scrub summary: active paths [/] 2024-06-06T04:33:51.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:51 smithi086 ceph-mon[36053]: scrub summary: active paths [/] 2024-06-06T04:33:51.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:51 smithi086 ceph-mon[36053]: scrub summary: idle+waiting paths [/] 2024-06-06T04:33:51.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:51 smithi086 ceph-mon[36053]: scrub summary: idle+waiting paths [/] 2024-06-06T04:33:51.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:51 smithi154 ceph-mon[36428]: scrub queued for path: / 2024-06-06T04:33:51.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:51 smithi154 ceph-mon[36428]: scrub summary: idle+waiting paths [/] 2024-06-06T04:33:51.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:51 smithi154 ceph-mon[36428]: scrub summary: active paths [/] 2024-06-06T04:33:51.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:51 smithi154 ceph-mon[36428]: scrub summary: active paths [/] 2024-06-06T04:33:51.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:51 smithi154 ceph-mon[36428]: scrub summary: idle+waiting paths [/] 2024-06-06T04:33:51.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:51 smithi154 ceph-mon[36428]: scrub summary: idle+waiting paths [/] 2024-06-06T04:33:51.756 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] dataascii.lo 2024-06-06T04:33:51.757 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] databin.lo 2024-06-06T04:33:51.757 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] datapid.lo 2024-06-06T04:33:51.757 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] file_lock.lo 2024-06-06T04:33:51.979 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] forker.lo 2024-06-06T04:33:52.002 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] pattern.lo 2024-06-06T04:33:52.067 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] open_flags.lo 2024-06-06T04:33:52.086 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] random_range.lo 2024-06-06T04:33:52.205 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] string_to_tokens.lo 2024-06-06T04:33:52.248 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] str_to_bytes.lo 2024-06-06T04:33:52.363 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] tlibio.lo 2024-06-06T04:33:52.386 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] write_log.lo 2024-06-06T04:33:52.452 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] random.lo 2024-06-06T04:33:52.469 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:52 smithi045 ceph-mon[32299]: pgmap v503: 129 pgs: 129 active+clean; 27 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 461 KiB/s wr, 226 op/s 2024-06-06T04:33:52.510 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:52 smithi086 ceph-mon[36053]: pgmap v503: 129 pgs: 129 active+clean; 27 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 461 KiB/s wr, 226 op/s 2024-06-06T04:33:52.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:52 smithi154 ceph-mon[36428]: pgmap v503: 129 pgs: 129 active+clean; 27 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 461 KiB/s wr, 226 op/s 2024-06-06T04:33:52.771 INFO:tasks.workunit.client.0.smithi045.stdout: [LD] 2024-06-06T04:33:53.016 INFO:tasks.workunit.client.0.smithi045.stdout:Building src 2024-06-06T04:33:53.016 INFO:tasks.workunit.client.0.smithi045.stdout:Building ltp 2024-06-06T04:33:53.016 INFO:tasks.workunit.client.0.smithi045.stdout:Building common 2024-06-06T04:33:53.016 INFO:tasks.workunit.client.0.smithi045.stdout:Building m4 2024-06-06T04:33:53.045 INFO:tasks.workunit.client.0.smithi045.stdout: [DEP] 2024-06-06T04:33:53.083 INFO:tasks.workunit.client.0.smithi045.stdout: [DEP] 2024-06-06T04:33:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.118+0000 7fe00452a700 1 -- 172.21.15.45:0/3493134571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc100a80 msgr2=0x7fdffc100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.118+0000 7fe00452a700 1 --2- 172.21.15.45:0/3493134571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc100a80 0x7fdffc100f00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009990 tx=0x7fdfec009ca0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.118+0000 7fe00452a700 1 -- 172.21.15.45:0/3493134571 shutdown_connections 2024-06-06T04:33:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.118+0000 7fe00452a700 1 --2- 172.21.15.45:0/3493134571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdffc101440 0x7fdffc10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.118+0000 7fe00452a700 1 --2- 172.21.15.45:0/3493134571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdffc108510 0x7fdffc1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.118+0000 7fe00452a700 1 --2- 172.21.15.45:0/3493134571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc100a80 0x7fdffc100f00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.118+0000 7fe00452a700 1 -- 172.21.15.45:0/3493134571 >> 172.21.15.45:0/3493134571 conn(0x7fdffc077a00 msgr2=0x7fdffc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.118+0000 7fe00452a700 1 -- 172.21.15.45:0/3493134571 shutdown_connections 2024-06-06T04:33:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.118+0000 7fe00452a700 1 -- 172.21.15.45:0/3493134571 wait complete. 2024-06-06T04:33:53.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.119+0000 7fe00452a700 1 Processor -- start 2024-06-06T04:33:53.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.119+0000 7fe00452a700 1 -- start start 2024-06-06T04:33:53.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe00452a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdffc100a80 0x7fdffc19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe00452a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdffc101440 0x7fdffc19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe00452a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc108510 0x7fdffc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe00452a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc112670 con 0x7fdffc108510 2024-06-06T04:33:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe00452a700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc1124f0 con 0x7fdffc101440 2024-06-06T04:33:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe00452a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc1127f0 con 0x7fdffc100a80 2024-06-06T04:33:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe001ac5700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdffc101440 0x7fdffc19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe001ac5700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdffc101440 0x7fdffc19ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:59944/0 (socket says 172.21.15.45:59944) 2024-06-06T04:33:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe002ac7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc108510 0x7fdffc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe001ac5700 1 -- 172.21.15.45:0/1507161832 learned_addr learned my addr 172.21.15.45:0/1507161832 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe0022c6700 1 --2- 172.21.15.45:0/1507161832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdffc100a80 0x7fdffc19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe001ac5700 1 -- 172.21.15.45:0/1507161832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdffc100a80 msgr2=0x7fdffc19e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe001ac5700 1 --2- 172.21.15.45:0/1507161832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdffc100a80 0x7fdffc19e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe001ac5700 1 -- 172.21.15.45:0/1507161832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc108510 msgr2=0x7fdffc198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe001ac5700 1 --2- 172.21.15.45:0/1507161832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc108510 0x7fdffc198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe001ac5700 1 -- 172.21.15.45:0/1507161832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdffc1990b0 con 0x7fdffc101440 2024-06-06T04:33:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.120+0000 7fe001ac5700 1 --2- 172.21.15.45:0/1507161832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdffc101440 0x7fdffc19ecd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdfec005910 tx=0x7fdfec0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.121+0000 7fdff37fe700 1 -- 172.21.15.45:0/1507161832 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec027070 con 0x7fdffc101440 2024-06-06T04:33:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.121+0000 7fdff37fe700 1 -- 172.21.15.45:0/1507161832 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfec022470 con 0x7fdffc101440 2024-06-06T04:33:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.121+0000 7fe00452a700 1 -- 172.21.15.45:0/1507161832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdffc199340 con 0x7fdffc101440 2024-06-06T04:33:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.121+0000 7fe00452a700 1 -- 172.21.15.45:0/1507161832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdffc101b90 con 0x7fdffc101440 2024-06-06T04:33:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.121+0000 7fdff37fe700 1 -- 172.21.15.45:0/1507161832 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec00b610 con 0x7fdffc101440 2024-06-06T04:33:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.124+0000 7fdff37fe700 1 -- 172.21.15.45:0/1507161832 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdfec02a070 con 0x7fdffc101440 2024-06-06T04:33:53.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.125+0000 7fdff37fe700 1 --2- 172.21.15.45:0/1507161832 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdfe80a2ca0 0x7fdfe80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.125+0000 7fe0022c6700 1 --2- 172.21.15.45:0/1507161832 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdfe80a2ca0 0x7fdfe80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:53.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.125+0000 7fdff37fe700 1 -- 172.21.15.45:0/1507161832 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fdfec0d00b0 con 0x7fdffc101440 2024-06-06T04:33:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.125+0000 7fe00edadb80 1 Processor -- start 2024-06-06T04:33:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.126+0000 7fe0022c6700 1 --2- 172.21.15.45:0/1507161832 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdfe80a2ca0 0x7fdfe80a5160 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdffc199a90 tx=0x7fdff8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:53.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.126+0000 7fe00edadb80 1 -- start start 2024-06-06T04:33:53.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.126+0000 7fe00edadb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181640 0x55cd42181a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.126+0000 7fe00edadb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd42181f60 0x55cd4217b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.126+0000 7fe00edadb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd4217bf20 0x55cd4217c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.126+0000 7fe00edadb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cd420cd810 con 0x55cd42181640 2024-06-06T04:33:53.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.126+0000 7fe00edadb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cd42075720 con 0x55cd42181f60 2024-06-06T04:33:53.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.126+0000 7fe00edadb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cd420c97d0 con 0x55cd4217bf20 2024-06-06T04:33:53.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.126+0000 7fe0022c6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181640 0x55cd42181a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:53.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.126+0000 7fe0022c6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181640 0x55cd42181a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43128/0 (socket says 172.21.15.45:43128) 2024-06-06T04:33:53.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.126+0000 7fe0022c6700 1 -- 172.21.15.45:0/1319465998 learned_addr learned my addr 172.21.15.45:0/1319465998 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:53.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.126+0000 7fe002ac7700 1 --2- 172.21.15.45:0/1319465998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd4217bf20 0x55cd4217c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:53.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.126+0000 7fe0022c6700 1 -- 172.21.15.45:0/1319465998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd4217bf20 msgr2=0x55cd4217c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:53.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.126+0000 7fe0022c6700 1 --2- 172.21.15.45:0/1319465998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd4217bf20 0x55cd4217c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.126+0000 7fe0022c6700 1 -- 172.21.15.45:0/1319465998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd42181f60 msgr2=0x55cd4217b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.126+0000 7fe0022c6700 1 --2- 172.21.15.45:0/1319465998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd42181f60 0x55cd4217b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.127+0000 7fe0022c6700 1 -- 172.21.15.45:0/1319465998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd4206bf50 con 0x55cd42181640 2024-06-06T04:33:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.127+0000 7fe0022c6700 1 --2- 172.21.15.45:0/1319465998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181640 0x55cd42181a20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdff801f710 tx=0x7fdff8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b941a09d222bd13f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.127+0000 7fdff0ff9700 1 -- 172.21.15.45:0/1319465998 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff801fbd0 con 0x55cd42181640 2024-06-06T04:33:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.127+0000 7fdff0ff9700 1 -- 172.21.15.45:0/1319465998 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdff801fd70 con 0x55cd42181640 2024-06-06T04:33:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.127+0000 7fdff0ff9700 1 -- 172.21.15.45:0/1319465998 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff801c640 con 0x55cd42181640 2024-06-06T04:33:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.128+0000 7fe00edadb80 1 -- 172.21.15.45:0/1319465998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181640 msgr2=0x55cd42181a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.128+0000 7fe00edadb80 1 --2- 172.21.15.45:0/1319465998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181640 0x55cd42181a20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdff801f710 tx=0x7fdff8017000 comp rx=0 tx=0).stop 2024-06-06T04:33:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.128+0000 7fe00edadb80 1 -- 172.21.15.45:0/1319465998 shutdown_connections 2024-06-06T04:33:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.128+0000 7fe00edadb80 1 --2- 172.21.15.45:0/1319465998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd4217bf20 0x55cd4217c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.128+0000 7fe00edadb80 1 --2- 172.21.15.45:0/1319465998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181640 0x55cd42181a20 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.128+0000 7fe00edadb80 1 --2- 172.21.15.45:0/1319465998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd42181f60 0x55cd4217b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.128+0000 7fe00edadb80 1 -- 172.21.15.45:0/1319465998 >> 172.21.15.45:0/1319465998 conn(0x55cd420a6cc0 msgr2=0x55cd42176850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.128+0000 7fe00edadb80 1 -- 172.21.15.45:0/1319465998 shutdown_connections 2024-06-06T04:33:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.128+0000 7fe00edadb80 1 -- 172.21.15.45:0/1319465998 wait complete. 2024-06-06T04:33:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.129+0000 7fe00edadb80 1 -- start start 2024-06-06T04:33:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.129+0000 7fe00edadb80 1 Processor -- start 2024-06-06T04:33:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.129+0000 7fe00edadb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd4217bf20 0x55cd42187f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.129+0000 7fe00edadb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181f60 0x55cd42185240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.129+0000 7fe00edadb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd421857d0 0x55cd42185c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.129+0000 7fe00edadb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cd421889d0 con 0x55cd42181f60 2024-06-06T04:33:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.129+0000 7fe00edadb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cd4206dbf0 con 0x55cd421857d0 2024-06-06T04:33:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.129+0000 7fe00edadb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cd42188b50 con 0x55cd4217bf20 2024-06-06T04:33:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.129+0000 7fe00edadb80 10 client.0 _ll_register_callbacks cb 0x55cd42067920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.129+0000 7fe001ac5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181f60 0x55cd42185240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.129+0000 7fe001ac5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181f60 0x55cd42185240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43130/0 (socket says 172.21.15.45:43130) 2024-06-06T04:33:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.129+0000 7fe001ac5700 1 -- 172.21.15.45:0/2073839049 learned_addr learned my addr 172.21.15.45:0/2073839049 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.129+0000 7fe002ac7700 1 --2- 172.21.15.45:0/2073839049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd421857d0 0x55cd42185c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.130+0000 7fe001ac5700 1 -- 172.21.15.45:0/2073839049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd4217bf20 msgr2=0x55cd42187f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T04:33:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.130+0000 7fe001ac5700 1 --2- 172.21.15.45:0/2073839049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd4217bf20 0x55cd42187f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.130+0000 7fe001ac5700 1 -- 172.21.15.45:0/2073839049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd421857d0 msgr2=0x55cd42185c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.130+0000 7fe001ac5700 1 --2- 172.21.15.45:0/2073839049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd421857d0 0x55cd42185c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.130+0000 7fe001ac5700 1 -- 172.21.15.45:0/2073839049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd4201c8d0 con 0x55cd42181f60 2024-06-06T04:33:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.130+0000 7fe002ac7700 1 --2- 172.21.15.45:0/2073839049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd421857d0 0x55cd42185c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:33:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.130+0000 7fe001ac5700 1 --2- 172.21.15.45:0/2073839049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181f60 0x55cd42185240 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0cd840 tx=0x7fdfec09c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63046c1d629bc531 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.130+0000 7fdfe5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:33:53.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.130+0000 7fdfe5ffb700 1 -- 172.21.15.45:0/2073839049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec09cce0 con 0x55cd42181f60 2024-06-06T04:33:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.130+0000 7fdfe5ffb700 1 -- 172.21.15.45:0/2073839049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fdfec0a4d30 con 0x55cd42181f60 2024-06-06T04:33:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.130+0000 7fe00edadb80 1 -- 172.21.15.45:0/2073839049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdff801fd70 con 0x55cd42181f60 2024-06-06T04:33:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.130+0000 7fdfe5ffb700 1 -- 172.21.15.45:0/2073839049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdfec09c5e0 con 0x55cd42181f60 2024-06-06T04:33:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.130+0000 7fe00edadb80 10 client.15816 fetch_fsmap learned FSMap version 30 2024-06-06T04:33:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.130+0000 7fe00edadb80 1 -- 172.21.15.45:0/2073839049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cd4219c1a0 con 0x55cd42181f60 2024-06-06T04:33:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.130+0000 7fdfe5ffb700 1 -- 172.21.15.45:0/2073839049 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdff801fd70 con 0x55cd42181f60 2024-06-06T04:33:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.131+0000 7fdfe5ffb700 1 -- 172.21.15.45:0/2073839049 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fdfec085c90 con 0x55cd42181f60 2024-06-06T04:33:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.131+0000 7fe00edadb80 10 client.15816 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:33:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.131+0000 7fe00edadb80 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:33:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.131+0000 7fe00edadb80 20 client.15816 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.131+0000 7fe00edadb80 1 --2- 172.21.15.45:0/2073839049 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55cd4219e6c0 0x55cd421a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.131+0000 7fe00edadb80 4 client.15816 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:33:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.131+0000 7fe00edadb80 1 -- 172.21.15.45:0/2073839049 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd420dad10 con 0x55cd4219e6c0 2024-06-06T04:33:53.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.131+0000 7fe0022c6700 1 --2- 172.21.15.45:0/2073839049 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55cd4219e6c0 0x55cd421a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:53.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.132+0000 7fdfe5ffb700 10 client.15816 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:53.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.132+0000 7fe0022c6700 1 --2- 172.21.15.45:0/2073839049 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55cd4219e6c0 0x55cd421a0ae0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40cf3fadc97f79ba server_cookie=d657341a94e20b82 in_seq=0 out_seq=0 2024-06-06T04:33:53.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.132+0000 7fdfe5ffb700 1 -- 172.21.15.45:0/2073839049 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cd420dad10 con 0x55cd4219e6c0 2024-06-06T04:33:53.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.132+0000 7fdfe5ffb700 10 client.15816 handle_command_reply: tid=0 2024-06-06T04:33:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.132+0000 7fe00edadb80 1 client.15816 shutdown 2024-06-06T04:33:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.132+0000 7fe00edadb80 1 -- 172.21.15.45:0/2073839049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181f60 msgr2=0x55cd42185240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.132+0000 7fe00edadb80 1 --2- 172.21.15.45:0/2073839049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181f60 0x55cd42185240 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0cd840 tx=0x7fdfec09c4e0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.132+0000 7fe00edadb80 1 -- 172.21.15.45:0/2073839049 shutdown_connections 2024-06-06T04:33:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.132+0000 7fe00edadb80 1 --2- 172.21.15.45:0/2073839049 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55cd4219e6c0 0x55cd421a0ae0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.133+0000 7fe00edadb80 1 --2- 172.21.15.45:0/2073839049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd421857d0 0x55cd42185c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.133+0000 7fe00edadb80 1 --2- 172.21.15.45:0/2073839049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd4217bf20 0x55cd42187f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.133+0000 7fe00edadb80 1 --2- 172.21.15.45:0/2073839049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181f60 0x55cd42185240 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.133+0000 7fe00edadb80 1 -- 172.21.15.45:0/2073839049 >> 172.21.15.45:0/2073839049 conn(0x55cd420a6cc0 msgr2=0x55cd421765f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.133+0000 7fdfe5ffb700 0 client.15816 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.133+0000 7fe00edadb80 1 -- 172.21.15.45:0/2073839049 shutdown_connections 2024-06-06T04:33:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.133+0000 7fe00edadb80 1 -- 172.21.15.45:0/2073839049 wait complete. 2024-06-06T04:33:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.133+0000 7fe00edadb80 20 client.15816 trim_cache size 0 max 16384 2024-06-06T04:33:53.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.160+0000 7fe00edadb80 1 Processor -- start 2024-06-06T04:33:53.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.160+0000 7fe00edadb80 1 -- start start 2024-06-06T04:33:53.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.161+0000 7fe00edadb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd4217bf20 0x55cd4217b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.161+0000 7fe00edadb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181f60 0x55cd42185200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.161+0000 7fe00edadb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd421857d0 0x55cd4217f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.161+0000 7fe00edadb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc1ae7f0 con 0x55cd42181f60 2024-06-06T04:33:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.161+0000 7fe00edadb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc1ac050 con 0x55cd421857d0 2024-06-06T04:33:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.161+0000 7fe00edadb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc1a9ea0 con 0x55cd4217bf20 2024-06-06T04:33:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.161+0000 7fe001ac5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181f60 0x55cd42185200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.161+0000 7fe001ac5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181f60 0x55cd42185200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43134/0 (socket says 172.21.15.45:43134) 2024-06-06T04:33:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.161+0000 7fe001ac5700 1 -- 172.21.15.45:0/524486549 learned_addr learned my addr 172.21.15.45:0/524486549 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.161+0000 7fe0022c6700 1 --2- 172.21.15.45:0/524486549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd4217bf20 0x55cd4217b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.161+0000 7fe002ac7700 1 --2- 172.21.15.45:0/524486549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd421857d0 0x55cd4217f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.162+0000 7fe001ac5700 1 -- 172.21.15.45:0/524486549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd4217bf20 msgr2=0x55cd4217b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.162+0000 7fe001ac5700 1 --2- 172.21.15.45:0/524486549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd4217bf20 0x55cd4217b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.162+0000 7fe001ac5700 1 -- 172.21.15.45:0/524486549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd421857d0 msgr2=0x55cd4217f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.162+0000 7fe001ac5700 1 --2- 172.21.15.45:0/524486549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd421857d0 0x55cd4217f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.162+0000 7fe001ac5700 1 -- 172.21.15.45:0/524486549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd4219adc0 con 0x55cd42181f60 2024-06-06T04:33:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.162+0000 7fe002ac7700 1 --2- 172.21.15.45:0/524486549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd421857d0 0x55cd4217f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.162+0000 7fe001ac5700 1 --2- 172.21.15.45:0/524486549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181f60 0x55cd42185200 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdfec098a40 tx=0x7fdfec009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69609058dff3491 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.162+0000 7fdfe7fff700 1 -- 172.21.15.45:0/524486549 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec0a1e60 con 0x55cd42181f60 2024-06-06T04:33:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.162+0000 7fdfe7fff700 1 -- 172.21.15.45:0/524486549 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfec09cce0 con 0x55cd42181f60 2024-06-06T04:33:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.162+0000 7fdfe7fff700 1 -- 172.21.15.45:0/524486549 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec0a4f70 con 0x55cd42181f60 2024-06-06T04:33:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.163+0000 7fe00edadb80 1 -- 172.21.15.45:0/524486549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181f60 msgr2=0x55cd42185200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.163+0000 7fe00edadb80 1 --2- 172.21.15.45:0/524486549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181f60 0x55cd42185200 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdfec098a40 tx=0x7fdfec009310 comp rx=0 tx=0).stop 2024-06-06T04:33:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.163+0000 7fe00edadb80 1 -- 172.21.15.45:0/524486549 shutdown_connections 2024-06-06T04:33:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.163+0000 7fe00edadb80 1 --2- 172.21.15.45:0/524486549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd421857d0 0x55cd4217f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.163+0000 7fe00edadb80 1 --2- 172.21.15.45:0/524486549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd4217bf20 0x55cd4217b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.163+0000 7fe00edadb80 1 --2- 172.21.15.45:0/524486549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd42181f60 0x55cd42185200 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.163+0000 7fe00edadb80 1 -- 172.21.15.45:0/524486549 >> 172.21.15.45:0/524486549 conn(0x55cd420a6cc0 msgr2=0x55cd42184af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.163+0000 7fe00edadb80 1 -- 172.21.15.45:0/524486549 shutdown_connections 2024-06-06T04:33:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.163+0000 7fe00edadb80 1 -- 172.21.15.45:0/524486549 wait complete. 2024-06-06T04:33:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.163+0000 7fe00edadb80 1 -- start start 2024-06-06T04:33:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.164+0000 7fe00edadb80 1 Processor -- start 2024-06-06T04:33:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.164+0000 7fe00edadb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd4217bf20 0x55cd42196700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.164+0000 7fe00edadb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd42181f60 0x55cd42196c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.164+0000 7fe00edadb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd421857d0 0x55cd421a5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.164+0000 7fe00edadb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc1aa510 con 0x55cd421857d0 2024-06-06T04:33:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.164+0000 7fe00edadb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc1ab980 con 0x55cd4217bf20 2024-06-06T04:33:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.164+0000 7fe00edadb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc1aabe0 con 0x55cd42181f60 2024-06-06T04:33:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.164+0000 7fe00edadb80 10 client.0 _ll_register_callbacks cb 0x55cd42067920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:33:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.164+0000 7fe002ac7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd421857d0 0x55cd421a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.164+0000 7fe002ac7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd421857d0 0x55cd421a5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43150/0 (socket says 172.21.15.45:43150) 2024-06-06T04:33:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.164+0000 7fe002ac7700 1 -- 172.21.15.45:0/3339055382 learned_addr learned my addr 172.21.15.45:0/3339055382 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:33:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.164+0000 7fe0022c6700 1 --2- 172.21.15.45:0/3339055382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd4217bf20 0x55cd42196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.164+0000 7fe001ac5700 1 --2- 172.21.15.45:0/3339055382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd42181f60 0x55cd42196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.165+0000 7fe002ac7700 1 -- 172.21.15.45:0/3339055382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd42181f60 msgr2=0x55cd42196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.165+0000 7fe002ac7700 1 --2- 172.21.15.45:0/3339055382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd42181f60 0x55cd42196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.165+0000 7fe002ac7700 1 -- 172.21.15.45:0/3339055382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd4217bf20 msgr2=0x55cd42196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:33:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.165+0000 7fe002ac7700 1 --2- 172.21.15.45:0/3339055382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd4217bf20 0x55cd42196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.165+0000 7fe002ac7700 1 -- 172.21.15.45:0/3339055382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd421a5b50 con 0x55cd421857d0 2024-06-06T04:33:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.165+0000 7fe0022c6700 1 --2- 172.21.15.45:0/3339055382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd4217bf20 0x55cd42196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:33:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.165+0000 7fe002ac7700 1 --2- 172.21.15.45:0/3339055382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd421857d0 0x55cd421a5560 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdff4004660 tx=0x7fdff4004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90333a10658799ba server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:33:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.165+0000 7fdfe5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:33:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.165+0000 7fdfe5ffb700 1 -- 172.21.15.45:0/3339055382 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4004bb0 con 0x55cd421857d0 2024-06-06T04:33:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.165+0000 7fe00edadb80 1 -- 172.21.15.45:0/3339055382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfec09cce0 con 0x55cd421857d0 2024-06-06T04:33:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.165+0000 7fdfe5ffb700 1 -- 172.21.15.45:0/3339055382 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fdff4004d50 con 0x55cd421857d0 2024-06-06T04:33:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.165+0000 7fdfe5ffb700 1 -- 172.21.15.45:0/3339055382 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdff4029050 con 0x55cd421857d0 2024-06-06T04:33:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.165+0000 7fe00edadb80 10 client.15822 fetch_fsmap learned FSMap version 30 2024-06-06T04:33:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.165+0000 7fe00edadb80 1 -- 172.21.15.45:0/3339055382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cd42208ce0 con 0x55cd421857d0 2024-06-06T04:33:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.165+0000 7fdfe5ffb700 1 -- 172.21.15.45:0/3339055382 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfec09cce0 con 0x55cd421857d0 2024-06-06T04:33:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.166+0000 7fdfe5ffb700 1 -- 172.21.15.45:0/3339055382 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fdff40055d0 con 0x55cd421857d0 2024-06-06T04:33:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.166+0000 7fe00edadb80 10 client.15822 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:33:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.166+0000 7fe00edadb80 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:33:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.166+0000 7fe00edadb80 20 client.15822 populate_metadata read hostname 'smithi045' 2024-06-06T04:33:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.166+0000 7fe00edadb80 1 --2- 172.21.15.45:0/3339055382 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55cd42209430 0x55cd4220b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:33:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.166+0000 7fe00edadb80 4 client.15822 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T04:33:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.166+0000 7fe00edadb80 1 -- 172.21.15.45:0/3339055382 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd420d5290 con 0x55cd42209430 2024-06-06T04:33:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.166+0000 7fe0022c6700 1 --2- 172.21.15.45:0/3339055382 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55cd42209430 0x55cd4220b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:33:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fe0022c6700 1 --2- 172.21.15.45:0/3339055382 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55cd42209430 0x55cd4220b850 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=724dcb73d99ae906 server_cookie=fb69d1e1d301255a in_seq=0 out_seq=0 2024-06-06T04:33:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fdfe5ffb700 10 client.15822 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fdfe5ffb700 1 -- 172.21.15.45:0/3339055382 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cd420d5290 con 0x55cd42209430 2024-06-06T04:33:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fdfe5ffb700 10 client.15822 handle_command_reply: tid=0 2024-06-06T04:33:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fe00edadb80 1 client.15822 shutdown 2024-06-06T04:33:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fe00edadb80 1 -- 172.21.15.45:0/3339055382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd421857d0 msgr2=0x55cd421a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:53.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fe00edadb80 1 --2- 172.21.15.45:0/3339055382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd421857d0 0x55cd421a5560 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdff4004660 tx=0x7fdff4004970 comp rx=0 tx=0).stop 2024-06-06T04:33:53.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fe00edadb80 1 -- 172.21.15.45:0/3339055382 shutdown_connections 2024-06-06T04:33:53.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fe00edadb80 1 --2- 172.21.15.45:0/3339055382 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55cd42209430 0x55cd4220b850 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fe00edadb80 1 --2- 172.21.15.45:0/3339055382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd421857d0 0x55cd421a5560 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fe00edadb80 1 --2- 172.21.15.45:0/3339055382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd4217bf20 0x55cd42196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fe00edadb80 1 --2- 172.21.15.45:0/3339055382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd42181f60 0x55cd42196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fe00edadb80 1 -- 172.21.15.45:0/3339055382 >> 172.21.15.45:0/3339055382 conn(0x55cd420a6cc0 msgr2=0x55cd4217f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:53.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fdfe5ffb700 0 client.15822 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:33:53.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fe00edadb80 1 -- 172.21.15.45:0/3339055382 shutdown_connections 2024-06-06T04:33:53.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fe00edadb80 1 -- 172.21.15.45:0/3339055382 wait complete. 2024-06-06T04:33:53.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.167+0000 7fe00edadb80 20 client.15822 trim_cache size 0 max 16384 2024-06-06T04:33:53.173 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:33:53.173 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-06T04:33:53.173 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-06-06T04:33:53.173 INFO:teuthology.orchestra.run.smithi045.stdout: "1b435196-96f7-41c3-b1bb-d366533cfe0c": { 2024-06-06T04:33:53.174 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-06-06T04:33:53.174 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "1b435196-96f7-41c3-b1bb-d366533cfe0c", 2024-06-06T04:33:53.174 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-06-06T04:33:53.174 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:33:53.174 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:33:53.174 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:33:53.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.168+0000 7fe00452a700 1 -- 172.21.15.45:0/1507161832 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdfe80a2ca0 msgr2=0x7fdfe80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:53.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.168+0000 7fe00452a700 1 --2- 172.21.15.45:0/1507161832 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdfe80a2ca0 0x7fdfe80a5160 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdffc199a90 tx=0x7fdff8009260 comp rx=0 tx=0).stop 2024-06-06T04:33:53.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.168+0000 7fe00452a700 1 -- 172.21.15.45:0/1507161832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdffc101440 msgr2=0x7fdffc19ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:33:53.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.168+0000 7fe00452a700 1 --2- 172.21.15.45:0/1507161832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdffc101440 0x7fdffc19ecd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdfec005910 tx=0x7fdfec0047b0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.168+0000 7fe00452a700 1 -- 172.21.15.45:0/1507161832 shutdown_connections 2024-06-06T04:33:53.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.168+0000 7fe00452a700 1 --2- 172.21.15.45:0/1507161832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc108510 0x7fdffc198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.168+0000 7fe00452a700 1 --2- 172.21.15.45:0/1507161832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdffc100a80 0x7fdffc19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.168+0000 7fe00452a700 1 --2- 172.21.15.45:0/1507161832 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fdfe80a2ca0 0x7fdfe80a5160 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.168+0000 7fe00452a700 1 --2- 172.21.15.45:0/1507161832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdffc101440 0x7fdffc19ecd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:33:53.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.168+0000 7fe00452a700 1 -- 172.21.15.45:0/1507161832 >> 172.21.15.45:0/1507161832 conn(0x7fdffc077a00 msgr2=0x7fdffc0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:33:53.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.168+0000 7fe00452a700 1 -- 172.21.15.45:0/1507161832 shutdown_connections 2024-06-06T04:33:53.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:33:53.168+0000 7fe00452a700 1 -- 172.21.15.45:0/1507161832 wait complete. 2024-06-06T04:33:53.951 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] doio 2024-06-06T04:33:53.951 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fsstress 2024-06-06T04:33:53.951 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fsx 2024-06-06T04:33:54.108 INFO:tasks.cephfs.filesystem:scrub status for tag:1b435196-96f7-41c3-b1bb-d366533cfe0c - {'path': '/', 'tag': '1b435196-96f7-41c3-b1bb-d366533cfe0c', 'options': 'recursive,force'} 2024-06-06T04:33:54.327 INFO:tasks.workunit.client.0.smithi045.stderr:fsstress.c: In function 'attr_list_path': 2024-06-06T04:33:54.327 INFO:tasks.workunit.client.0.smithi045.stderr:fsstress.c:859:9: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2024-06-06T04:33:54.327 INFO:tasks.workunit.client.0.smithi045.stderr: 859 | rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2024-06-06T04:33:54.327 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~ 2024-06-06T04:33:54.327 INFO:tasks.workunit.client.0.smithi045.stderr:In file included from ../src/global.h:25, 2024-06-06T04:33:54.327 INFO:tasks.workunit.client.0.smithi045.stderr: from fsstress.c:12: 2024-06-06T04:33:54.327 INFO:tasks.workunit.client.0.smithi045.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2024-06-06T04:33:54.327 INFO:tasks.workunit.client.0.smithi045.stderr: 165 | extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2024-06-06T04:33:54.327 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~ 2024-06-06T04:33:54.327 INFO:tasks.workunit.client.0.smithi045.stderr:fsstress.c: In function 'attr_remove_path': 2024-06-06T04:33:54.328 INFO:tasks.workunit.client.0.smithi045.stderr:fsstress.c:878:9: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2024-06-06T04:33:54.328 INFO:tasks.workunit.client.0.smithi045.stderr: 878 | rval = attr_remove(name->path, attrname, flags); 2024-06-06T04:33:54.328 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~ 2024-06-06T04:33:54.328 INFO:tasks.workunit.client.0.smithi045.stderr:In file included from ../src/global.h:25, 2024-06-06T04:33:54.328 INFO:tasks.workunit.client.0.smithi045.stderr: from fsstress.c:12: 2024-06-06T04:33:54.328 INFO:tasks.workunit.client.0.smithi045.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2024-06-06T04:33:54.328 INFO:tasks.workunit.client.0.smithi045.stderr: 152 | extern int attr_remove (const char *__path, const char *__attrname, 2024-06-06T04:33:54.328 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~ 2024-06-06T04:33:54.328 INFO:tasks.workunit.client.0.smithi045.stderr:fsstress.c: In function 'attr_set_path': 2024-06-06T04:33:54.328 INFO:tasks.workunit.client.0.smithi045.stderr:fsstress.c:898:9: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2024-06-06T04:33:54.328 INFO:tasks.workunit.client.0.smithi045.stderr: 898 | rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2024-06-06T04:33:54.328 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~ 2024-06-06T04:33:54.329 INFO:tasks.workunit.client.0.smithi045.stderr:In file included from ../src/global.h:25, 2024-06-06T04:33:54.329 INFO:tasks.workunit.client.0.smithi045.stderr: from fsstress.c:12: 2024-06-06T04:33:54.329 INFO:tasks.workunit.client.0.smithi045.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2024-06-06T04:33:54.329 INFO:tasks.workunit.client.0.smithi045.stderr: 139 | extern int attr_set (const char *__path, const char *__attrname, 2024-06-06T04:33:54.329 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~ 2024-06-06T04:33:54.345 INFO:tasks.workunit.client.0.smithi045.stderr:fsstress.c: In function 'allocsp_f': 2024-06-06T04:33:54.345 INFO:tasks.workunit.client.0.smithi045.stderr:fsstress.c:2047:32: error: 'XFS_IOC_ALLOCSP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-06-06T04:33:54.346 INFO:tasks.workunit.client.0.smithi045.stderr: 2047 | e = xfsctl(f.path, fd, XFS_IOC_ALLOCSP64, &fl) < 0 ? errno : 0; 2024-06-06T04:33:54.346 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~~~ 2024-06-06T04:33:54.346 INFO:tasks.workunit.client.0.smithi045.stderr: | XFS_IOC_RESVSP64 2024-06-06T04:33:54.346 INFO:tasks.workunit.client.0.smithi045.stderr:fsstress.c:2047:32: note: each undeclared identifier is reported only once for each function it appears in 2024-06-06T04:33:54.375 INFO:tasks.workunit.client.0.smithi045.stderr:fsstress.c: In function 'freesp_f': 2024-06-06T04:33:54.376 INFO:tasks.workunit.client.0.smithi045.stderr:fsstress.c:3640:32: error: 'XFS_IOC_FREESP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-06-06T04:33:54.376 INFO:tasks.workunit.client.0.smithi045.stderr: 3640 | e = xfsctl(f.path, fd, XFS_IOC_FREESP64, &fl) < 0 ? errno : 0; 2024-06-06T04:33:54.376 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~~ 2024-06-06T04:33:54.376 INFO:tasks.workunit.client.0.smithi045.stderr: | XFS_IOC_RESVSP64 2024-06-06T04:33:54.418 INFO:tasks.workunit.client.0.smithi045.stderr:gmake[3]: *** [Makefile:46: fsstress] Error 1 2024-06-06T04:33:54.418 INFO:tasks.workunit.client.0.smithi045.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-06-06T04:33:54.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:54 smithi045 ceph-mon[32299]: pgmap v504: 129 pgs: 129 active+clean; 27 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 327 KiB/s wr, 220 op/s 2024-06-06T04:33:54.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:54 smithi086 ceph-mon[36053]: pgmap v504: 129 pgs: 129 active+clean; 27 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 327 KiB/s wr, 220 op/s 2024-06-06T04:33:54.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:54 smithi154 ceph-mon[36428]: pgmap v504: 129 pgs: 129 active+clean; 27 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 327 KiB/s wr, 220 op/s 2024-06-06T04:33:54.820 INFO:tasks.workunit.client.0.smithi045.stderr:doio.c: In function 'parse_cmdline': 2024-06-06T04:33:54.821 INFO:tasks.workunit.client.0.smithi045.stderr:doio.c:3113:33: warning: '%s' directive argument is null [-Wformat-overflow=] 2024-06-06T04:33:54.821 INFO:tasks.workunit.client.0.smithi045.stderr: 3113 | fprintf(stderr, 2024-06-06T04:33:54.821 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~ 2024-06-06T04:33:54.821 INFO:tasks.workunit.client.0.smithi045.stderr: 3114 | "%s%s: Illegal -C arg (%s). Must be one of: ", 2024-06-06T04:33:54.821 INFO:tasks.workunit.client.0.smithi045.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-06T04:33:54.821 INFO:tasks.workunit.client.0.smithi045.stderr: 3115 | Prog, TagName, tok); 2024-06-06T04:33:54.821 INFO:tasks.workunit.client.0.smithi045.stderr: | ~~~~~~~~~~~~~~~~~~~ 2024-06-06T04:33:54.839 INFO:tasks.workunit.client.0.smithi045.stderr:fsx.c: In function 'main': 2024-06-06T04:33:54.839 INFO:tasks.workunit.client.0.smithi045.stderr:fsx.c:2662:33: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2024-06-06T04:33:54.839 INFO:tasks.workunit.client.0.smithi045.stderr: 2662 | strncpy(opsfile, optarg, sizeof(opsfile)); 2024-06-06T04:33:54.839 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-06T04:33:54.839 INFO:tasks.workunit.client.0.smithi045.stderr:fsx.c:2632:25: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2024-06-06T04:33:54.839 INFO:tasks.workunit.client.0.smithi045.stderr: 2632 | strncpy(dname, optarg, sizeof(dname)); 2024-06-06T04:33:54.839 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-06T04:33:54.975 INFO:tasks.workunit.client.0.smithi045.stderr:gmake[2]: *** [include/buildrules:30: ltp] Error 2 2024-06-06T04:33:54.976 INFO:tasks.workunit.client.0.smithi045.stderr:gmake[2]: *** Waiting for unfinished jobs.... 2024-06-06T04:33:56.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:56 smithi045 ceph-mon[32299]: pgmap v505: 129 pgs: 129 active+clean; 29 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 555 KiB/s wr, 462 op/s 2024-06-06T04:33:56.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:56 smithi086 ceph-mon[36053]: pgmap v505: 129 pgs: 129 active+clean; 29 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 555 KiB/s wr, 462 op/s 2024-06-06T04:33:56.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:56 smithi154 ceph-mon[36428]: pgmap v505: 129 pgs: 129 active+clean; 29 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 555 KiB/s wr, 462 op/s 2024-06-06T04:33:57.550 INFO:tasks.workunit.client.0.smithi045.stdout: [DEP] 2024-06-06T04:33:57.550 INFO:tasks.workunit.client.0.smithi045.stdout: [DEP] 2024-06-06T04:33:57.554 INFO:tasks.workunit.client.0.smithi045.stdout: [DEP] 2024-06-06T04:33:58.188 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] dirstress 2024-06-06T04:33:58.203 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fill 2024-06-06T04:33:58.204 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fill2 2024-06-06T04:33:58.209 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] getpagesize 2024-06-06T04:33:58.465 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] holes 2024-06-06T04:33:58.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:33:58 smithi045 ceph-mon[32299]: pgmap v506: 129 pgs: 129 active+clean; 29 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 501 KiB/s wr, 453 op/s 2024-06-06T04:33:58.492 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] lstat64 2024-06-06T04:33:58.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:33:58 smithi086 ceph-mon[36053]: pgmap v506: 129 pgs: 129 active+clean; 29 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 501 KiB/s wr, 453 op/s 2024-06-06T04:33:58.616 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] nametest 2024-06-06T04:33:58.628 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] permname 2024-06-06T04:33:58.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:33:58 smithi154 ceph-mon[36428]: pgmap v506: 129 pgs: 129 active+clean; 29 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 501 KiB/s wr, 453 op/s 2024-06-06T04:33:58.824 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] randholes 2024-06-06T04:33:58.838 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] runas 2024-06-06T04:33:59.019 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] truncfile 2024-06-06T04:33:59.067 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] usemem 2024-06-06T04:33:59.174 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] mmapcat 2024-06-06T04:33:59.256 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] append_reader 2024-06-06T04:33:59.343 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] append_writer 2024-06-06T04:33:59.364 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] dirperf 2024-06-06T04:33:59.454 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] metaperf 2024-06-06T04:33:59.555 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] devzero 2024-06-06T04:33:59.651 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] feature 2024-06-06T04:33:59.787 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] alloc 2024-06-06T04:33:59.895 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fault 2024-06-06T04:33:59.920 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fstest 2024-06-06T04:34:00.042 INFO:tasks.workunit.client.0.smithi045.stderr:alloc.c: In function 'main': 2024-06-06T04:34:00.043 INFO:tasks.workunit.client.0.smithi045.stderr:alloc.c:154:1: error: 'bozo' undeclared (first use in this function) 2024-06-06T04:34:00.043 INFO:tasks.workunit.client.0.smithi045.stderr: 154 | bozo! 2024-06-06T04:34:00.043 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~ 2024-06-06T04:34:00.043 INFO:tasks.workunit.client.0.smithi045.stderr:alloc.c:154:1: note: each undeclared identifier is reported only once for each function it appears in 2024-06-06T04:34:00.043 INFO:tasks.workunit.client.0.smithi045.stderr:alloc.c:154:5: error: expected ';' before '!' token 2024-06-06T04:34:00.043 INFO:tasks.workunit.client.0.smithi045.stderr: 154 | bozo! 2024-06-06T04:34:00.043 INFO:tasks.workunit.client.0.smithi045.stderr: | ^ 2024-06-06T04:34:00.043 INFO:tasks.workunit.client.0.smithi045.stderr: | ; 2024-06-06T04:34:00.043 INFO:tasks.workunit.client.0.smithi045.stderr:alloc.c:184:25: error: 'rflag' undeclared (first use in this function); did you mean 'nflag'? 2024-06-06T04:34:00.043 INFO:tasks.workunit.client.0.smithi045.stderr: 184 | rflag = 1; 2024-06-06T04:34:00.043 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~ 2024-06-06T04:34:00.043 INFO:tasks.workunit.client.0.smithi045.stderr: | nflag 2024-06-06T04:34:00.044 INFO:tasks.workunit.client.0.smithi045.stderr:alloc.c:320:5: error: expected ';' before '!' token 2024-06-06T04:34:00.044 INFO:tasks.workunit.client.0.smithi045.stderr: 320 | bozo! 2024-06-06T04:34:00.044 INFO:tasks.workunit.client.0.smithi045.stderr: | ^ 2024-06-06T04:34:00.045 INFO:tasks.workunit.client.0.smithi045.stderr: | ; 2024-06-06T04:34:00.045 INFO:tasks.workunit.client.0.smithi045.stderr:alloc.c:128:25: warning: variable 'f' set but not used [-Wunused-but-set-variable] 2024-06-06T04:34:00.045 INFO:tasks.workunit.client.0.smithi045.stderr: 128 | struct flock64 f; 2024-06-06T04:34:00.045 INFO:tasks.workunit.client.0.smithi045.stderr: | ^ 2024-06-06T04:34:00.048 INFO:tasks.workunit.client.0.smithi045.stderr:gmake[3]: *** [Makefile:82: alloc] Error 1 2024-06-06T04:34:00.048 INFO:tasks.workunit.client.0.smithi045.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-06-06T04:34:00.049 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_access_root 2024-06-06T04:34:00.213 INFO:tasks.workunit.client.0.smithi045.stderr:fstest.c: In function 'run_child': 2024-06-06T04:34:00.213 INFO:tasks.workunit.client.0.smithi045.stderr:fstest.c:95:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-06-06T04:34:00.213 INFO:tasks.workunit.client.0.smithi045.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-06-06T04:34:00.213 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~ 2024-06-06T04:34:00.213 INFO:tasks.workunit.client.0.smithi045.stderr:fstest.c:95:24: note: directive argument in the range [0, 2147483647] 2024-06-06T04:34:00.213 INFO:tasks.workunit.client.0.smithi045.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-06-06T04:34:00.213 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~ 2024-06-06T04:34:00.213 INFO:tasks.workunit.client.0.smithi045.stderr:fstest.c:95:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-06-06T04:34:00.213 INFO:tasks.workunit.client.0.smithi045.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-06-06T04:34:00.214 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-06T04:34:00.214 INFO:tasks.workunit.client.0.smithi045.stderr:fstest.c:166:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-06-06T04:34:00.214 INFO:tasks.workunit.client.0.smithi045.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-06-06T04:34:00.214 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~ 2024-06-06T04:34:00.214 INFO:tasks.workunit.client.0.smithi045.stderr:fstest.c:166:24: note: directive argument in the range [0, 2147483647] 2024-06-06T04:34:00.214 INFO:tasks.workunit.client.0.smithi045.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-06-06T04:34:00.214 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~ 2024-06-06T04:34:00.214 INFO:tasks.workunit.client.0.smithi045.stderr:fstest.c:166:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-06-06T04:34:00.214 INFO:tasks.workunit.client.0.smithi045.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-06-06T04:34:00.214 INFO:tasks.workunit.client.0.smithi045.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-06T04:34:00.323 INFO:tasks.workunit.client.0.smithi045.stderr:gmake[2]: *** [include/buildrules:30: src] Error 2 2024-06-06T04:34:00.323 INFO:tasks.workunit.client.0.smithi045.stderr:make[1]: *** [Makefile:53: default] Error 2 2024-06-06T04:34:00.324 INFO:tasks.workunit.client.0.smithi045.stderr:make: *** [Makefile:51: default] Error 2 2024-06-06T04:34:00.325 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-06-06T04:34:00.325 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-06-06T04:34:00.325 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-06T04:34:00.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:00 smithi045 ceph-mon[32299]: pgmap v507: 129 pgs: 129 active+clean; 30 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 578 KiB/s wr, 459 op/s 2024-06-06T04:34:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:00 smithi045 ceph-mon[32299]: scrub summary: idle 2024-06-06T04:34:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:00 smithi045 ceph-mon[32299]: scrub summary: idle 2024-06-06T04:34:00.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:00 smithi086 ceph-mon[36053]: pgmap v507: 129 pgs: 129 active+clean; 30 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 578 KiB/s wr, 459 op/s 2024-06-06T04:34:00.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:00 smithi086 ceph-mon[36053]: scrub summary: idle 2024-06-06T04:34:00.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:00 smithi086 ceph-mon[36053]: scrub summary: idle 2024-06-06T04:34:00.628 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi045 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d746ca5346f34eef7e826a1eb71e59e02b682ae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-06-06T04:34:00.856 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5bbd1e28c2ad42fab247b8ab1a790b45 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi045 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d746ca5346f34eef7e826a1eb71e59e02b682ae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-06-06T04:34:00.859 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-06-06T04:34:00.868 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi045.front.sepia.ceph.com 2024-06-06T04:34:00.873 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:00 smithi154 ceph-mon[36428]: pgmap v507: 129 pgs: 129 active+clean; 30 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 578 KiB/s wr, 459 op/s 2024-06-06T04:34:00.873 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:00 smithi154 ceph-mon[36428]: scrub summary: idle 2024-06-06T04:34:00.874 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:00 smithi154 ceph-mon[36428]: scrub summary: idle 2024-06-06T04:34:00.882 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-06-06T04:34:00.892 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-06-06T04:34:01.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:01 smithi045 ceph-mon[32299]: scrub summary: idle 2024-06-06T04:34:01.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:01 smithi086 ceph-mon[36053]: scrub summary: idle 2024-06-06T04:34:01.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:01 smithi154 ceph-mon[36428]: scrub summary: idle 2024-06-06T04:34:02.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:02 smithi045 ceph-mon[32299]: pgmap v508: 129 pgs: 129 active+clean; 30 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 386 KiB/s wr, 415 op/s 2024-06-06T04:34:02.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:02 smithi086 ceph-mon[36053]: pgmap v508: 129 pgs: 129 active+clean; 30 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 386 KiB/s wr, 415 op/s 2024-06-06T04:34:02.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:02 smithi154 ceph-mon[36428]: pgmap v508: 129 pgs: 129 active+clean; 30 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 386 KiB/s wr, 415 op/s 2024-06-06T04:34:04.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:04 smithi045 ceph-mon[32299]: pgmap v509: 129 pgs: 129 active+clean; 30 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 390 KiB/s wr, 391 op/s 2024-06-06T04:34:04.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:04 smithi086 ceph-mon[36053]: pgmap v509: 129 pgs: 129 active+clean; 30 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 390 KiB/s wr, 391 op/s 2024-06-06T04:34:04.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:04 smithi154 ceph-mon[36428]: pgmap v509: 129 pgs: 129 active+clean; 30 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 390 KiB/s wr, 391 op/s 2024-06-06T04:34:05.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:34:05.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:34:05.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:34:06.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:06 smithi045 ceph-mon[32299]: pgmap v510: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 414 KiB/s wr, 297 op/s 2024-06-06T04:34:06.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:06 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:34:06.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:06 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:34:06.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:06 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:34:06.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:06 smithi086 ceph-mon[36053]: pgmap v510: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 414 KiB/s wr, 297 op/s 2024-06-06T04:34:06.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:06 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:34:06.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:06 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:34:06.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:06 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:34:06.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:06 smithi154 ceph-mon[36428]: pgmap v510: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 414 KiB/s wr, 297 op/s 2024-06-06T04:34:06.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:06 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:34:06.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:06 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:34:06.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:06 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:34:08.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:08 smithi045 ceph-mon[32299]: pgmap v511: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 184 KiB/s wr, 50 op/s 2024-06-06T04:34:08.482 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:08 smithi154 ceph-mon[36428]: pgmap v511: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 184 KiB/s wr, 50 op/s 2024-06-06T04:34:08.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:08 smithi086 ceph-mon[36053]: pgmap v511: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 184 KiB/s wr, 50 op/s 2024-06-06T04:34:10.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:10 smithi086 ceph-mon[36053]: pgmap v512: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 157 KiB/s wr, 44 op/s 2024-06-06T04:34:10.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:10 smithi154 ceph-mon[36428]: pgmap v512: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 157 KiB/s wr, 44 op/s 2024-06-06T04:34:10.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:10 smithi045 ceph-mon[32299]: pgmap v512: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 157 KiB/s wr, 44 op/s 2024-06-06T04:34:12.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:12 smithi086 ceph-mon[36053]: pgmap v513: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 75 KiB/s wr, 34 op/s 2024-06-06T04:34:12.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:12 smithi154 ceph-mon[36428]: pgmap v513: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 75 KiB/s wr, 34 op/s 2024-06-06T04:34:12.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:12 smithi045 ceph-mon[32299]: pgmap v513: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 75 KiB/s wr, 34 op/s 2024-06-06T04:34:14.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:14 smithi086 ceph-mon[36053]: pgmap v514: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 74 KiB/s wr, 34 op/s 2024-06-06T04:34:14.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:14 smithi154 ceph-mon[36428]: pgmap v514: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 74 KiB/s wr, 34 op/s 2024-06-06T04:34:14.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:14 smithi045 ceph-mon[32299]: pgmap v514: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 74 KiB/s wr, 34 op/s 2024-06-06T04:34:16.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:16 smithi086 ceph-mon[36053]: pgmap v515: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 59 KiB/s wr, 25 op/s 2024-06-06T04:34:16.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:16 smithi154 ceph-mon[36428]: pgmap v515: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 59 KiB/s wr, 25 op/s 2024-06-06T04:34:16.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:16 smithi045 ceph-mon[32299]: pgmap v515: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 59 KiB/s wr, 25 op/s 2024-06-06T04:34:18.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:18 smithi086 ceph-mon[36053]: pgmap v516: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-06-06T04:34:18.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:18 smithi154 ceph-mon[36428]: pgmap v516: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-06-06T04:34:18.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:18 smithi045 ceph-mon[32299]: pgmap v516: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-06-06T04:34:20.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:20 smithi086 ceph-mon[36053]: pgmap v517: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-06-06T04:34:20.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:20 smithi154 ceph-mon[36428]: pgmap v517: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-06-06T04:34:20.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:20 smithi045 ceph-mon[32299]: pgmap v517: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-06-06T04:34:22.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:22 smithi086 ceph-mon[36053]: pgmap v518: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.0 KiB/s wr, 6 op/s 2024-06-06T04:34:22.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:22 smithi154 ceph-mon[36428]: pgmap v518: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.0 KiB/s wr, 6 op/s 2024-06-06T04:34:22.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:22 smithi045 ceph-mon[32299]: pgmap v518: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.0 KiB/s wr, 6 op/s 2024-06-06T04:34:24.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T04:34:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:34:24.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:24 smithi045 ceph-mon[32299]: pgmap v519: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-06-06T04:34:24.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:24 smithi086 ceph-mon[36053]: pgmap v519: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-06-06T04:34:24.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:24 smithi154 ceph-mon[36428]: pgmap v519: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-06-06T04:34:26.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.105+0000 7f04db3b9700 1 -- 172.21.15.45:0/2402709309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d406b5b0 msgr2=0x7f04d410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:26.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.105+0000 7f04db3b9700 1 --2- 172.21.15.45:0/2402709309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d406b5b0 0x7f04d410f2e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00b0d0 tx=0x7f04cc00b3e0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.105+0000 7f04db3b9700 1 -- 172.21.15.45:0/2402709309 shutdown_connections 2024-06-06T04:34:26.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.105+0000 7f04db3b9700 1 --2- 172.21.15.45:0/2402709309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d406b5b0 0x7f04d410f2e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.105+0000 7f04db3b9700 1 --2- 172.21.15.45:0/2402709309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d4107390 0x7f04d4107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.105+0000 7f04db3b9700 1 --2- 172.21.15.45:0/2402709309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04d406abf0 0x7f04d406b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.105+0000 7f04db3b9700 1 -- 172.21.15.45:0/2402709309 >> 172.21.15.45:0/2402709309 conn(0x7f04d4078340 msgr2=0x7f04d4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:26.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.105+0000 7f04db3b9700 1 -- 172.21.15.45:0/2402709309 shutdown_connections 2024-06-06T04:34:26.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.105+0000 7f04db3b9700 1 -- 172.21.15.45:0/2402709309 wait complete. 2024-06-06T04:34:26.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.106+0000 7f04db3b9700 1 Processor -- start 2024-06-06T04:34:26.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.106+0000 7f04db3b9700 1 -- start start 2024-06-06T04:34:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.106+0000 7f04db3b9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04d406abf0 0x7f04d4101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.106+0000 7f04db3b9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d406b5b0 0x7f04d4101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.106+0000 7f04db3b9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d4107390 0x7f04d4101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.106+0000 7f04db3b9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f04d4112680 con 0x7f04d406b5b0 2024-06-06T04:34:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.106+0000 7f04db3b9700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04d4112500 con 0x7f04d4107390 2024-06-06T04:34:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.106+0000 7f04db3b9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f04d4112800 con 0x7f04d406abf0 2024-06-06T04:34:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.106+0000 7f04d8954700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d406b5b0 0x7f04d4101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.106+0000 7f04d8954700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d406b5b0 0x7f04d4101980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59780/0 (socket says 172.21.15.45:59780) 2024-06-06T04:34:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.106+0000 7f04d8954700 1 -- 172.21.15.45:0/3460044793 learned_addr learned my addr 172.21.15.45:0/3460044793 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.106+0000 7f04d9155700 1 --2- 172.21.15.45:0/3460044793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04d406abf0 0x7f04d4101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.106+0000 7f04d9956700 1 --2- 172.21.15.45:0/3460044793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d4107390 0x7f04d4101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.107+0000 7f04d8954700 1 -- 172.21.15.45:0/3460044793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04d406abf0 msgr2=0x7f04d4101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.107+0000 7f04d8954700 1 --2- 172.21.15.45:0/3460044793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04d406abf0 0x7f04d4101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.107+0000 7f04d8954700 1 -- 172.21.15.45:0/3460044793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d4107390 msgr2=0x7f04d4101ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.107+0000 7f04d8954700 1 --2- 172.21.15.45:0/3460044793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d4107390 0x7f04d4101ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.107+0000 7f04d8954700 1 -- 172.21.15.45:0/3460044793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d41a7050 con 0x7f04d406b5b0 2024-06-06T04:34:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.107+0000 7f04d8954700 1 --2- 172.21.15.45:0/3460044793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d406b5b0 0x7f04d4101980 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f04c400b530 tx=0x7f04c400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.107+0000 7f04ca7fc700 1 -- 172.21.15.45:0/3460044793 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c4015020 con 0x7f04d406b5b0 2024-06-06T04:34:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.107+0000 7f04db3b9700 1 -- 172.21.15.45:0/3460044793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d41a7340 con 0x7f04d406b5b0 2024-06-06T04:34:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.107+0000 7f04ca7fc700 1 -- 172.21.15.45:0/3460044793 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04c4011ce0 con 0x7f04d406b5b0 2024-06-06T04:34:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.107+0000 7f04ca7fc700 1 -- 172.21.15.45:0/3460044793 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c4010d60 con 0x7f04d406b5b0 2024-06-06T04:34:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.107+0000 7f04db3b9700 1 -- 172.21.15.45:0/3460044793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d41a7880 con 0x7f04d406b5b0 2024-06-06T04:34:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.109+0000 7f04ca7fc700 1 -- 172.21.15.45:0/3460044793 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f04c4018020 con 0x7f04d406b5b0 2024-06-06T04:34:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.110+0000 7f04ca7fc700 1 --2- 172.21.15.45:0/3460044793 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f04c00a2c20 0x7f04c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.110+0000 7f04d9155700 1 --2- 172.21.15.45:0/3460044793 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f04c00a2c20 0x7f04c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.110+0000 7f04ca7fc700 1 -- 172.21.15.45:0/3460044793 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f04c40ced00 con 0x7f04d406b5b0 2024-06-06T04:34:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.110+0000 7f04d9155700 1 --2- 172.21.15.45:0/3460044793 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f04c00a2c20 0x7f04c00a50e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f04d0005fd0 tx=0x7f04d000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:26.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04e5c3cb80 1 Processor -- start 2024-06-06T04:34:26.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04e5c3cb80 1 -- start start 2024-06-06T04:34:26.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04e5c3cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555922836f60 0x555922837340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04e5c3cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x555922831260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04e5c3cb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559228317a0 0x555922831bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04e5c3cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555922723f60 con 0x555922837880 2024-06-06T04:34:26.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04e5c3cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555922782850 con 0x555922836f60 2024-06-06T04:34:26.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04e5c3cb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5559226ea460 con 0x5559228317a0 2024-06-06T04:34:26.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04d8954700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x555922831260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04d8954700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x555922831260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59782/0 (socket says 172.21.15.45:59782) 2024-06-06T04:34:26.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04d8954700 1 -- 172.21.15.45:0/3977504378 learned_addr learned my addr 172.21.15.45:0/3977504378 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:26.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04d9155700 1 --2- 172.21.15.45:0/3977504378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555922836f60 0x555922837340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04d9956700 1 --2- 172.21.15.45:0/3977504378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559228317a0 0x555922831bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04d8954700 1 -- 172.21.15.45:0/3977504378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559228317a0 msgr2=0x555922831bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04d8954700 1 --2- 172.21.15.45:0/3977504378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559228317a0 0x555922831bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04d8954700 1 -- 172.21.15.45:0/3977504378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555922836f60 msgr2=0x555922837340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04d8954700 1 --2- 172.21.15.45:0/3977504378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555922836f60 0x555922837340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04d8954700 1 -- 172.21.15.45:0/3977504378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559226d18d0 con 0x555922837880 2024-06-06T04:34:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.111+0000 7f04d9155700 1 --2- 172.21.15.45:0/3977504378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555922836f60 0x555922837340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:34:26.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.112+0000 7f04d8954700 1 --2- 172.21.15.45:0/3977504378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x555922831260 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f04c40975a0 tx=0x7f04c409ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b337e631690855a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:26.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.112+0000 7f04bf7fe700 1 -- 172.21.15.45:0/3977504378 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c409b840 con 0x555922837880 2024-06-06T04:34:26.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.112+0000 7f04bf7fe700 1 -- 172.21.15.45:0/3977504378 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04c409b140 con 0x555922837880 2024-06-06T04:34:26.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.112+0000 7f04bf7fe700 1 -- 172.21.15.45:0/3977504378 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c409f120 con 0x555922837880 2024-06-06T04:34:26.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.112+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/3977504378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 msgr2=0x555922831260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:26.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.112+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/3977504378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x555922831260 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f04c40975a0 tx=0x7f04c409ac80 comp rx=0 tx=0).stop 2024-06-06T04:34:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.112+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/3977504378 shutdown_connections 2024-06-06T04:34:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.113+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/3977504378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5559228317a0 0x555922831bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.113+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/3977504378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555922836f60 0x555922837340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.113+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/3977504378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x555922831260 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.113+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/3977504378 >> 172.21.15.45:0/3977504378 conn(0x55592275e080 msgr2=0x55592282c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.113+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/3977504378 shutdown_connections 2024-06-06T04:34:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.113+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/3977504378 wait complete. 2024-06-06T04:34:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.113+0000 7f04e5c3cb80 1 -- start start 2024-06-06T04:34:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.113+0000 7f04e5c3cb80 1 Processor -- start 2024-06-06T04:34:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04e5c3cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 0x55592283d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04e5c3cb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555922836f60 0x55592283a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04e5c3cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x55592283ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04e5c3cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555922722fc0 con 0x555922837880 2024-06-06T04:34:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04e5c3cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55592277e810 con 0x5559228317a0 2024-06-06T04:34:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04e5c3cb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55592283e260 con 0x555922836f60 2024-06-06T04:34:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04e5c3cb80 10 client.0 _ll_register_callbacks cb 0x5559227185b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:34:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04d9155700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 0x55592283d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04d9956700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x55592283ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04d8954700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555922836f60 0x55592283a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04d9956700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x55592283ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59798/0 (socket says 172.21.15.45:59798) 2024-06-06T04:34:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04d9155700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 0x55592283d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:37530/0 (socket says 172.21.15.45:37530) 2024-06-06T04:34:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04d9956700 1 -- 172.21.15.45:0/280889834 learned_addr learned my addr 172.21.15.45:0/280889834 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04d9956700 1 -- 172.21.15.45:0/280889834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555922836f60 msgr2=0x55592283a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04d9956700 1 --2- 172.21.15.45:0/280889834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555922836f60 0x55592283a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04d9956700 1 -- 172.21.15.45:0/280889834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 msgr2=0x55592283d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04d9956700 1 --2- 172.21.15.45:0/280889834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 0x55592283d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04d9956700 1 -- 172.21.15.45:0/280889834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555922748750 con 0x555922837880 2024-06-06T04:34:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04d9956700 1 --2- 172.21.15.45:0/280889834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x55592283ade0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f04cc004cb0 tx=0x7f04cc004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b4758caa549bd1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.114+0000 7f04bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:34:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.115+0000 7f04bcff9700 1 -- 172.21.15.45:0/280889834 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc021070 con 0x555922837880 2024-06-06T04:34:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.115+0000 7f04bcff9700 1 -- 172.21.15.45:0/280889834 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f04cc0076e0 con 0x555922837880 2024-06-06T04:34:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.115+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/280889834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c409b140 con 0x555922837880 2024-06-06T04:34:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.115+0000 7f04bcff9700 1 -- 172.21.15.45:0/280889834 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04cc02e050 con 0x555922837880 2024-06-06T04:34:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.115+0000 7f04e5c3cb80 10 client.15834 fetch_fsmap learned FSMap version 30 2024-06-06T04:34:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.115+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/280889834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559228489f0 con 0x555922837880 2024-06-06T04:34:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.115+0000 7f04bcff9700 1 -- 172.21.15.45:0/280889834 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04c409b140 con 0x555922837880 2024-06-06T04:34:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.115+0000 7f04bcff9700 1 -- 172.21.15.45:0/280889834 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f04cc011070 con 0x555922837880 2024-06-06T04:34:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.115+0000 7f04e5c3cb80 10 client.15834 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:34:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.115+0000 7f04e5c3cb80 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:34:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.115+0000 7f04e5c3cb80 20 client.15834 populate_metadata read hostname 'smithi045' 2024-06-06T04:34:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.115+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/280889834 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55592284b030 0x55592284d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.115+0000 7f04e5c3cb80 4 client.15834 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:34:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.115+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/280889834 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55592278fd50 con 0x55592284b030 2024-06-06T04:34:26.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.116+0000 7f04d9155700 1 --2- 172.21.15.45:0/280889834 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55592284b030 0x55592284d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.116+0000 7f04d9155700 1 --2- 172.21.15.45:0/280889834 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55592284b030 0x55592284d460 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd62cd0f4d81046f server_cookie=20a7285ef9faed5 in_seq=0 out_seq=0 2024-06-06T04:34:26.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.116+0000 7f04bcff9700 10 client.15834 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:34:26.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.117+0000 7f04bcff9700 1 -- 172.21.15.45:0/280889834 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55592278fd50 con 0x55592284b030 2024-06-06T04:34:26.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.117+0000 7f04bcff9700 10 client.15834 handle_command_reply: tid=0 2024-06-06T04:34:26.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.117+0000 7f04e5c3cb80 1 client.15834 shutdown 2024-06-06T04:34:26.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.117+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/280889834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 msgr2=0x55592283ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:26.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.117+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/280889834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x55592283ade0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f04cc004cb0 tx=0x7f04cc004ce0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.117+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/280889834 shutdown_connections 2024-06-06T04:34:26.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.117+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/280889834 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x55592284b030 0x55592284d460 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.117+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/280889834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x55592283ade0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.117+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/280889834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 0x55592283d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.117+0000 7f04bcff9700 0 client.15834 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:34:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.117+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/280889834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555922836f60 0x55592283a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.117+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/280889834 >> 172.21.15.45:0/280889834 conn(0x55592275e080 msgr2=0x55592282b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.117+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/280889834 shutdown_connections 2024-06-06T04:34:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.117+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/280889834 wait complete. 2024-06-06T04:34:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.117+0000 7f04e5c3cb80 20 client.15834 trim_cache size 0 max 16384 2024-06-06T04:34:26.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.138+0000 7f04e5c3cb80 1 Processor -- start 2024-06-06T04:34:26.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.138+0000 7f04e5c3cb80 1 -- start start 2024-06-06T04:34:26.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.138+0000 7f04e5c3cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 0x555922831260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.138+0000 7f04e5c3cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922836f60 0x555922838430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.138+0000 7f04e5c3cb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555922837880 0x55592282be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.138+0000 7f04e5c3cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41b2d60 con 0x555922836f60 2024-06-06T04:34:26.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.138+0000 7f04e5c3cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41b05c0 con 0x5559228317a0 2024-06-06T04:34:26.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.138+0000 7f04e5c3cb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41ae410 con 0x555922837880 2024-06-06T04:34:26.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.138+0000 7f04d8954700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922836f60 0x555922838430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.138+0000 7f04d8954700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922836f60 0x555922838430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59806/0 (socket says 172.21.15.45:59806) 2024-06-06T04:34:26.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.138+0000 7f04d8954700 1 -- 172.21.15.45:0/1498965523 learned_addr learned my addr 172.21.15.45:0/1498965523 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:26.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.138+0000 7f04d9155700 1 --2- 172.21.15.45:0/1498965523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 0x555922831260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.138+0000 7f04d9956700 1 --2- 172.21.15.45:0/1498965523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555922837880 0x55592282be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.139+0000 7f04d8954700 1 -- 172.21.15.45:0/1498965523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555922837880 msgr2=0x55592282be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.139+0000 7f04d8954700 1 --2- 172.21.15.45:0/1498965523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555922837880 0x55592282be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.139+0000 7f04d8954700 1 -- 172.21.15.45:0/1498965523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 msgr2=0x555922831260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.139+0000 7f04d8954700 1 --2- 172.21.15.45:0/1498965523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 0x555922831260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.139+0000 7f04d8954700 1 -- 172.21.15.45:0/1498965523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55592284dae0 con 0x555922836f60 2024-06-06T04:34:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.139+0000 7f04d9155700 1 --2- 172.21.15.45:0/1498965523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 0x555922831260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:34:26.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.139+0000 7f04d8954700 1 --2- 172.21.15.45:0/1498965523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922836f60 0x555922838430 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f04c40a3880 tx=0x7f04c4009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882400d6c6d858b1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:26.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.139+0000 7f04beffd700 1 -- 172.21.15.45:0/1498965523 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c409b840 con 0x555922836f60 2024-06-06T04:34:26.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.139+0000 7f04beffd700 1 -- 172.21.15.45:0/1498965523 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04c40a3db0 con 0x555922836f60 2024-06-06T04:34:26.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.139+0000 7f04beffd700 1 -- 172.21.15.45:0/1498965523 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c40a3920 con 0x555922836f60 2024-06-06T04:34:26.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.139+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/1498965523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922836f60 msgr2=0x555922838430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:26.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/1498965523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922836f60 0x555922838430 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f04c40a3880 tx=0x7f04c4009300 comp rx=0 tx=0).stop 2024-06-06T04:34:26.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/1498965523 shutdown_connections 2024-06-06T04:34:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/1498965523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555922837880 0x55592282be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/1498965523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 0x555922831260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/1498965523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922836f60 0x555922838430 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/1498965523 >> 172.21.15.45:0/1498965523 conn(0x55592275e080 msgr2=0x555922837d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/1498965523 shutdown_connections 2024-06-06T04:34:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/1498965523 wait complete. 2024-06-06T04:34:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 1 -- start start 2024-06-06T04:34:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 1 Processor -- start 2024-06-06T04:34:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 0x555922843550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555922836f60 0x555922843a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x555922840520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41aea80 con 0x555922837880 2024-06-06T04:34:26.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41afef0 con 0x5559228317a0 2024-06-06T04:34:26.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41af150 con 0x555922836f60 2024-06-06T04:34:26.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.140+0000 7f04e5c3cb80 10 client.0 _ll_register_callbacks cb 0x5559227185b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:34:26.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04d9956700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x555922840520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04d9956700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x555922840520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59818/0 (socket says 172.21.15.45:59818) 2024-06-06T04:34:26.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04d9956700 1 -- 172.21.15.45:0/3551944292 learned_addr learned my addr 172.21.15.45:0/3551944292 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:26.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04d9155700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 0x555922843550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04d8954700 1 --2- 172.21.15.45:0/3551944292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555922836f60 0x555922843a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04d9956700 1 -- 172.21.15.45:0/3551944292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555922836f60 msgr2=0x555922843a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:26.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04d9956700 1 --2- 172.21.15.45:0/3551944292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555922836f60 0x555922843a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04d9956700 1 -- 172.21.15.45:0/3551944292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 msgr2=0x555922843550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:26.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04d9956700 1 --2- 172.21.15.45:0/3551944292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 0x555922843550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04d9956700 1 -- 172.21.15.45:0/3551944292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555922840a60 con 0x555922837880 2024-06-06T04:34:26.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04d9956700 1 --2- 172.21.15.45:0/3551944292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x555922840520 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00b420 tx=0x7f04cc004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc61d96b868da9ff server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:26.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:34:26.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04bcff9700 1 -- 172.21.15.45:0/3551944292 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc037060 con 0x555922837880 2024-06-06T04:34:26.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04bcff9700 1 -- 172.21.15.45:0/3551944292 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f04cc005f70 con 0x555922837880 2024-06-06T04:34:26.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/3551944292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c40a3db0 con 0x555922837880 2024-06-06T04:34:26.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04bcff9700 1 -- 172.21.15.45:0/3551944292 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04cc03e050 con 0x555922837880 2024-06-06T04:34:26.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04e5c3cb80 10 client.15840 fetch_fsmap learned FSMap version 30 2024-06-06T04:34:26.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.141+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/3551944292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559228b6400 con 0x555922837880 2024-06-06T04:34:26.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.142+0000 7f04bcff9700 1 -- 172.21.15.45:0/3551944292 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04c40a3db0 con 0x555922837880 2024-06-06T04:34:26.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.142+0000 7f04bcff9700 1 -- 172.21.15.45:0/3551944292 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f04cc011070 con 0x555922837880 2024-06-06T04:34:26.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.142+0000 7f04e5c3cb80 10 client.15840 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:34:26.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.142+0000 7f04e5c3cb80 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:34:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.142+0000 7f04e5c3cb80 20 client.15840 populate_metadata read hostname 'smithi045' 2024-06-06T04:34:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.142+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/3551944292 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5559228b6b40 0x5559228b8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.142+0000 7f04e5c3cb80 4 client.15840 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T04:34:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.142+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/3551944292 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55592278a2d0 con 0x5559228b6b40 2024-06-06T04:34:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.142+0000 7f04d9155700 1 --2- 172.21.15.45:0/3551944292 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5559228b6b40 0x5559228b8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.142+0000 7f04bcff9700 10 client.15840 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:34:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.142+0000 7f04d9155700 1 --2- 172.21.15.45:0/3551944292 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5559228b6b40 0x5559228b8f60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1bf9368e9d21ade server_cookie=9f21df26d4fde50c in_seq=0 out_seq=0 2024-06-06T04:34:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.143+0000 7f04bcff9700 1 -- 172.21.15.45:0/3551944292 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55592278a2d0 con 0x5559228b6b40 2024-06-06T04:34:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.143+0000 7f04bcff9700 10 client.15840 handle_command_reply: tid=0 2024-06-06T04:34:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.143+0000 7f04e5c3cb80 1 client.15840 shutdown 2024-06-06T04:34:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.143+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/3551944292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 msgr2=0x555922840520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.143+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/3551944292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x555922840520 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00b420 tx=0x7f04cc004d50 comp rx=0 tx=0).stop 2024-06-06T04:34:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.143+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/3551944292 shutdown_connections 2024-06-06T04:34:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.143+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/3551944292 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x5559228b6b40 0x5559228b8f60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.143+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/3551944292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555922837880 0x555922840520 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.143+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/3551944292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5559228317a0 0x555922843550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.143+0000 7f04bcff9700 0 client.15840 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:34:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.143+0000 7f04e5c3cb80 1 --2- 172.21.15.45:0/3551944292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555922836f60 0x555922843a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.143+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/3551944292 >> 172.21.15.45:0/3551944292 conn(0x55592275e080 msgr2=0x55592283c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:26.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.143+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/3551944292 shutdown_connections 2024-06-06T04:34:26.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.143+0000 7f04e5c3cb80 1 -- 172.21.15.45:0/3551944292 wait complete. 2024-06-06T04:34:26.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.143+0000 7f04e5c3cb80 20 client.15840 trim_cache size 0 max 16384 2024-06-06T04:34:26.157 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:34:26.157 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-06-06T04:34:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-06-06T04:34:26.158 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:34:26.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.144+0000 7f04db3b9700 1 -- 172.21.15.45:0/3460044793 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f04c00a2c20 msgr2=0x7f04c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:26.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.144+0000 7f04db3b9700 1 --2- 172.21.15.45:0/3460044793 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f04c00a2c20 0x7f04c00a50e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f04d0005fd0 tx=0x7f04d000a560 comp rx=0 tx=0).stop 2024-06-06T04:34:26.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.144+0000 7f04db3b9700 1 -- 172.21.15.45:0/3460044793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d406b5b0 msgr2=0x7f04d4101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:26.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.144+0000 7f04db3b9700 1 --2- 172.21.15.45:0/3460044793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d406b5b0 0x7f04d4101980 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f04c400b530 tx=0x7f04c400b8f0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.144+0000 7f04db3b9700 1 -- 172.21.15.45:0/3460044793 shutdown_connections 2024-06-06T04:34:26.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.144+0000 7f04db3b9700 1 --2- 172.21.15.45:0/3460044793 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f04c00a2c20 0x7f04c00a50e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.144+0000 7f04db3b9700 1 --2- 172.21.15.45:0/3460044793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04d4107390 0x7f04d4101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.144+0000 7f04db3b9700 1 --2- 172.21.15.45:0/3460044793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04d406abf0 0x7f04d4101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.144+0000 7f04db3b9700 1 --2- 172.21.15.45:0/3460044793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04d406b5b0 0x7f04d4101980 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:26.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.144+0000 7f04db3b9700 1 -- 172.21.15.45:0/3460044793 >> 172.21.15.45:0/3460044793 conn(0x7f04d4078340 msgr2=0x7f04d40ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:26.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.144+0000 7f04db3b9700 1 -- 172.21.15.45:0/3460044793 shutdown_connections 2024-06-06T04:34:26.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:26.144+0000 7f04db3b9700 1 -- 172.21.15.45:0/3460044793 wait complete. 2024-06-06T04:34:26.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:26 smithi086 ceph-mon[36053]: pgmap v520: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.9 KiB/s wr, 6 op/s 2024-06-06T04:34:26.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:26 smithi154 ceph-mon[36428]: pgmap v520: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.9 KiB/s wr, 6 op/s 2024-06-06T04:34:26.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:26 smithi045 ceph-mon[32299]: pgmap v520: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.9 KiB/s wr, 6 op/s 2024-06-06T04:34:26.935 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-06T04:34:26.935 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:34:27.190 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:34:28.439 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:28 smithi045 ceph-mon[32299]: pgmap v521: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-06-06T04:34:28.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:28 smithi086 ceph-mon[36053]: pgmap v521: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-06-06T04:34:28.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:28 smithi154 ceph-mon[36428]: pgmap v521: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-06-06T04:34:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.992+0000 7fca3ab75700 1 -- 172.21.15.45:0/2762402099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca341084e0 msgr2=0x7fca341088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.992+0000 7fca3ab75700 1 --2- 172.21.15.45:0/2762402099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca341084e0 0x7fca341088c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fca280099c0 tx=0x7fca28009cd0 comp rx=0 tx=0).stop 2024-06-06T04:34:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.992+0000 7fca3ab75700 1 -- 172.21.15.45:0/2762402099 shutdown_connections 2024-06-06T04:34:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.992+0000 7fca3ab75700 1 --2- 172.21.15.45:0/2762402099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca3406ab10 0x7fca3410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.992+0000 7fca3ab75700 1 --2- 172.21.15.45:0/2762402099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca341084e0 0x7fca341088c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:28.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.992+0000 7fca3ab75700 1 --2- 172.21.15.45:0/2762402099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca3406a170 0x7fca3406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:28.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.992+0000 7fca3ab75700 1 -- 172.21.15.45:0/2762402099 >> 172.21.15.45:0/2762402099 conn(0x7fca340782c0 msgr2=0x7fca340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:28.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.992+0000 7fca3ab75700 1 -- 172.21.15.45:0/2762402099 shutdown_connections 2024-06-06T04:34:28.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.992+0000 7fca3ab75700 1 -- 172.21.15.45:0/2762402099 wait complete. 2024-06-06T04:34:28.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.992+0000 7fca3ab75700 1 Processor -- start 2024-06-06T04:34:28.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.992+0000 7fca3ab75700 1 -- start start 2024-06-06T04:34:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca3ab75700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca3406a170 0x7fca34101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca3ab75700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca3406ab10 0x7fca341019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca3ab75700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca341084e0 0x7fca34101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca3ab75700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fca341125d0 con 0x7fca341084e0 2024-06-06T04:34:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca3ab75700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca34112450 con 0x7fca3406a170 2024-06-06T04:34:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca3ab75700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fca34112750 con 0x7fca3406ab10 2024-06-06T04:34:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca39112700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca341084e0 0x7fca34101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca39112700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca341084e0 0x7fca34101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57098/0 (socket says 172.21.15.45:57098) 2024-06-06T04:34:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca39112700 1 -- 172.21.15.45:0/1627564592 learned_addr learned my addr 172.21.15.45:0/1627564592 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca38911700 1 --2- 172.21.15.45:0/1627564592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca3406a170 0x7fca34101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca33fff700 1 --2- 172.21.15.45:0/1627564592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca3406ab10 0x7fca341019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca39112700 1 -- 172.21.15.45:0/1627564592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca3406ab10 msgr2=0x7fca341019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca39112700 1 --2- 172.21.15.45:0/1627564592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca3406ab10 0x7fca341019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca39112700 1 -- 172.21.15.45:0/1627564592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca3406a170 msgr2=0x7fca34101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca39112700 1 --2- 172.21.15.45:0/1627564592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca3406a170 0x7fca34101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca39112700 1 -- 172.21.15.45:0/1627564592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca341a6f70 con 0x7fca341084e0 2024-06-06T04:34:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.993+0000 7fca39112700 1 --2- 172.21.15.45:0/1627564592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca341084e0 0x7fca34101ee0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fca2c007d70 tx=0x7fca2c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.994+0000 7fca31ffb700 1 -- 172.21.15.45:0/1627564592 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c016020 con 0x7fca341084e0 2024-06-06T04:34:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.994+0000 7fca3ab75700 1 -- 172.21.15.45:0/1627564592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca341a7260 con 0x7fca341084e0 2024-06-06T04:34:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.994+0000 7fca31ffb700 1 -- 172.21.15.45:0/1627564592 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca2c00ad40 con 0x7fca341084e0 2024-06-06T04:34:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.994+0000 7fca31ffb700 1 -- 172.21.15.45:0/1627564592 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c011620 con 0x7fca341084e0 2024-06-06T04:34:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.994+0000 7fca3ab75700 1 -- 172.21.15.45:0/1627564592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca341a7750 con 0x7fca341084e0 2024-06-06T04:34:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.995+0000 7fca3ab75700 1 -- 172.21.15.45:0/1627564592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca34069180 con 0x7fca341084e0 2024-06-06T04:34:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.996+0000 7fca31ffb700 1 -- 172.21.15.45:0/1627564592 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fca2c019020 con 0x7fca341084e0 2024-06-06T04:34:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.997+0000 7fca31ffb700 1 --2- 172.21.15.45:0/1627564592 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fca240a2c20 0x7fca240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.997+0000 7fca38911700 1 --2- 172.21.15.45:0/1627564592 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fca240a2c20 0x7fca240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.997+0000 7fca31ffb700 1 -- 172.21.15.45:0/1627564592 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fca2c0cf480 con 0x7fca341084e0 2024-06-06T04:34:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:28.997+0000 7fca38911700 1 --2- 172.21.15.45:0/1627564592 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fca240a2c20 0x7fca240a50e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fca28009750 tx=0x7fca28005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:29.002+0000 7fca31ffb700 1 -- 172.21.15.45:0/1627564592 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fca2c020020 con 0x7fca341084e0 2024-06-06T04:34:29.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:29.254+0000 7fca3ab75700 1 -- 172.21.15.45:0/1627564592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca3402dca0 con 0x7fca341084e0 2024-06-06T04:34:29.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:29.258+0000 7fca31ffb700 1 -- 172.21.15.45:0/1627564592 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11862 (secure 0 0 0) 0x7fca2c019310 con 0x7fca341084e0 2024-06-06T04:34:29.258 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:34:29.259 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T04:27:24.462897+0000","modified":"2024-06-06T04:33:18.907139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24451,24463,24473]},"id":1}]} 2024-06-06T04:34:29.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:29.260+0000 7fca3ab75700 1 -- 172.21.15.45:0/1627564592 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fca240a2c20 msgr2=0x7fca240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:29.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:29.260+0000 7fca3ab75700 1 --2- 172.21.15.45:0/1627564592 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fca240a2c20 0x7fca240a50e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fca28009750 tx=0x7fca28005fd0 comp rx=0 tx=0).stop 2024-06-06T04:34:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:29.260+0000 7fca3ab75700 1 -- 172.21.15.45:0/1627564592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca341084e0 msgr2=0x7fca34101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:29.260+0000 7fca3ab75700 1 --2- 172.21.15.45:0/1627564592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca341084e0 0x7fca34101ee0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fca2c007d70 tx=0x7fca2c00a340 comp rx=0 tx=0).stop 2024-06-06T04:34:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:29.260+0000 7fca3ab75700 1 -- 172.21.15.45:0/1627564592 shutdown_connections 2024-06-06T04:34:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:29.260+0000 7fca3ab75700 1 --2- 172.21.15.45:0/1627564592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca341084e0 0x7fca34101ee0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:29.260+0000 7fca3ab75700 1 --2- 172.21.15.45:0/1627564592 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fca240a2c20 0x7fca240a50e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:29.260+0000 7fca3ab75700 1 --2- 172.21.15.45:0/1627564592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca3406a170 0x7fca34101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:29.260+0000 7fca3ab75700 1 --2- 172.21.15.45:0/1627564592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca3406ab10 0x7fca341019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:29.260+0000 7fca3ab75700 1 -- 172.21.15.45:0/1627564592 >> 172.21.15.45:0/1627564592 conn(0x7fca340782c0 msgr2=0x7fca340ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:29.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:29.260+0000 7fca3ab75700 1 -- 172.21.15.45:0/1627564592 shutdown_connections 2024-06-06T04:34:29.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:29.260+0000 7fca3ab75700 1 -- 172.21.15.45:0/1627564592 wait complete. 2024-06-06T04:34:29.262 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 30 2024-06-06T04:34:29.901 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-06T04:34:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:34:30.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:30 smithi045 ceph-mon[32299]: pgmap v522: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-06-06T04:34:30.587 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:30 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1627564592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:34:30.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:30 smithi086 ceph-mon[36053]: pgmap v522: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-06-06T04:34:30.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:30 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1627564592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:34:30.684 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:30 smithi154 ceph-mon[36428]: pgmap v522: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-06-06T04:34:30.684 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:30 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1627564592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:34:32.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.015+0000 7f96a1633700 1 -- 172.21.15.45:0/3138044925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c100a00 msgr2=0x7f969c100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:32.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.015+0000 7f96a1633700 1 --2- 172.21.15.45:0/3138044925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c100a00 0x7f969c100e80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f96840099c0 tx=0x7f9684009cd0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.015+0000 7f96a1633700 1 -- 172.21.15.45:0/3138044925 shutdown_connections 2024-06-06T04:34:32.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.015+0000 7f96a1633700 1 --2- 172.21.15.45:0/3138044925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f969c1013c0 0x7f969c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.015+0000 7f96a1633700 1 --2- 172.21.15.45:0/3138044925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f969c108580 0x7f969c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.015+0000 7f96a1633700 1 --2- 172.21.15.45:0/3138044925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c100a00 0x7f969c100e80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.015+0000 7f96a1633700 1 -- 172.21.15.45:0/3138044925 >> 172.21.15.45:0/3138044925 conn(0x7f969c077990 msgr2=0x7f969c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:32.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.015+0000 7f96a1633700 1 -- 172.21.15.45:0/3138044925 shutdown_connections 2024-06-06T04:34:32.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.015+0000 7f96a1633700 1 -- 172.21.15.45:0/3138044925 wait complete. 2024-06-06T04:34:32.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.015+0000 7f96a1633700 1 Processor -- start 2024-06-06T04:34:32.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.016+0000 7f96a1633700 1 -- start start 2024-06-06T04:34:32.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.016+0000 7f96a1633700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f969c100a00 0x7f969c101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.016+0000 7f96a1633700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f969c1013c0 0x7f969c102120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.016+0000 7f96a1633700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c108580 0x7f969c102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.016+0000 7f96a1633700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f969c1126a0 con 0x7f969c108580 2024-06-06T04:34:32.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.016+0000 7f96a1633700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f969c112520 con 0x7f969c1013c0 2024-06-06T04:34:32.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.016+0000 7f96a1633700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f969c112820 con 0x7f969c100a00 2024-06-06T04:34:32.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.016+0000 7f969b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c108580 0x7f969c102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.016+0000 7f969b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c108580 0x7f969c102660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57110/0 (socket says 172.21.15.45:57110) 2024-06-06T04:34:32.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.016+0000 7f969b7fe700 1 -- 172.21.15.45:0/655960337 learned_addr learned my addr 172.21.15.45:0/655960337 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:32.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.016+0000 7f969a7fc700 1 --2- 172.21.15.45:0/655960337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f969c1013c0 0x7f969c102120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.016+0000 7f969affd700 1 --2- 172.21.15.45:0/655960337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f969c100a00 0x7f969c101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.017+0000 7f969b7fe700 1 -- 172.21.15.45:0/655960337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f969c100a00 msgr2=0x7f969c101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:32.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.017+0000 7f969b7fe700 1 --2- 172.21.15.45:0/655960337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f969c100a00 0x7f969c101be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.017+0000 7f969b7fe700 1 -- 172.21.15.45:0/655960337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f969c1013c0 msgr2=0x7f969c102120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:32.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.017+0000 7f969b7fe700 1 --2- 172.21.15.45:0/655960337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f969c1013c0 0x7f969c102120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.017+0000 7f969b7fe700 1 -- 172.21.15.45:0/655960337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969c1a7080 con 0x7f969c108580 2024-06-06T04:34:32.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.017+0000 7f969b7fe700 1 --2- 172.21.15.45:0/655960337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c108580 0x7f969c102660 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f969400efc0 tx=0x7f969400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:32.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.017+0000 7f9693fff700 1 -- 172.21.15.45:0/655960337 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96940092e0 con 0x7f969c108580 2024-06-06T04:34:32.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.017+0000 7f96a1633700 1 -- 172.21.15.45:0/655960337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f969c1a7310 con 0x7f969c108580 2024-06-06T04:34:32.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.017+0000 7f9693fff700 1 -- 172.21.15.45:0/655960337 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9694004bb0 con 0x7f969c108580 2024-06-06T04:34:32.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.017+0000 7f9693fff700 1 -- 172.21.15.45:0/655960337 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694004020 con 0x7f969c108580 2024-06-06T04:34:32.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.017+0000 7f96a1633700 1 -- 172.21.15.45:0/655960337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f969c1a77d0 con 0x7f969c108580 2024-06-06T04:34:32.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.019+0000 7f9693fff700 1 -- 172.21.15.45:0/655960337 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f969401b030 con 0x7f969c108580 2024-06-06T04:34:32.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.020+0000 7f9693fff700 1 --2- 172.21.15.45:0/655960337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f96880a2c20 0x7f96880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.020+0000 7f969affd700 1 --2- 172.21.15.45:0/655960337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f96880a2c20 0x7f96880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.020+0000 7f9693fff700 1 -- 172.21.15.45:0/655960337 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f96940cedb0 con 0x7f969c108580 2024-06-06T04:34:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.021+0000 7f969affd700 1 --2- 172.21.15.45:0/655960337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f96880a2c20 0x7f96880a50e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f968c0045b0 tx=0x7f968c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.021+0000 7f96abeb6b80 1 Processor -- start 2024-06-06T04:34:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.021+0000 7f96abeb6b80 1 -- start start 2024-06-06T04:34:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.021+0000 7f96abeb6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08d00640 0x564d08d00a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.021+0000 7f96abeb6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00f60 0x564d08cfa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.021+0000 7f96abeb6b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08cfaf20 0x564d08cfb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.021+0000 7f96abeb6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564d08c4c810 con 0x564d08d00f60 2024-06-06T04:34:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.022+0000 7f96abeb6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564d08bf4720 con 0x564d08d00640 2024-06-06T04:34:32.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.022+0000 7f96abeb6b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564d08c487d0 con 0x564d08cfaf20 2024-06-06T04:34:32.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.022+0000 7f969a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00f60 0x564d08cfa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.022+0000 7f969a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00f60 0x564d08cfa9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57126/0 (socket says 172.21.15.45:57126) 2024-06-06T04:34:32.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.022+0000 7f969a7fc700 1 -- 172.21.15.45:0/734542422 learned_addr learned my addr 172.21.15.45:0/734542422 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:32.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.022+0000 7f969affd700 1 --2- 172.21.15.45:0/734542422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08d00640 0x564d08d00a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.022+0000 7f969b7fe700 1 --2- 172.21.15.45:0/734542422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08cfaf20 0x564d08cfb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.022+0000 7f969a7fc700 1 -- 172.21.15.45:0/734542422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08cfaf20 msgr2=0x564d08cfb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:32.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.022+0000 7f969a7fc700 1 --2- 172.21.15.45:0/734542422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08cfaf20 0x564d08cfb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.022+0000 7f969a7fc700 1 -- 172.21.15.45:0/734542422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08d00640 msgr2=0x564d08d00a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.022+0000 7f969a7fc700 1 --2- 172.21.15.45:0/734542422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08d00640 0x564d08d00a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.022+0000 7f969a7fc700 1 -- 172.21.15.45:0/734542422 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d08beaf50 con 0x564d08d00f60 2024-06-06T04:34:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.022+0000 7f969a7fc700 1 --2- 172.21.15.45:0/734542422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00f60 0x564d08cfa9e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f96840099c0 tx=0x7f9684005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c2067f7d6ad101 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.022+0000 7f96917fa700 1 -- 172.21.15.45:0/734542422 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684028070 con 0x564d08d00f60 2024-06-06T04:34:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.022+0000 7f96917fa700 1 -- 172.21.15.45:0/734542422 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96840037c0 con 0x564d08d00f60 2024-06-06T04:34:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.022+0000 7f96917fa700 1 -- 172.21.15.45:0/734542422 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968400b9f0 con 0x564d08d00f60 2024-06-06T04:34:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.023+0000 7f96abeb6b80 1 -- 172.21.15.45:0/734542422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00f60 msgr2=0x564d08cfa9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.023+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/734542422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00f60 0x564d08cfa9e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f96840099c0 tx=0x7f9684005bc0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.023+0000 7f96abeb6b80 1 -- 172.21.15.45:0/734542422 shutdown_connections 2024-06-06T04:34:32.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.023+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/734542422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08cfaf20 0x564d08cfb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.023+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/734542422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08d00640 0x564d08d00a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.023+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/734542422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00f60 0x564d08cfa9e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.023+0000 7f96abeb6b80 1 -- 172.21.15.45:0/734542422 >> 172.21.15.45:0/734542422 conn(0x564d08c25cc0 msgr2=0x564d08cf5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:32.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.023+0000 7f96abeb6b80 1 -- 172.21.15.45:0/734542422 shutdown_connections 2024-06-06T04:34:32.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.023+0000 7f96abeb6b80 1 -- 172.21.15.45:0/734542422 wait complete. 2024-06-06T04:34:32.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.024+0000 7f96abeb6b80 1 -- start start 2024-06-06T04:34:32.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.024+0000 7f96abeb6b80 1 Processor -- start 2024-06-06T04:34:32.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.024+0000 7f96abeb6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08cfaf20 0x564d08d06ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.024+0000 7f96abeb6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00640 0x564d08d03fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.024+0000 7f96abeb6b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08d00f60 0x564d08d044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.024+0000 7f96abeb6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564d08d07970 con 0x564d08d00640 2024-06-06T04:34:32.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.024+0000 7f96abeb6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564d08becbf0 con 0x564d08cfaf20 2024-06-06T04:34:32.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.024+0000 7f96abeb6b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564d08d07af0 con 0x564d08d00f60 2024-06-06T04:34:32.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.024+0000 7f96abeb6b80 10 client.0 _ll_register_callbacks cb 0x564d08be6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:34:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.024+0000 7f969a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00640 0x564d08d03fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.024+0000 7f969a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00640 0x564d08d03fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57128/0 (socket says 172.21.15.45:57128) 2024-06-06T04:34:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.024+0000 7f969a7fc700 1 -- 172.21.15.45:0/1852753450 learned_addr learned my addr 172.21.15.45:0/1852753450 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.024+0000 7f969affd700 1 --2- 172.21.15.45:0/1852753450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08cfaf20 0x564d08d06ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.024+0000 7f969b7fe700 1 --2- 172.21.15.45:0/1852753450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08d00f60 0x564d08d044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.025+0000 7f969a7fc700 1 -- 172.21.15.45:0/1852753450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08d00f60 msgr2=0x564d08d044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.025+0000 7f969a7fc700 1 --2- 172.21.15.45:0/1852753450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08d00f60 0x564d08d044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.025+0000 7f969a7fc700 1 -- 172.21.15.45:0/1852753450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08cfaf20 msgr2=0x564d08d06ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.025+0000 7f969a7fc700 1 --2- 172.21.15.45:0/1852753450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08cfaf20 0x564d08d06ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.025+0000 7f969a7fc700 1 -- 172.21.15.45:0/1852753450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d08b9b8d0 con 0x564d08d00640 2024-06-06T04:34:32.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.025+0000 7f969a7fc700 1 --2- 172.21.15.45:0/1852753450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00640 0x564d08d03fa0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9684005cd0 tx=0x7f9684005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6abbbb22eb361aa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:32.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.025+0000 7f96827fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:34:32.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.025+0000 7f96827fc700 1 -- 172.21.15.45:0/1852753450 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684028070 con 0x564d08d00640 2024-06-06T04:34:32.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.025+0000 7f96827fc700 1 -- 172.21.15.45:0/1852753450 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f968400b990 con 0x564d08d00640 2024-06-06T04:34:32.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.025+0000 7f96abeb6b80 1 -- 172.21.15.45:0/1852753450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96840037c0 con 0x564d08d00640 2024-06-06T04:34:32.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.025+0000 7f96827fc700 1 -- 172.21.15.45:0/1852753450 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f968403b050 con 0x564d08d00640 2024-06-06T04:34:32.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.025+0000 7f96abeb6b80 10 client.15858 fetch_fsmap learned FSMap version 30 2024-06-06T04:34:32.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.025+0000 7f96abeb6b80 1 -- 172.21.15.45:0/1852753450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564d08d122b0 con 0x564d08d00640 2024-06-06T04:34:32.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.025+0000 7f96827fc700 1 -- 172.21.15.45:0/1852753450 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96840037c0 con 0x564d08d00640 2024-06-06T04:34:32.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.026+0000 7f96827fc700 1 -- 172.21.15.45:0/1852753450 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f9684023720 con 0x564d08d00640 2024-06-06T04:34:32.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.026+0000 7f96abeb6b80 10 client.15858 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:34:32.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.026+0000 7f96abeb6b80 10 client.15858 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-06T04:34:32.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.026+0000 7f96abeb6b80 20 client.15858 populate_metadata read hostname 'smithi045' 2024-06-06T04:34:32.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.026+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/1852753450 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x564d08d14780 0x564d08d16b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.026+0000 7f96abeb6b80 4 client.15858 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:34:32.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.026+0000 7f96abeb6b80 1 -- 172.21.15.45:0/1852753450 --> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d08c59d10 con 0x564d08d14780 2024-06-06T04:34:32.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.026+0000 7f969affd700 1 --2- 172.21.15.45:0/1852753450 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x564d08d14780 0x564d08d16b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.026+0000 7f96827fc700 10 client.15858 ms_handle_connect on v2:172.21.15.154:6836/1140071744 2024-06-06T04:34:32.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.026+0000 7f969affd700 1 --2- 172.21.15.45:0/1852753450 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x564d08d14780 0x564d08d16b70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b289bab0167dab7 server_cookie=1c3f1759f3302ca4 in_seq=0 out_seq=0 2024-06-06T04:34:32.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.027+0000 7f96827fc700 1 -- 172.21.15.45:0/1852753450 <== mds.2 v2:172.21.15.154:6836/1140071744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564d08c59d10 con 0x564d08d14780 2024-06-06T04:34:32.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.027+0000 7f96827fc700 10 client.15858 handle_command_reply: tid=0 2024-06-06T04:34:32.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.027+0000 7f96abeb6b80 1 client.15858 shutdown 2024-06-06T04:34:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.027+0000 7f96abeb6b80 1 -- 172.21.15.45:0/1852753450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00640 msgr2=0x564d08d03fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.027+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/1852753450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00640 0x564d08d03fa0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9684005cd0 tx=0x7f9684005300 comp rx=0 tx=0).stop 2024-06-06T04:34:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.027+0000 7f96abeb6b80 1 -- 172.21.15.45:0/1852753450 shutdown_connections 2024-06-06T04:34:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.027+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/1852753450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08d00f60 0x564d08d044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.027+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/1852753450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08cfaf20 0x564d08d06ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.027+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/1852753450 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x564d08d14780 0x564d08d16b70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.027+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/1852753450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00640 0x564d08d03fa0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.027+0000 7f96abeb6b80 1 -- 172.21.15.45:0/1852753450 >> 172.21.15.45:0/1852753450 conn(0x564d08c25cc0 msgr2=0x564d08cf99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.027+0000 7f96827fc700 0 client.15858 ms_handle_reset on v2:172.21.15.154:6836/1140071744 2024-06-06T04:34:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.028+0000 7f96abeb6b80 1 -- 172.21.15.45:0/1852753450 shutdown_connections 2024-06-06T04:34:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.028+0000 7f96abeb6b80 1 -- 172.21.15.45:0/1852753450 wait complete. 2024-06-06T04:34:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.028+0000 7f96abeb6b80 20 client.15858 trim_cache size 0 max 16384 2024-06-06T04:34:32.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.043+0000 7f96abeb6b80 1 Processor -- start 2024-06-06T04:34:32.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.043+0000 7f96abeb6b80 1 -- start start 2024-06-06T04:34:32.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.043+0000 7f96abeb6b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08cfaf20 0x564d08cfa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.043+0000 7f96abeb6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08d00640 0x564d08d01400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.043+0000 7f96abeb6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00f60 0x564d08d01940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.043+0000 7f96abeb6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f969c1b2d50 con 0x564d08d00f60 2024-06-06T04:34:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.043+0000 7f96abeb6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f969c1b05b0 con 0x564d08d00640 2024-06-06T04:34:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.043+0000 7f96abeb6b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f969c1ae400 con 0x564d08cfaf20 2024-06-06T04:34:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.043+0000 7f969b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00f60 0x564d08d01940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.043+0000 7f969b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00f60 0x564d08d01940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57138/0 (socket says 172.21.15.45:57138) 2024-06-06T04:34:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.043+0000 7f969b7fe700 1 -- 172.21.15.45:0/2102721850 learned_addr learned my addr 172.21.15.45:0/2102721850 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.044+0000 7f969affd700 1 --2- 172.21.15.45:0/2102721850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08cfaf20 0x564d08cfa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.044+0000 7f969a7fc700 1 --2- 172.21.15.45:0/2102721850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08d00640 0x564d08d01400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.044+0000 7f969b7fe700 1 -- 172.21.15.45:0/2102721850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08cfaf20 msgr2=0x564d08cfa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.044+0000 7f969b7fe700 1 --2- 172.21.15.45:0/2102721850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08cfaf20 0x564d08cfa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.044+0000 7f969b7fe700 1 -- 172.21.15.45:0/2102721850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08d00640 msgr2=0x564d08d01400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.044+0000 7f969b7fe700 1 --2- 172.21.15.45:0/2102721850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08d00640 0x564d08d01400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.044+0000 7f969b7fe700 1 -- 172.21.15.45:0/2102721850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d08d07010 con 0x564d08d00f60 2024-06-06T04:34:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.044+0000 7f969b7fe700 1 --2- 172.21.15.45:0/2102721850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00f60 0x564d08d01940 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f96940090a0 tx=0x7f969400cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583a525bd90243b1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:32.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.044+0000 7f9690ff9700 1 -- 172.21.15.45:0/2102721850 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96940b3070 con 0x564d08d00f60 2024-06-06T04:34:32.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.044+0000 7f9690ff9700 1 -- 172.21.15.45:0/2102721850 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f969409b980 con 0x564d08d00f60 2024-06-06T04:34:32.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.044+0000 7f9690ff9700 1 -- 172.21.15.45:0/2102721850 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694099580 con 0x564d08d00f60 2024-06-06T04:34:32.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.045+0000 7f96abeb6b80 1 -- 172.21.15.45:0/2102721850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00f60 msgr2=0x564d08d01940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:32.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.045+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/2102721850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00f60 0x564d08d01940 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f96940090a0 tx=0x7f969400cfd0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.045+0000 7f96abeb6b80 1 -- 172.21.15.45:0/2102721850 shutdown_connections 2024-06-06T04:34:32.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.045+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/2102721850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08d00f60 0x564d08d01940 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.045+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/2102721850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08cfaf20 0x564d08cfa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.045+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/2102721850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08d00640 0x564d08d01400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.045+0000 7f96abeb6b80 1 -- 172.21.15.45:0/2102721850 >> 172.21.15.45:0/2102721850 conn(0x564d08c25cc0 msgr2=0x564d08cf5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:32.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.045+0000 7f96abeb6b80 1 -- 172.21.15.45:0/2102721850 shutdown_connections 2024-06-06T04:34:32.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.045+0000 7f96abeb6b80 1 -- 172.21.15.45:0/2102721850 wait complete. 2024-06-06T04:34:32.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.045+0000 7f96abeb6b80 1 -- start start 2024-06-06T04:34:32.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f96abeb6b80 1 Processor -- start 2024-06-06T04:34:32.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f96abeb6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08cfaf20 0x564d08d17150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f96abeb6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08d00640 0x564d08d17690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f96abeb6b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08d00f60 0x564d08d7d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f96abeb6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f969c1aea70 con 0x564d08cfaf20 2024-06-06T04:34:32.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f96abeb6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f969c1afee0 con 0x564d08d00640 2024-06-06T04:34:32.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f96abeb6b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f969c1af140 con 0x564d08d00f60 2024-06-06T04:34:32.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f96abeb6b80 10 client.0 _ll_register_callbacks cb 0x564d08be6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:34:32.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f969affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08cfaf20 0x564d08d17150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f969affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08cfaf20 0x564d08d17150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57148/0 (socket says 172.21.15.45:57148) 2024-06-06T04:34:32.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f969affd700 1 -- 172.21.15.45:0/1479552591 learned_addr learned my addr 172.21.15.45:0/1479552591 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:32.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f969b7fe700 1 --2- 172.21.15.45:0/1479552591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08d00f60 0x564d08d7d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f969a7fc700 1 --2- 172.21.15.45:0/1479552591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08d00640 0x564d08d17690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f969affd700 1 -- 172.21.15.45:0/1479552591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08d00f60 msgr2=0x564d08d7d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f969affd700 1 --2- 172.21.15.45:0/1479552591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08d00f60 0x564d08d7d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f969affd700 1 -- 172.21.15.45:0/1479552591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08d00640 msgr2=0x564d08d17690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f969affd700 1 --2- 172.21.15.45:0/1479552591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08d00640 0x564d08d17690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f969affd700 1 -- 172.21.15.45:0/1479552591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d08d7d5f0 con 0x564d08cfaf20 2024-06-06T04:34:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.046+0000 7f969a7fc700 1 --2- 172.21.15.45:0/1479552591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08d00640 0x564d08d17690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:34:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.047+0000 7f969affd700 1 --2- 172.21.15.45:0/1479552591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08cfaf20 0x564d08d17150 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x564d08d07b60 tx=0x7f968c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2460f33da07f70e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.047+0000 7f96827fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:34:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.047+0000 7f96827fc700 1 -- 172.21.15.45:0/1479552591 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c02a870 con 0x564d08cfaf20 2024-06-06T04:34:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.047+0000 7f96827fc700 1 -- 172.21.15.45:0/1479552591 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f968c02aa10 con 0x564d08cfaf20 2024-06-06T04:34:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.047+0000 7f96abeb6b80 1 -- 172.21.15.45:0/1479552591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f969409b980 con 0x564d08cfaf20 2024-06-06T04:34:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.047+0000 7f96827fc700 1 -- 172.21.15.45:0/1479552591 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f968c03c050 con 0x564d08cfaf20 2024-06-06T04:34:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.047+0000 7f96abeb6b80 10 client.15864 fetch_fsmap learned FSMap version 30 2024-06-06T04:34:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.047+0000 7f96abeb6b80 1 -- 172.21.15.45:0/1479552591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564d08d7e8f0 con 0x564d08cfaf20 2024-06-06T04:34:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.047+0000 7f96827fc700 1 -- 172.21.15.45:0/1479552591 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f969409b980 con 0x564d08cfaf20 2024-06-06T04:34:32.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.047+0000 7f96827fc700 1 -- 172.21.15.45:0/1479552591 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f968c011440 con 0x564d08cfaf20 2024-06-06T04:34:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.047+0000 7f96abeb6b80 10 client.15864 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:34:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.047+0000 7f96abeb6b80 10 client.15864 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-06T04:34:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.047+0000 7f96abeb6b80 20 client.15864 populate_metadata read hostname 'smithi045' 2024-06-06T04:34:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.048+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/1479552591 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x564d08d7f000 0x564d08d81420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.048+0000 7f96abeb6b80 4 client.15864 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T04:34:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.048+0000 7f96abeb6b80 1 -- 172.21.15.45:0/1479552591 --> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d08c54290 con 0x564d08d7f000 2024-06-06T04:34:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.048+0000 7f969a7fc700 1 --2- 172.21.15.45:0/1479552591 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x564d08d7f000 0x564d08d81420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:32.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.048+0000 7f969a7fc700 1 --2- 172.21.15.45:0/1479552591 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x564d08d7f000 0x564d08d81420 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=232d28ae46df73ad server_cookie=4c34f3901ff26c64 in_seq=0 out_seq=0 2024-06-06T04:34:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.048+0000 7f96827fc700 10 client.15864 ms_handle_connect on v2:172.21.15.154:6836/1140071744 2024-06-06T04:34:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.048+0000 7f96827fc700 1 -- 172.21.15.45:0/1479552591 <== mds.2 v2:172.21.15.154:6836/1140071744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d08c54290 con 0x564d08d7f000 2024-06-06T04:34:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.048+0000 7f96827fc700 10 client.15864 handle_command_reply: tid=0 2024-06-06T04:34:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.048+0000 7f96abeb6b80 1 client.15864 shutdown 2024-06-06T04:34:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.048+0000 7f96abeb6b80 1 -- 172.21.15.45:0/1479552591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08cfaf20 msgr2=0x564d08d17150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.048+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/1479552591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08cfaf20 0x564d08d17150 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x564d08d07b60 tx=0x7f968c0047c0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.049+0000 7f96abeb6b80 1 -- 172.21.15.45:0/1479552591 shutdown_connections 2024-06-06T04:34:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.049+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/1479552591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564d08d00f60 0x564d08d7d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.049+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/1479552591 >> [v2:172.21.15.154:6836/1140071744,v1:172.21.15.154:6837/1140071744] conn(0x564d08d7f000 0x564d08d81420 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.049+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/1479552591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d08cfaf20 0x564d08d17150 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.049+0000 7f96abeb6b80 1 --2- 172.21.15.45:0/1479552591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d08d00640 0x564d08d17690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.049+0000 7f96827fc700 0 client.15864 ms_handle_reset on v2:172.21.15.154:6836/1140071744 2024-06-06T04:34:32.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.049+0000 7f96abeb6b80 1 -- 172.21.15.45:0/1479552591 >> 172.21.15.45:0/1479552591 conn(0x564d08c25cc0 msgr2=0x564d08d01340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:32.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.049+0000 7f96abeb6b80 1 -- 172.21.15.45:0/1479552591 shutdown_connections 2024-06-06T04:34:32.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.049+0000 7f96abeb6b80 1 -- 172.21.15.45:0/1479552591 wait complete. 2024-06-06T04:34:32.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.049+0000 7f96abeb6b80 20 client.15864 trim_cache size 0 max 16384 2024-06-06T04:34:32.059 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-06-06T04:34:32.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.049+0000 7f96a1633700 1 -- 172.21.15.45:0/655960337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f96880a2c20 msgr2=0x7f96880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:32.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.049+0000 7f96a1633700 1 --2- 172.21.15.45:0/655960337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f96880a2c20 0x7f96880a50e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f968c0045b0 tx=0x7f968c00b3c0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.049+0000 7f96a1633700 1 -- 172.21.15.45:0/655960337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c108580 msgr2=0x7f969c102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:32.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.049+0000 7f96a1633700 1 --2- 172.21.15.45:0/655960337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c108580 0x7f969c102660 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f969400efc0 tx=0x7f969400c3a0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.050+0000 7f96a1633700 1 -- 172.21.15.45:0/655960337 shutdown_connections 2024-06-06T04:34:32.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.050+0000 7f96a1633700 1 --2- 172.21.15.45:0/655960337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c108580 0x7f969c102660 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.050+0000 7f96a1633700 1 --2- 172.21.15.45:0/655960337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f969c100a00 0x7f969c101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.050+0000 7f96a1633700 1 --2- 172.21.15.45:0/655960337 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f96880a2c20 0x7f96880a50e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.050+0000 7f96a1633700 1 --2- 172.21.15.45:0/655960337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f969c1013c0 0x7f969c102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:32.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.050+0000 7f96a1633700 1 -- 172.21.15.45:0/655960337 >> 172.21.15.45:0/655960337 conn(0x7f969c077990 msgr2=0x7f969c0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:32.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.050+0000 7f96a1633700 1 -- 172.21.15.45:0/655960337 shutdown_connections 2024-06-06T04:34:32.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:32.050+0000 7f96a1633700 1 -- 172.21.15.45:0/655960337 wait complete. 2024-06-06T04:34:32.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:32 smithi045 ceph-mon[32299]: pgmap v523: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-06-06T04:34:32.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:32 smithi086 ceph-mon[36053]: pgmap v523: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-06-06T04:34:32.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:32 smithi154 ceph-mon[36428]: pgmap v523: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-06-06T04:34:32.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-06T04:34:33.219 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:34:34.407 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:34 smithi045 ceph-mon[32299]: pgmap v524: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-06T04:34:34.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:34 smithi086 ceph-mon[36053]: pgmap v524: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-06T04:34:34.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:34 smithi154 ceph-mon[36428]: pgmap v524: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-06T04:34:34.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.997+0000 7fd570184700 1 -- 172.21.15.45:0/863439074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56806ac00 msgr2=0x7fd56806afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:34.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.997+0000 7fd570184700 1 --2- 172.21.15.45:0/863439074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56806ac00 0x7fd56806afe0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd564009990 tx=0x7fd564009ca0 comp rx=0 tx=0).stop 2024-06-06T04:34:34.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.997+0000 7fd570184700 1 -- 172.21.15.45:0/863439074 shutdown_connections 2024-06-06T04:34:34.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.997+0000 7fd570184700 1 --2- 172.21.15.45:0/863439074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd56810efd0 0x7fd5681113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:34.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.997+0000 7fd570184700 1 --2- 172.21.15.45:0/863439074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56806ac00 0x7fd56806afe0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:34.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.997+0000 7fd570184700 1 --2- 172.21.15.45:0/863439074 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd56806b520 0x7fd56810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:34.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.997+0000 7fd570184700 1 -- 172.21.15.45:0/863439074 >> 172.21.15.45:0/863439074 conn(0x7fd5680773e0 msgr2=0x7fd5680777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:34.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.997+0000 7fd570184700 1 -- 172.21.15.45:0/863439074 shutdown_connections 2024-06-06T04:34:34.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.997+0000 7fd570184700 1 -- 172.21.15.45:0/863439074 wait complete. 2024-06-06T04:34:34.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd570184700 1 Processor -- start 2024-06-06T04:34:34.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd570184700 1 -- start start 2024-06-06T04:34:35.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd570184700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd56806ac00 0x7fd56819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd570184700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd56806b520 0x7fd56819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd570184700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56810efd0 0x7fd56819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd570184700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd5681148d0 con 0x7fd56810efd0 2024-06-06T04:34:35.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd570184700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd568114750 con 0x7fd56806b520 2024-06-06T04:34:35.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd570184700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd568114a50 con 0x7fd56806ac00 2024-06-06T04:34:35.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd56e721700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56810efd0 0x7fd56819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd56e721700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56810efd0 0x7fd56819f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57170/0 (socket says 172.21.15.45:57170) 2024-06-06T04:34:35.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd56e721700 1 -- 172.21.15.45:0/369193693 learned_addr learned my addr 172.21.15.45:0/369193693 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:35.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd56df20700 1 --2- 172.21.15.45:0/369193693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd56806ac00 0x7fd56819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd56d71f700 1 --2- 172.21.15.45:0/369193693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd56806b520 0x7fd56819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd56e721700 1 -- 172.21.15.45:0/369193693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd56806ac00 msgr2=0x7fd56819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:35.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd56e721700 1 --2- 172.21.15.45:0/369193693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd56806ac00 0x7fd56819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd56e721700 1 -- 172.21.15.45:0/369193693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd56806b520 msgr2=0x7fd56819b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:35.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd56e721700 1 --2- 172.21.15.45:0/369193693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd56806b520 0x7fd56819b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.998+0000 7fd56e721700 1 -- 172.21.15.45:0/369193693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56819fd70 con 0x7fd56810efd0 2024-06-06T04:34:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.999+0000 7fd56e721700 1 --2- 172.21.15.45:0/369193693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56810efd0 0x7fd56819f630 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd56000ef90 tx=0x7fd56000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.999+0000 7fd55effd700 1 -- 172.21.15.45:0/369193693 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5600092e0 con 0x7fd56810efd0 2024-06-06T04:34:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.999+0000 7fd570184700 1 -- 172.21.15.45:0/369193693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5681a0000 con 0x7fd56810efd0 2024-06-06T04:34:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.999+0000 7fd55effd700 1 -- 172.21.15.45:0/369193693 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd560004bb0 con 0x7fd56810efd0 2024-06-06T04:34:35.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.999+0000 7fd55effd700 1 -- 172.21.15.45:0/369193693 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560004040 con 0x7fd56810efd0 2024-06-06T04:34:35.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:34.999+0000 7fd570184700 1 -- 172.21.15.45:0/369193693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5681a78e0 con 0x7fd56810efd0 2024-06-06T04:34:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.001+0000 7fd55effd700 1 -- 172.21.15.45:0/369193693 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd56001b030 con 0x7fd56810efd0 2024-06-06T04:34:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.002+0000 7fd55effd700 1 --2- 172.21.15.45:0/369193693 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd5540a2c20 0x7fd5540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.002+0000 7fd56df20700 1 --2- 172.21.15.45:0/369193693 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd5540a2c20 0x7fd5540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.002+0000 7fd55effd700 1 -- 172.21.15.45:0/369193693 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fd5600cee80 con 0x7fd56810efd0 2024-06-06T04:34:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.002+0000 7fd56df20700 1 --2- 172.21.15.45:0/369193693 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd5540a2c20 0x7fd5540a50e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd5640098e0 tx=0x7fd564005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.003+0000 7fd57aa07b80 1 Processor -- start 2024-06-06T04:34:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.003+0000 7fd57aa07b80 1 -- start start 2024-06-06T04:34:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.003+0000 7fd57aa07b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b147905f60 0x55b147906340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.003+0000 7fd57aa07b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b147906880 0x55b147900260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.003+0000 7fd57aa07b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1479007a0 0x55b147900bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.003+0000 7fd57aa07b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b1477f2f60 con 0x55b1479007a0 2024-06-06T04:34:35.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.003+0000 7fd57aa07b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b147851850 con 0x55b147906880 2024-06-06T04:34:35.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.003+0000 7fd57aa07b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b1477b9460 con 0x55b147905f60 2024-06-06T04:34:35.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.003+0000 7fd56e721700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1479007a0 0x55b147900bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.003+0000 7fd56e721700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1479007a0 0x55b147900bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57186/0 (socket says 172.21.15.45:57186) 2024-06-06T04:34:35.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.003+0000 7fd56e721700 1 -- 172.21.15.45:0/563239506 learned_addr learned my addr 172.21.15.45:0/563239506 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:35.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.003+0000 7fd56df20700 1 --2- 172.21.15.45:0/563239506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b147905f60 0x55b147906340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.003+0000 7fd56d71f700 1 --2- 172.21.15.45:0/563239506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b147906880 0x55b147900260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.004+0000 7fd56e721700 1 -- 172.21.15.45:0/563239506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b147905f60 msgr2=0x55b147906340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:35.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.004+0000 7fd56e721700 1 --2- 172.21.15.45:0/563239506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b147905f60 0x55b147906340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.004+0000 7fd56e721700 1 -- 172.21.15.45:0/563239506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b147906880 msgr2=0x55b147900260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:35.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.004+0000 7fd56e721700 1 --2- 172.21.15.45:0/563239506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b147906880 0x55b147900260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.004+0000 7fd56e721700 1 -- 172.21.15.45:0/563239506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1477a08d0 con 0x55b1479007a0 2024-06-06T04:34:35.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.004+0000 7fd56e721700 1 --2- 172.21.15.45:0/563239506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1479007a0 0x55b147900bc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd560097d80 tx=0x7fd56009ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78104b010b4344b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:35.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.004+0000 7fd553fff700 1 -- 172.21.15.45:0/563239506 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56009b3b0 con 0x55b1479007a0 2024-06-06T04:34:35.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.004+0000 7fd553fff700 1 -- 172.21.15.45:0/563239506 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd56009f760 con 0x55b1479007a0 2024-06-06T04:34:35.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.004+0000 7fd553fff700 1 -- 172.21.15.45:0/563239506 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5600a3d60 con 0x55b1479007a0 2024-06-06T04:34:35.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.005+0000 7fd57aa07b80 1 -- 172.21.15.45:0/563239506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1479007a0 msgr2=0x55b147900bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:35.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.005+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/563239506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1479007a0 0x55b147900bc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd560097d80 tx=0x7fd56009ad10 comp rx=0 tx=0).stop 2024-06-06T04:34:35.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.005+0000 7fd57aa07b80 1 -- 172.21.15.45:0/563239506 shutdown_connections 2024-06-06T04:34:35.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.005+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/563239506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1479007a0 0x55b147900bc0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.005+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/563239506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b147905f60 0x55b147906340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.005+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/563239506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b147906880 0x55b147900260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.005+0000 7fd57aa07b80 1 -- 172.21.15.45:0/563239506 >> 172.21.15.45:0/563239506 conn(0x55b14782d080 msgr2=0x55b1478fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:35.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.005+0000 7fd57aa07b80 1 -- 172.21.15.45:0/563239506 shutdown_connections 2024-06-06T04:34:35.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.005+0000 7fd57aa07b80 1 -- 172.21.15.45:0/563239506 wait complete. 2024-06-06T04:34:35.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.005+0000 7fd57aa07b80 1 -- start start 2024-06-06T04:34:35.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.005+0000 7fd57aa07b80 1 Processor -- start 2024-06-06T04:34:35.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd57aa07b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1479007a0 0x55b14790c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd57aa07b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b147905f60 0x55b147909910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd57aa07b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b147909e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd57aa07b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b1477f1fc0 con 0x55b147906880 2024-06-06T04:34:35.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd57aa07b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b14784d810 con 0x55b1479007a0 2024-06-06T04:34:35.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd57aa07b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b14790d260 con 0x55b147905f60 2024-06-06T04:34:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd57aa07b80 10 client.0 _ll_register_callbacks cb 0x55b1477e75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:34:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd56e721700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b147909e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd56e721700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b147909e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57188/0 (socket says 172.21.15.45:57188) 2024-06-06T04:34:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd56e721700 1 -- 172.21.15.45:0/1392634629 learned_addr learned my addr 172.21.15.45:0/1392634629 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd56d71f700 1 --2- 172.21.15.45:0/1392634629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b147905f60 0x55b147909910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd56df20700 1 --2- 172.21.15.45:0/1392634629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1479007a0 0x55b14790c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd56e721700 1 -- 172.21.15.45:0/1392634629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b147905f60 msgr2=0x55b147909910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd56e721700 1 --2- 172.21.15.45:0/1392634629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b147905f60 0x55b147909910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd56e721700 1 -- 172.21.15.45:0/1392634629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1479007a0 msgr2=0x55b14790c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd56e721700 1 --2- 172.21.15.45:0/1392634629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1479007a0 0x55b14790c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd56e721700 1 -- 172.21.15.45:0/1392634629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b147817750 con 0x55b147906880 2024-06-06T04:34:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.006+0000 7fd56df20700 1 --2- 172.21.15.45:0/1392634629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1479007a0 0x55b14790c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:34:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.007+0000 7fd56e721700 1 --2- 172.21.15.45:0/1392634629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b147909e50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd5600cc930 tx=0x7fd56009bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3301593e6fa270 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.007+0000 7fd5517fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:34:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.007+0000 7fd5517fa700 1 -- 172.21.15.45:0/1392634629 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560099450 con 0x55b147906880 2024-06-06T04:34:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.007+0000 7fd5517fa700 1 -- 172.21.15.45:0/1392634629 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fd5600996b0 con 0x55b147906880 2024-06-06T04:34:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.007+0000 7fd57aa07b80 1 -- 172.21.15.45:0/1392634629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd56009f760 con 0x55b147906880 2024-06-06T04:34:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.007+0000 7fd5517fa700 1 -- 172.21.15.45:0/1392634629 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd56009a800 con 0x55b147906880 2024-06-06T04:34:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.007+0000 7fd57aa07b80 10 client.15876 fetch_fsmap learned FSMap version 30 2024-06-06T04:34:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.007+0000 7fd57aa07b80 1 -- 172.21.15.45:0/1392634629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b147917a10 con 0x55b147906880 2024-06-06T04:34:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.007+0000 7fd5517fa700 1 -- 172.21.15.45:0/1392634629 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd56009f760 con 0x55b147906880 2024-06-06T04:34:35.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.007+0000 7fd5517fa700 1 -- 172.21.15.45:0/1392634629 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fd56007d080 con 0x55b147906880 2024-06-06T04:34:35.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.007+0000 7fd57aa07b80 10 client.15876 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:34:35.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.007+0000 7fd57aa07b80 10 client.15876 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T04:34:35.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.007+0000 7fd57aa07b80 20 client.15876 populate_metadata read hostname 'smithi045' 2024-06-06T04:34:35.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.007+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/1392634629 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55b147919ee0 0x55b14791c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.007+0000 7fd57aa07b80 4 client.15876 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:34:35.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.007+0000 7fd57aa07b80 1 -- 172.21.15.45:0/1392634629 --> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b14785ed50 con 0x55b147919ee0 2024-06-06T04:34:35.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.008+0000 7fd56df20700 1 --2- 172.21.15.45:0/1392634629 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55b147919ee0 0x55b14791c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.008+0000 7fd56df20700 1 --2- 172.21.15.45:0/1392634629 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55b147919ee0 0x55b14791c2d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5cd981d4248df4c server_cookie=b539abba658eda7e in_seq=0 out_seq=0 2024-06-06T04:34:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.008+0000 7fd5517fa700 10 client.15876 ms_handle_connect on v2:172.21.15.154:6838/982197202 2024-06-06T04:34:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.009+0000 7fd5517fa700 1 -- 172.21.15.45:0/1392634629 <== mds.1 v2:172.21.15.154:6838/982197202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b14785ed50 con 0x55b147919ee0 2024-06-06T04:34:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.009+0000 7fd5517fa700 10 client.15876 handle_command_reply: tid=0 2024-06-06T04:34:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.009+0000 7fd57aa07b80 1 client.15876 shutdown 2024-06-06T04:34:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.009+0000 7fd57aa07b80 1 -- 172.21.15.45:0/1392634629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 msgr2=0x55b147909e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.009+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/1392634629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b147909e50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd5600cc930 tx=0x7fd56009bbe0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.009+0000 7fd57aa07b80 1 -- 172.21.15.45:0/1392634629 shutdown_connections 2024-06-06T04:34:35.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.009+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/1392634629 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55b147919ee0 0x55b14791c2d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.009+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/1392634629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b147909e50 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.009+0000 7fd5517fa700 0 client.15876 ms_handle_reset on v2:172.21.15.154:6838/982197202 2024-06-06T04:34:35.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.009+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/1392634629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1479007a0 0x55b14790c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.009+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/1392634629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b147905f60 0x55b147909910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.009+0000 7fd57aa07b80 1 -- 172.21.15.45:0/1392634629 >> 172.21.15.45:0/1392634629 conn(0x55b14782d080 msgr2=0x55b1478fa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:35.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.009+0000 7fd57aa07b80 1 -- 172.21.15.45:0/1392634629 shutdown_connections 2024-06-06T04:34:35.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.009+0000 7fd57aa07b80 1 -- 172.21.15.45:0/1392634629 wait complete. 2024-06-06T04:34:35.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.009+0000 7fd57aa07b80 20 client.15876 trim_cache size 0 max 16384 2024-06-06T04:34:35.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd57aa07b80 1 Processor -- start 2024-06-06T04:34:35.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd57aa07b80 1 -- start start 2024-06-06T04:34:35.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd57aa07b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1479007a0 0x55b147900260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd57aa07b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b147905f60 0x55b1479074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd57aa07b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b14790b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd57aa07b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd5681b2e30 con 0x55b147906880 2024-06-06T04:34:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd57aa07b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5681b0690 con 0x55b147905f60 2024-06-06T04:34:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd57aa07b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd5681ae4e0 con 0x55b1479007a0 2024-06-06T04:34:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd56e721700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b14790b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd56e721700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b14790b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57196/0 (socket says 172.21.15.45:57196) 2024-06-06T04:34:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd56e721700 1 -- 172.21.15.45:0/2342640776 learned_addr learned my addr 172.21.15.45:0/2342640776 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd56d71f700 1 --2- 172.21.15.45:0/2342640776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b147905f60 0x55b1479074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd56df20700 1 --2- 172.21.15.45:0/2342640776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1479007a0 0x55b147900260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd56e721700 1 -- 172.21.15.45:0/2342640776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1479007a0 msgr2=0x55b147900260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd56e721700 1 --2- 172.21.15.45:0/2342640776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1479007a0 0x55b147900260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd56e721700 1 -- 172.21.15.45:0/2342640776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b147905f60 msgr2=0x55b1479074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:35.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd56e721700 1 --2- 172.21.15.45:0/2342640776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b147905f60 0x55b1479074f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd56e721700 1 -- 172.21.15.45:0/2342640776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b147907e80 con 0x55b147906880 2024-06-06T04:34:35.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.025+0000 7fd56df20700 1 --2- 172.21.15.45:0/2342640776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1479007a0 0x55b147900260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:34:35.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.026+0000 7fd56e721700 1 --2- 172.21.15.45:0/2342640776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b14790b850 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd560009fd0 tx=0x7fd5600cc610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97161a49fc17ab3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:35.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.026+0000 7fd5537fe700 1 -- 172.21.15.45:0/2342640776 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56009f760 con 0x55b147906880 2024-06-06T04:34:35.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.026+0000 7fd5537fe700 1 -- 172.21.15.45:0/2342640776 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd560099450 con 0x55b147906880 2024-06-06T04:34:35.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.026+0000 7fd5537fe700 1 -- 172.21.15.45:0/2342640776 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56009a3e0 con 0x55b147906880 2024-06-06T04:34:35.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.026+0000 7fd57aa07b80 1 -- 172.21.15.45:0/2342640776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 msgr2=0x55b14790b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:35.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.026+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/2342640776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b14790b850 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd560009fd0 tx=0x7fd5600cc610 comp rx=0 tx=0).stop 2024-06-06T04:34:35.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.026+0000 7fd57aa07b80 1 -- 172.21.15.45:0/2342640776 shutdown_connections 2024-06-06T04:34:35.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.026+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/2342640776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b14790b850 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.026+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/2342640776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b1479007a0 0x55b147900260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.026+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/2342640776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b147905f60 0x55b1479074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.026+0000 7fd57aa07b80 1 -- 172.21.15.45:0/2342640776 >> 172.21.15.45:0/2342640776 conn(0x55b14782d080 msgr2=0x55b14790b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:35.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.026+0000 7fd57aa07b80 1 -- 172.21.15.45:0/2342640776 shutdown_connections 2024-06-06T04:34:35.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.026+0000 7fd57aa07b80 1 -- 172.21.15.45:0/2342640776 wait complete. 2024-06-06T04:34:35.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.027+0000 7fd57aa07b80 1 -- start start 2024-06-06T04:34:35.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.027+0000 7fd57aa07b80 1 Processor -- start 2024-06-06T04:34:35.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.027+0000 7fd57aa07b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1479007a0 0x55b14791fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.027+0000 7fd57aa07b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b147905f60 0x55b14791cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.027+0000 7fd57aa07b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b14791d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.027+0000 7fd57aa07b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd5681aeb50 con 0x55b147906880 2024-06-06T04:34:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.027+0000 7fd57aa07b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5681affc0 con 0x55b1479007a0 2024-06-06T04:34:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.027+0000 7fd57aa07b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd5681af220 con 0x55b147905f60 2024-06-06T04:34:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.027+0000 7fd57aa07b80 10 client.0 _ll_register_callbacks cb 0x55b1477e75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:34:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.027+0000 7fd56e721700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b14791d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.027+0000 7fd56e721700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b14791d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57208/0 (socket says 172.21.15.45:57208) 2024-06-06T04:34:35.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.027+0000 7fd56e721700 1 -- 172.21.15.45:0/2774248417 learned_addr learned my addr 172.21.15.45:0/2774248417 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:35.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.027+0000 7fd56df20700 1 --2- 172.21.15.45:0/2774248417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1479007a0 0x55b14791fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.028+0000 7fd56d71f700 1 --2- 172.21.15.45:0/2774248417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b147905f60 0x55b14791cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.028+0000 7fd56e721700 1 -- 172.21.15.45:0/2774248417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b147905f60 msgr2=0x55b14791cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:35.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.028+0000 7fd56e721700 1 --2- 172.21.15.45:0/2774248417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b147905f60 0x55b14791cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.028+0000 7fd56e721700 1 -- 172.21.15.45:0/2774248417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1479007a0 msgr2=0x55b14791fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:35.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.028+0000 7fd56e721700 1 --2- 172.21.15.45:0/2774248417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1479007a0 0x55b14791fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.028+0000 7fd56e721700 1 -- 172.21.15.45:0/2774248417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b147983820 con 0x55b147906880 2024-06-06T04:34:35.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.028+0000 7fd56e721700 1 --2- 172.21.15.45:0/2774248417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b14791d670 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd56009ba10 tx=0x7fd5600998c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c8eb9a5aa539c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:35.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.028+0000 7fd5517fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:34:35.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.028+0000 7fd5517fa700 1 -- 172.21.15.45:0/2774248417 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56009f760 con 0x55b147906880 2024-06-06T04:34:35.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.028+0000 7fd5517fa700 1 -- 172.21.15.45:0/2774248417 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fd56009a5a0 con 0x55b147906880 2024-06-06T04:34:35.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.028+0000 7fd57aa07b80 1 -- 172.21.15.45:0/2774248417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd560099450 con 0x55b147906880 2024-06-06T04:34:35.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.028+0000 7fd5517fa700 1 -- 172.21.15.45:0/2774248417 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd5600bb0b0 con 0x55b147906880 2024-06-06T04:34:35.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.028+0000 7fd57aa07b80 10 client.15882 fetch_fsmap learned FSMap version 30 2024-06-06T04:34:35.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.028+0000 7fd57aa07b80 1 -- 172.21.15.45:0/2774248417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b147983820 con 0x55b147906880 2024-06-06T04:34:35.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.028+0000 7fd5517fa700 1 -- 172.21.15.45:0/2774248417 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd560099450 con 0x55b147906880 2024-06-06T04:34:35.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.028+0000 7fd5517fa700 1 -- 172.21.15.45:0/2774248417 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fd560003a00 con 0x55b147906880 2024-06-06T04:34:35.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.029+0000 7fd57aa07b80 10 client.15882 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:34:35.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.029+0000 7fd57aa07b80 10 client.15882 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-06T04:34:35.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.029+0000 7fd57aa07b80 20 client.15882 populate_metadata read hostname 'smithi045' 2024-06-06T04:34:35.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.029+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/2774248417 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55b147984b90 0x55b147986f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:35.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.029+0000 7fd57aa07b80 4 client.15882 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T04:34:35.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.029+0000 7fd57aa07b80 1 -- 172.21.15.45:0/2774248417 --> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1478592d0 con 0x55b147984b90 2024-06-06T04:34:35.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.029+0000 7fd56df20700 1 --2- 172.21.15.45:0/2774248417 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55b147984b90 0x55b147986f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:35.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.029+0000 7fd56df20700 1 --2- 172.21.15.45:0/2774248417 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55b147984b90 0x55b147986f80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c8324f459d22b13 server_cookie=5b11eb5c7e9fc240 in_seq=0 out_seq=0 2024-06-06T04:34:35.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.029+0000 7fd5517fa700 10 client.15882 ms_handle_connect on v2:172.21.15.154:6838/982197202 2024-06-06T04:34:35.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.029+0000 7fd5517fa700 1 -- 172.21.15.45:0/2774248417 <== mds.1 v2:172.21.15.154:6838/982197202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1478592d0 con 0x55b147984b90 2024-06-06T04:34:35.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.029+0000 7fd5517fa700 10 client.15882 handle_command_reply: tid=0 2024-06-06T04:34:35.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.029+0000 7fd57aa07b80 1 client.15882 shutdown 2024-06-06T04:34:35.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.029+0000 7fd57aa07b80 1 -- 172.21.15.45:0/2774248417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 msgr2=0x55b14791d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:35.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.029+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/2774248417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b14791d670 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd56009ba10 tx=0x7fd5600998c0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.030+0000 7fd57aa07b80 1 -- 172.21.15.45:0/2774248417 shutdown_connections 2024-06-06T04:34:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.030+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/2774248417 >> [v2:172.21.15.154:6838/982197202,v1:172.21.15.154:6839/982197202] conn(0x55b147984b90 0x55b147986f80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.030+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/2774248417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b147906880 0x55b14791d670 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.030+0000 7fd5517fa700 0 client.15882 ms_handle_reset on v2:172.21.15.154:6838/982197202 2024-06-06T04:34:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.030+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/2774248417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1479007a0 0x55b14791fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.030+0000 7fd57aa07b80 1 --2- 172.21.15.45:0/2774248417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b147905f60 0x55b14791cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.030+0000 7fd57aa07b80 1 -- 172.21.15.45:0/2774248417 >> 172.21.15.45:0/2774248417 conn(0x55b14782d080 msgr2=0x55b14790b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.030+0000 7fd57aa07b80 1 -- 172.21.15.45:0/2774248417 shutdown_connections 2024-06-06T04:34:35.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.030+0000 7fd57aa07b80 1 -- 172.21.15.45:0/2774248417 wait complete. 2024-06-06T04:34:35.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.030+0000 7fd57aa07b80 20 client.15882 trim_cache size 0 max 16384 2024-06-06T04:34:35.040 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-06-06T04:34:35.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.030+0000 7fd570184700 1 -- 172.21.15.45:0/369193693 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd5540a2c20 msgr2=0x7fd5540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:35.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.030+0000 7fd570184700 1 --2- 172.21.15.45:0/369193693 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd5540a2c20 0x7fd5540a50e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd5640098e0 tx=0x7fd564005960 comp rx=0 tx=0).stop 2024-06-06T04:34:35.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.030+0000 7fd570184700 1 -- 172.21.15.45:0/369193693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56810efd0 msgr2=0x7fd56819f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:35.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.030+0000 7fd570184700 1 --2- 172.21.15.45:0/369193693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56810efd0 0x7fd56819f630 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd56000ef90 tx=0x7fd56000c3a0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.031+0000 7fd570184700 1 -- 172.21.15.45:0/369193693 shutdown_connections 2024-06-06T04:34:35.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.031+0000 7fd570184700 1 --2- 172.21.15.45:0/369193693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56810efd0 0x7fd56819f630 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.031+0000 7fd570184700 1 --2- 172.21.15.45:0/369193693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd56806ac00 0x7fd56819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.031+0000 7fd570184700 1 --2- 172.21.15.45:0/369193693 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd5540a2c20 0x7fd5540a50e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.031+0000 7fd570184700 1 --2- 172.21.15.45:0/369193693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd56806b520 0x7fd56819b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:35.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.031+0000 7fd570184700 1 -- 172.21.15.45:0/369193693 >> 172.21.15.45:0/369193693 conn(0x7fd5680773e0 msgr2=0x7fd5681106e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:35.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.031+0000 7fd570184700 1 -- 172.21.15.45:0/369193693 shutdown_connections 2024-06-06T04:34:35.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:35.031+0000 7fd570184700 1 -- 172.21.15.45:0/369193693 wait complete. 2024-06-06T04:34:35.765 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-06T04:34:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:34:36.378 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:36 smithi045 ceph-mon[32299]: pgmap v525: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-06T04:34:36.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:36 smithi086 ceph-mon[36053]: pgmap v525: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-06T04:34:36.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:36 smithi154 ceph-mon[36428]: pgmap v525: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-06T04:34:37.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.890+0000 7f4488d26700 1 -- 172.21.15.45:0/1806488384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4484106c00 msgr2=0x7f4484106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:37.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.890+0000 7f4488d26700 1 --2- 172.21.15.45:0/1806488384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4484106c00 0x7f4484106fe0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f44740099c0 tx=0x7f4474009cd0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.890+0000 7f4488d26700 1 -- 172.21.15.45:0/1806488384 shutdown_connections 2024-06-06T04:34:37.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.890+0000 7f4488d26700 1 --2- 172.21.15.45:0/1806488384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4484101510 0x7f448410d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.890+0000 7f4488d26700 1 --2- 172.21.15.45:0/1806488384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4484106c00 0x7f4484106fe0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.890+0000 7f4488d26700 1 --2- 172.21.15.45:0/1806488384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4484100b50 0x7f4484100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.890+0000 7f4488d26700 1 -- 172.21.15.45:0/1806488384 >> 172.21.15.45:0/1806488384 conn(0x7f4484077990 msgr2=0x7f44840fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:37.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.890+0000 7f4488d26700 1 -- 172.21.15.45:0/1806488384 shutdown_connections 2024-06-06T04:34:37.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.890+0000 7f4488d26700 1 -- 172.21.15.45:0/1806488384 wait complete. 2024-06-06T04:34:37.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f4488d26700 1 Processor -- start 2024-06-06T04:34:37.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f4488d26700 1 -- start start 2024-06-06T04:34:37.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f4488d26700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4484100b50 0x7f448419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f4488d26700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4484101510 0x7f448419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f4488d26700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4484106c00 0x7f4484198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f4488d26700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f44841115e0 con 0x7f4484100b50 2024-06-06T04:34:37.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f4488d26700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4484111460 con 0x7f4484106c00 2024-06-06T04:34:37.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f4488d26700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4484111760 con 0x7f4484101510 2024-06-06T04:34:37.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f448259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4484100b50 0x7f448419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f448259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4484100b50 0x7f448419e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44578/0 (socket says 172.21.15.45:44578) 2024-06-06T04:34:37.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f448259c700 1 -- 172.21.15.45:0/841587958 learned_addr learned my addr 172.21.15.45:0/841587958 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:37.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f4481d9b700 1 --2- 172.21.15.45:0/841587958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4484101510 0x7f448419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f4482d9d700 1 --2- 172.21.15.45:0/841587958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4484106c00 0x7f4484198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f448259c700 1 -- 172.21.15.45:0/841587958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4484101510 msgr2=0x7f448419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:37.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f448259c700 1 --2- 172.21.15.45:0/841587958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4484101510 0x7f448419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f448259c700 1 -- 172.21.15.45:0/841587958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4484106c00 msgr2=0x7f4484198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:37.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f448259c700 1 --2- 172.21.15.45:0/841587958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4484106c00 0x7f4484198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.891+0000 7f448259c700 1 -- 172.21.15.45:0/841587958 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4484199090 con 0x7f4484100b50 2024-06-06T04:34:37.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.892+0000 7f448259c700 1 --2- 172.21.15.45:0/841587958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4484100b50 0x7f448419e770 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f44740098e0 tx=0x7f4474005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:37.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.892+0000 7f44737fe700 1 -- 172.21.15.45:0/841587958 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474027070 con 0x7f4484100b50 2024-06-06T04:34:37.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.892+0000 7f44737fe700 1 -- 172.21.15.45:0/841587958 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4474003f80 con 0x7f4484100b50 2024-06-06T04:34:37.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.892+0000 7f4488d26700 1 -- 172.21.15.45:0/841587958 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4484199320 con 0x7f4484100b50 2024-06-06T04:34:37.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.892+0000 7f44737fe700 1 -- 172.21.15.45:0/841587958 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474022d50 con 0x7f4484100b50 2024-06-06T04:34:37.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.892+0000 7f4488d26700 1 -- 172.21.15.45:0/841587958 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44841a5640 con 0x7f4484100b50 2024-06-06T04:34:37.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.894+0000 7f44737fe700 1 -- 172.21.15.45:0/841587958 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f447402a070 con 0x7f4484100b50 2024-06-06T04:34:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.895+0000 7f44737fe700 1 --2- 172.21.15.45:0/841587958 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f446c0a2ca0 0x7f446c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.895+0000 7f4481d9b700 1 --2- 172.21.15.45:0/841587958 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f446c0a2ca0 0x7f446c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.895+0000 7f44737fe700 1 -- 172.21.15.45:0/841587958 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f44740d02b0 con 0x7f4484100b50 2024-06-06T04:34:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.895+0000 7f4481d9b700 1 --2- 172.21.15.45:0/841587958 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f446c0a2ca0 0x7f446c0a5160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4484106200 tx=0x7f4478009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.896+0000 7f44935a9b80 1 Processor -- start 2024-06-06T04:34:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.896+0000 7f44935a9b80 1 -- start start 2024-06-06T04:34:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.896+0000 7f44935a9b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 0x558c558423b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.896+0000 7f44935a9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c558428f0 0x558c558466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.896+0000 7f44935a9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 0x558c55847090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.896+0000 7f44935a9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558c5578f810 con 0x558c558428f0 2024-06-06T04:34:37.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.896+0000 7f44935a9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558c55737720 con 0x558c55846c30 2024-06-06T04:34:37.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.896+0000 7f44935a9b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558c5578b7d0 con 0x558c55841fd0 2024-06-06T04:34:37.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.896+0000 7f4481d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c558428f0 0x558c558466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.896+0000 7f4481d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c558428f0 0x558c558466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44592/0 (socket says 172.21.15.45:44592) 2024-06-06T04:34:37.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.896+0000 7f4481d9b700 1 -- 172.21.15.45:0/1250460342 learned_addr learned my addr 172.21.15.45:0/1250460342 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:37.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.896+0000 7f448259c700 1 --2- 172.21.15.45:0/1250460342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 0x558c558423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.896+0000 7f4482d9d700 1 --2- 172.21.15.45:0/1250460342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 0x558c55847090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.897+0000 7f4481d9b700 1 -- 172.21.15.45:0/1250460342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 msgr2=0x558c558423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:37.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.897+0000 7f4481d9b700 1 --2- 172.21.15.45:0/1250460342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 0x558c558423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.897+0000 7f4481d9b700 1 -- 172.21.15.45:0/1250460342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 msgr2=0x558c55847090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:37.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.897+0000 7f4481d9b700 1 --2- 172.21.15.45:0/1250460342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 0x558c55847090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.897+0000 7f4481d9b700 1 -- 172.21.15.45:0/1250460342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c5572df50 con 0x558c558428f0 2024-06-06T04:34:37.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.897+0000 7f4482d9d700 1 --2- 172.21.15.45:0/1250460342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 0x558c55847090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:34:37.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.897+0000 7f4481d9b700 1 --2- 172.21.15.45:0/1250460342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c558428f0 0x558c558466f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4478009fd0 tx=0x7f4478011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643db1538b94abb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:37.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.897+0000 7f4470ff9700 1 -- 172.21.15.45:0/1250460342 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447801b040 con 0x558c558428f0 2024-06-06T04:34:37.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.897+0000 7f4470ff9700 1 -- 172.21.15.45:0/1250460342 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4478024070 con 0x558c558428f0 2024-06-06T04:34:37.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.897+0000 7f4470ff9700 1 -- 172.21.15.45:0/1250460342 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44780136d0 con 0x558c558428f0 2024-06-06T04:34:37.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.898+0000 7f44935a9b80 1 -- 172.21.15.45:0/1250460342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c558428f0 msgr2=0x558c558466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:37.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.898+0000 7f44935a9b80 1 --2- 172.21.15.45:0/1250460342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c558428f0 0x558c558466f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4478009fd0 tx=0x7f4478011000 comp rx=0 tx=0).stop 2024-06-06T04:34:37.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.898+0000 7f44935a9b80 1 -- 172.21.15.45:0/1250460342 shutdown_connections 2024-06-06T04:34:37.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.898+0000 7f44935a9b80 1 --2- 172.21.15.45:0/1250460342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 0x558c55847090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.898+0000 7f44935a9b80 1 --2- 172.21.15.45:0/1250460342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 0x558c558423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.898+0000 7f44935a9b80 1 --2- 172.21.15.45:0/1250460342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c558428f0 0x558c558466f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.898+0000 7f44935a9b80 1 -- 172.21.15.45:0/1250460342 >> 172.21.15.45:0/1250460342 conn(0x558c55768cc0 msgr2=0x558c55838850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:37.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.898+0000 7f44935a9b80 1 -- 172.21.15.45:0/1250460342 shutdown_connections 2024-06-06T04:34:37.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.898+0000 7f44935a9b80 1 -- 172.21.15.45:0/1250460342 wait complete. 2024-06-06T04:34:37.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.898+0000 7f44935a9b80 1 -- start start 2024-06-06T04:34:37.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.898+0000 7f44935a9b80 1 Processor -- start 2024-06-06T04:34:37.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f44935a9b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 0x558c5584e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f44935a9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c558428f0 0x558c5584b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f44935a9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c55846c30 0x558c5584ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f44935a9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558c5584ed70 con 0x558c55846c30 2024-06-06T04:34:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f44935a9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558c5572fbf0 con 0x558c558428f0 2024-06-06T04:34:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f44935a9b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558c5584eef0 con 0x558c55841fd0 2024-06-06T04:34:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f44935a9b80 10 client.0 _ll_register_callbacks cb 0x558c55729920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:34:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f4482d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c55846c30 0x558c5584ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f448259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 0x558c5584e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f4482d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c55846c30 0x558c5584ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44606/0 (socket says 172.21.15.45:44606) 2024-06-06T04:34:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f448259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 0x558c5584e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.45:51728/0 (socket says 172.21.15.45:51728) 2024-06-06T04:34:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f4482d9d700 1 -- 172.21.15.45:0/2357530741 learned_addr learned my addr 172.21.15.45:0/2357530741 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f4481d9b700 1 --2- 172.21.15.45:0/2357530741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c558428f0 0x558c5584b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f4482d9d700 1 -- 172.21.15.45:0/2357530741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 msgr2=0x558c5584e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f4482d9d700 1 --2- 172.21.15.45:0/2357530741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 0x558c5584e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f4482d9d700 1 -- 172.21.15.45:0/2357530741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c558428f0 msgr2=0x558c5584b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f4482d9d700 1 --2- 172.21.15.45:0/2357530741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c558428f0 0x558c5584b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f4482d9d700 1 -- 172.21.15.45:0/2357530741 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c556de8d0 con 0x558c55846c30 2024-06-06T04:34:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f448259c700 1 --2- 172.21.15.45:0/2357530741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 0x558c5584e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:34:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f4482d9d700 1 --2- 172.21.15.45:0/2357530741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c55846c30 0x558c5584ba20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f447c004140 tx=0x7f447c004450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c37a972640c693 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.899+0000 7f4469ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:34:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.900+0000 7f4469ffb700 1 -- 172.21.15.45:0/2357530741 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c01a020 con 0x558c55846c30 2024-06-06T04:34:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.900+0000 7f4469ffb700 1 -- 172.21.15.45:0/2357530741 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f447c0079d0 con 0x558c55846c30 2024-06-06T04:34:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.900+0000 7f44935a9b80 1 -- 172.21.15.45:0/2357530741 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4478024070 con 0x558c55846c30 2024-06-06T04:34:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.900+0000 7f4469ffb700 1 -- 172.21.15.45:0/2357530741 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f447c025050 con 0x558c55846c30 2024-06-06T04:34:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.900+0000 7f44935a9b80 10 client.15894 fetch_fsmap learned FSMap version 30 2024-06-06T04:34:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.900+0000 7f44935a9b80 1 -- 172.21.15.45:0/2357530741 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558c55859760 con 0x558c55846c30 2024-06-06T04:34:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.900+0000 7f4469ffb700 1 -- 172.21.15.45:0/2357530741 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4478024070 con 0x558c55846c30 2024-06-06T04:34:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.900+0000 7f4469ffb700 1 -- 172.21.15.45:0/2357530741 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f447c0153c0 con 0x558c55846c30 2024-06-06T04:34:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.900+0000 7f44935a9b80 10 client.15894 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:34:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.900+0000 7f44935a9b80 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:34:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.900+0000 7f44935a9b80 20 client.15894 populate_metadata read hostname 'smithi045' 2024-06-06T04:34:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.900+0000 7f44935a9b80 1 --2- 172.21.15.45:0/2357530741 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x558c5585bda0 0x558c5585e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.900+0000 7f44935a9b80 4 client.15894 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T04:34:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.900+0000 7f44935a9b80 1 -- 172.21.15.45:0/2357530741 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c5579cd10 con 0x558c5585bda0 2024-06-06T04:34:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.901+0000 7f448259c700 1 --2- 172.21.15.45:0/2357530741 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x558c5585bda0 0x558c5585e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.901+0000 7f448259c700 1 --2- 172.21.15.45:0/2357530741 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x558c5585bda0 0x558c5585e190 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8286b9afabf8cca9 server_cookie=1545f9b6f5032b5f in_seq=0 out_seq=0 2024-06-06T04:34:37.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.901+0000 7f4469ffb700 10 client.15894 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:34:37.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.902+0000 7f4469ffb700 1 -- 172.21.15.45:0/2357530741 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558c5579cd10 con 0x558c5585bda0 2024-06-06T04:34:37.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.902+0000 7f4469ffb700 10 client.15894 handle_command_reply: tid=0 2024-06-06T04:34:37.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.902+0000 7f44935a9b80 1 client.15894 shutdown 2024-06-06T04:34:37.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.902+0000 7f44935a9b80 1 -- 172.21.15.45:0/2357530741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c55846c30 msgr2=0x558c5584ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:37.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.902+0000 7f44935a9b80 1 --2- 172.21.15.45:0/2357530741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c55846c30 0x558c5584ba20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f447c004140 tx=0x7f447c004450 comp rx=0 tx=0).stop 2024-06-06T04:34:37.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.902+0000 7f44935a9b80 1 -- 172.21.15.45:0/2357530741 shutdown_connections 2024-06-06T04:34:37.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.902+0000 7f44935a9b80 1 --2- 172.21.15.45:0/2357530741 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x558c5585bda0 0x558c5585e190 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.902+0000 7f44935a9b80 1 --2- 172.21.15.45:0/2357530741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c55846c30 0x558c5584ba20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.902+0000 7f44935a9b80 1 --2- 172.21.15.45:0/2357530741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 0x558c5584e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.902+0000 7f44935a9b80 1 --2- 172.21.15.45:0/2357530741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c558428f0 0x558c5584b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.902+0000 7f44935a9b80 1 -- 172.21.15.45:0/2357530741 >> 172.21.15.45:0/2357530741 conn(0x558c55768cc0 msgr2=0x558c5583c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:37.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.902+0000 7f4469ffb700 0 client.15894 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:34:37.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.902+0000 7f44935a9b80 1 -- 172.21.15.45:0/2357530741 shutdown_connections 2024-06-06T04:34:37.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.902+0000 7f44935a9b80 1 -- 172.21.15.45:0/2357530741 wait complete. 2024-06-06T04:34:37.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.902+0000 7f44935a9b80 20 client.15894 trim_cache size 0 max 16384 2024-06-06T04:34:37.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.917+0000 7f44935a9b80 1 Processor -- start 2024-06-06T04:34:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.917+0000 7f44935a9b80 1 -- start start 2024-06-06T04:34:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.917+0000 7f44935a9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c55841fd0 0x558c5584b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f44935a9b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c558428f0 0x558c5584b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f44935a9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 0x558c55863af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f44935a9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f44841ae8f0 con 0x558c55841fd0 2024-06-06T04:34:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f44935a9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44841ac150 con 0x558c55846c30 2024-06-06T04:34:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f44935a9b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f44841a9fa0 con 0x558c558428f0 2024-06-06T04:34:37.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f448259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c55841fd0 0x558c5584b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f448259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c55841fd0 0x558c5584b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44612/0 (socket says 172.21.15.45:44612) 2024-06-06T04:34:37.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f448259c700 1 -- 172.21.15.45:0/1437014143 learned_addr learned my addr 172.21.15.45:0/1437014143 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:37.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f4481d9b700 1 --2- 172.21.15.45:0/1437014143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c558428f0 0x558c5584b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f448259c700 1 -- 172.21.15.45:0/1437014143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c558428f0 msgr2=0x558c5584b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:37.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f448259c700 1 --2- 172.21.15.45:0/1437014143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c558428f0 0x558c5584b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f448259c700 1 -- 172.21.15.45:0/1437014143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 msgr2=0x558c55863af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:37.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f4482d9d700 1 --2- 172.21.15.45:0/1437014143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 0x558c55863af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f448259c700 1 --2- 172.21.15.45:0/1437014143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 0x558c55863af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f448259c700 1 -- 172.21.15.45:0/1437014143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c5584bf20 con 0x558c55841fd0 2024-06-06T04:34:37.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f4481d9b700 1 --2- 172.21.15.45:0/1437014143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c558428f0 0x558c5584b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:34:37.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f448259c700 1 --2- 172.21.15.45:0/1437014143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c55841fd0 0x558c5584b4a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4474005b40 tx=0x7f44740cd860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83192878c4fdae59 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:37.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f446bfff700 1 -- 172.21.15.45:0/1437014143 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44740cee60 con 0x558c55841fd0 2024-06-06T04:34:37.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f446bfff700 1 -- 172.21.15.45:0/1437014143 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44740a0b30 con 0x558c55841fd0 2024-06-06T04:34:37.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.918+0000 7f446bfff700 1 -- 172.21.15.45:0/1437014143 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447409c9e0 con 0x558c55841fd0 2024-06-06T04:34:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.919+0000 7f44935a9b80 1 -- 172.21.15.45:0/1437014143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c55841fd0 msgr2=0x558c5584b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.919+0000 7f44935a9b80 1 --2- 172.21.15.45:0/1437014143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c55841fd0 0x558c5584b4a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4474005b40 tx=0x7f44740cd860 comp rx=0 tx=0).stop 2024-06-06T04:34:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.919+0000 7f44935a9b80 1 -- 172.21.15.45:0/1437014143 shutdown_connections 2024-06-06T04:34:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.919+0000 7f44935a9b80 1 --2- 172.21.15.45:0/1437014143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 0x558c55863af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.919+0000 7f44935a9b80 1 --2- 172.21.15.45:0/1437014143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c55841fd0 0x558c5584b4a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.919+0000 7f44935a9b80 1 --2- 172.21.15.45:0/1437014143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c558428f0 0x558c5584b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.919+0000 7f44935a9b80 1 -- 172.21.15.45:0/1437014143 >> 172.21.15.45:0/1437014143 conn(0x558c55768cc0 msgr2=0x558c55849e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.919+0000 7f44935a9b80 1 -- 172.21.15.45:0/1437014143 shutdown_connections 2024-06-06T04:34:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.919+0000 7f44935a9b80 1 -- 172.21.15.45:0/1437014143 wait complete. 2024-06-06T04:34:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.919+0000 7f44935a9b80 1 -- start start 2024-06-06T04:34:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.919+0000 7f44935a9b80 1 Processor -- start 2024-06-06T04:34:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f44935a9b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 0x558c55854820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f44935a9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c558428f0 0x558c5584fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f44935a9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 0x558c558500e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f44935a9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f44841aa610 con 0x558c558428f0 2024-06-06T04:34:37.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f44935a9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44841aba80 con 0x558c55846c30 2024-06-06T04:34:37.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f44935a9b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f44841aace0 con 0x558c55841fd0 2024-06-06T04:34:37.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f44935a9b80 10 client.0 _ll_register_callbacks cb 0x558c55729920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T04:34:37.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f4481d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c558428f0 0x558c5584fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f4481d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c558428f0 0x558c5584fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44614/0 (socket says 172.21.15.45:44614) 2024-06-06T04:34:37.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f4481d9b700 1 -- 172.21.15.45:0/448128319 learned_addr learned my addr 172.21.15.45:0/448128319 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:37.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f448259c700 1 --2- 172.21.15.45:0/448128319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 0x558c55854820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f4481d9b700 1 -- 172.21.15.45:0/448128319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 msgr2=0x558c55854820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:37.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f4481d9b700 1 --2- 172.21.15.45:0/448128319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 0x558c55854820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f4481d9b700 1 -- 172.21.15.45:0/448128319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 msgr2=0x558c558500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T04:34:37.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f4482d9d700 1 --2- 172.21.15.45:0/448128319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 0x558c558500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f4481d9b700 1 --2- 172.21.15.45:0/448128319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 0x558c558500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f4481d9b700 1 -- 172.21.15.45:0/448128319 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c55850920 con 0x558c558428f0 2024-06-06T04:34:37.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f448259c700 1 --2- 172.21.15.45:0/448128319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 0x558c55854820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:34:37.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f4482d9d700 1 --2- 172.21.15.45:0/448128319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 0x558c558500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:34:37.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f4481d9b700 1 --2- 172.21.15.45:0/448128319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c558428f0 0x558c5584fba0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f447800e9d0 tx=0x7f4478015b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc1ab0a08839bd4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:37.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.920+0000 7f4469ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-06-06T04:34:37.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.921+0000 7f4469ffb700 1 -- 172.21.15.45:0/448128319 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478029810 con 0x558c558428f0 2024-06-06T04:34:37.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.921+0000 7f4469ffb700 1 -- 172.21.15.45:0/448128319 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f44780299b0 con 0x558c558428f0 2024-06-06T04:34:37.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.921+0000 7f44935a9b80 1 -- 172.21.15.45:0/448128319 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44740a0b30 con 0x558c558428f0 2024-06-06T04:34:37.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.921+0000 7f4469ffb700 1 -- 172.21.15.45:0/448128319 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f447803c050 con 0x558c558428f0 2024-06-06T04:34:37.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.921+0000 7f44935a9b80 10 client.15900 fetch_fsmap learned FSMap version 30 2024-06-06T04:34:37.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.921+0000 7f44935a9b80 1 -- 172.21.15.45:0/448128319 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558c558c69e0 con 0x558c558428f0 2024-06-06T04:34:37.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.921+0000 7f4469ffb700 1 -- 172.21.15.45:0/448128319 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44740a0b30 con 0x558c558428f0 2024-06-06T04:34:37.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.921+0000 7f4469ffb700 1 -- 172.21.15.45:0/448128319 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f447801e030 con 0x558c558428f0 2024-06-06T04:34:37.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.921+0000 7f44935a9b80 10 client.15900 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T04:34:37.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.921+0000 7f44935a9b80 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-06T04:34:37.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.921+0000 7f44935a9b80 20 client.15900 populate_metadata read hostname 'smithi045' 2024-06-06T04:34:37.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.921+0000 7f44935a9b80 1 --2- 172.21.15.45:0/448128319 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x558c558c6e60 0x558c558c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:37.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.921+0000 7f44935a9b80 4 client.15900 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T04:34:37.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.921+0000 7f44935a9b80 1 -- 172.21.15.45:0/448128319 --> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c55797290 con 0x558c558c6e60 2024-06-06T04:34:37.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f448259c700 1 --2- 172.21.15.45:0/448128319 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x558c558c6e60 0x558c558c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:37.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f4469ffb700 10 client.15900 ms_handle_connect on v2:172.21.15.86:6838/816791193 2024-06-06T04:34:37.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f448259c700 1 --2- 172.21.15.45:0/448128319 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x558c558c6e60 0x558c558c9280 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b399e78dac0178e server_cookie=ae57789527ffda17 in_seq=0 out_seq=0 2024-06-06T04:34:37.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f4469ffb700 1 -- 172.21.15.45:0/448128319 <== mds.0 v2:172.21.15.86:6838/816791193 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c55797290 con 0x558c558c6e60 2024-06-06T04:34:37.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f4469ffb700 10 client.15900 handle_command_reply: tid=0 2024-06-06T04:34:37.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f44935a9b80 1 client.15900 shutdown 2024-06-06T04:34:37.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f44935a9b80 1 -- 172.21.15.45:0/448128319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c558428f0 msgr2=0x558c5584fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:37.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f44935a9b80 1 --2- 172.21.15.45:0/448128319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c558428f0 0x558c5584fba0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f447800e9d0 tx=0x7f4478015b50 comp rx=0 tx=0).stop 2024-06-06T04:34:37.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f44935a9b80 1 -- 172.21.15.45:0/448128319 shutdown_connections 2024-06-06T04:34:37.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f44935a9b80 1 --2- 172.21.15.45:0/448128319 >> [v2:172.21.15.86:6838/816791193,v1:172.21.15.86:6839/816791193] conn(0x558c558c6e60 0x558c558c9280 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f44935a9b80 1 --2- 172.21.15.45:0/448128319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c55846c30 0x558c558500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f44935a9b80 1 --2- 172.21.15.45:0/448128319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558c55841fd0 0x558c55854820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f44935a9b80 1 --2- 172.21.15.45:0/448128319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c558428f0 0x558c5584fba0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f44935a9b80 1 -- 172.21.15.45:0/448128319 >> 172.21.15.45:0/448128319 conn(0x558c55768cc0 msgr2=0x558c55849dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:37.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f4469ffb700 0 client.15900 ms_handle_reset on v2:172.21.15.86:6838/816791193 2024-06-06T04:34:37.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f44935a9b80 1 -- 172.21.15.45:0/448128319 shutdown_connections 2024-06-06T04:34:37.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f44935a9b80 1 -- 172.21.15.45:0/448128319 wait complete. 2024-06-06T04:34:37.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.922+0000 7f44935a9b80 20 client.15900 trim_cache size 0 max 16384 2024-06-06T04:34:37.933 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-06-06T04:34:37.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.923+0000 7f4488d26700 1 -- 172.21.15.45:0/841587958 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f446c0a2ca0 msgr2=0x7f446c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:37.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.923+0000 7f4488d26700 1 --2- 172.21.15.45:0/841587958 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f446c0a2ca0 0x7f446c0a5160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4484106200 tx=0x7f4478009190 comp rx=0 tx=0).stop 2024-06-06T04:34:37.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.923+0000 7f4488d26700 1 -- 172.21.15.45:0/841587958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4484100b50 msgr2=0x7f448419e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:37.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.923+0000 7f4488d26700 1 --2- 172.21.15.45:0/841587958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4484100b50 0x7f448419e770 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f44740098e0 tx=0x7f4474005f00 comp rx=0 tx=0).stop 2024-06-06T04:34:37.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.923+0000 7f4488d26700 1 -- 172.21.15.45:0/841587958 shutdown_connections 2024-06-06T04:34:37.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.923+0000 7f4488d26700 1 --2- 172.21.15.45:0/841587958 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f446c0a2ca0 0x7f446c0a5160 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.923+0000 7f4488d26700 1 --2- 172.21.15.45:0/841587958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4484106c00 0x7f4484198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.923+0000 7f4488d26700 1 --2- 172.21.15.45:0/841587958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4484100b50 0x7f448419e770 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.923+0000 7f4488d26700 1 --2- 172.21.15.45:0/841587958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4484101510 0x7f448419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:37.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.923+0000 7f4488d26700 1 -- 172.21.15.45:0/841587958 >> 172.21.15.45:0/841587958 conn(0x7f4484077990 msgr2=0x7f44840ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:37.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.923+0000 7f4488d26700 1 -- 172.21.15.45:0/841587958 shutdown_connections 2024-06-06T04:34:37.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:37.923+0000 7f4488d26700 1 -- 172.21.15.45:0/841587958 wait complete. 2024-06-06T04:34:38.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:38 smithi086 ceph-mon[36053]: pgmap v526: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-06-06T04:34:38.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:38 smithi154 ceph-mon[36428]: pgmap v526: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-06-06T04:34:38.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:38 smithi045 ceph-mon[32299]: pgmap v526: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-06-06T04:34:38.807 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-06-06T04:34:38.812 INFO:tasks.fwd_scrub:done joining 2024-06-06T04:34:38.813 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-06-06T04:34:38.823 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-06T04:34:38.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.938+0000 7f32c4ef5640 1 -- 172.21.15.45:0/956245732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0102490 msgr2=0x7f32c0108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.938+0000 7f32c4ef5640 1 --2- 172.21.15.45:0/956245732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0102490 0x7f32c0108d20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009900 tx=0x7f32b802a410 comp rx=0 tx=0).stop 2024-06-06T04:34:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.938+0000 7f32c4ef5640 1 -- 172.21.15.45:0/956245732 shutdown_connections 2024-06-06T04:34:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.938+0000 7f32c4ef5640 1 --2- 172.21.15.45:0/956245732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0101ad0 0x7f32c0101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:38.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.938+0000 7f32c4ef5640 1 --2- 172.21.15.45:0/956245732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c0102490 0x7f32c0108d20 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.938+0000 7f32c4ef5640 1 --2- 172.21.15.45:0/956245732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32c01008e0 0x7f32c0100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.938+0000 7f32c4ef5640 1 -- 172.21.15.45:0/956245732 >> 172.21.15.45:0/956245732 conn(0x7f32c00fbe50 msgr2=0x7f32c00fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.938+0000 7f32c4ef5640 1 -- 172.21.15.45:0/956245732 shutdown_connections 2024-06-06T04:34:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.938+0000 7f32c4ef5640 1 -- 172.21.15.45:0/956245732 wait complete. 2024-06-06T04:34:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32c4ef5640 1 Processor -- start 2024-06-06T04:34:38.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32c4ef5640 1 -- start start 2024-06-06T04:34:38.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32c4ef5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c01008e0 0x7f32c00696a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:38.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32c4ef5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32c0101ad0 0x7f32c019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:38.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32c4ef5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0102490 0x7f32c019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:38.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32c4ef5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f32c010c3f0 con 0x7f32c01008e0 2024-06-06T04:34:38.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32c4ef5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32c010c270 con 0x7f32c0102490 2024-06-06T04:34:38.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32c4ef5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f32c010c570 con 0x7f32c0101ad0 2024-06-06T04:34:38.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32be59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c01008e0 0x7f32c00696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:38.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32be59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c01008e0 0x7f32c00696a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44632/0 (socket says 172.21.15.45:44632) 2024-06-06T04:34:38.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32be59c640 1 -- 172.21.15.45:0/1438556509 learned_addr learned my addr 172.21.15.45:0/1438556509 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:38.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32bdd9b640 1 --2- 172.21.15.45:0/1438556509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32c0101ad0 0x7f32c019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:38.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32bed9d640 1 --2- 172.21.15.45:0/1438556509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0102490 0x7f32c019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:38.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32be59c640 1 -- 172.21.15.45:0/1438556509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32c0101ad0 msgr2=0x7f32c019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:38.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32be59c640 1 --2- 172.21.15.45:0/1438556509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32c0101ad0 0x7f32c019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:38.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32be59c640 1 -- 172.21.15.45:0/1438556509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0102490 msgr2=0x7f32c019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:38.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32be59c640 1 --2- 172.21.15.45:0/1438556509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0102490 0x7f32c019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:38.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32be59c640 1 -- 172.21.15.45:0/1438556509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c019f810 con 0x7f32c01008e0 2024-06-06T04:34:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.939+0000 7f32bed9d640 1 --2- 172.21.15.45:0/1438556509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0102490 0x7f32c019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:34:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.940+0000 7f32be59c640 1 --2- 172.21.15.45:0/1438556509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c01008e0 0x7f32c00696a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009fd0 tx=0x7f32b000c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.940+0000 7f32af7fe640 1 -- 172.21.15.45:0/1438556509 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0011470 con 0x7f32c01008e0 2024-06-06T04:34:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.940+0000 7f32af7fe640 1 -- 172.21.15.45:0/1438556509 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32b0011ac0 con 0x7f32c01008e0 2024-06-06T04:34:38.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.940+0000 7f32af7fe640 1 -- 172.21.15.45:0/1438556509 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b001aab0 con 0x7f32c01008e0 2024-06-06T04:34:38.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.940+0000 7f32c4ef5640 1 -- 172.21.15.45:0/1438556509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c019faa0 con 0x7f32c01008e0 2024-06-06T04:34:38.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.940+0000 7f32c4ef5640 1 -- 172.21.15.45:0/1438556509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c019fff0 con 0x7f32c01008e0 2024-06-06T04:34:38.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.941+0000 7f32c4ef5640 1 -- 172.21.15.45:0/1438556509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c0107c80 con 0x7f32c01008e0 2024-06-06T04:34:38.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.942+0000 7f32af7fe640 1 -- 172.21.15.45:0/1438556509 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f32b0011c70 con 0x7f32c01008e0 2024-06-06T04:34:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.943+0000 7f32af7fe640 1 --2- 172.21.15.45:0/1438556509 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f32a80a2b90 0x7f32a80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.943+0000 7f32bdd9b640 1 --2- 172.21.15.45:0/1438556509 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f32a80a2b90 0x7f32a80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.943+0000 7f32af7fe640 1 -- 172.21.15.45:0/1438556509 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f32b00cf9a0 con 0x7f32c01008e0 2024-06-06T04:34:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.943+0000 7f32bdd9b640 1 --2- 172.21.15.45:0/1438556509 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f32a80a2b90 0x7f32a80a5050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f32c0069410 tx=0x7f32b400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:38.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:38.947+0000 7f32af7fe640 1 -- 172.21.15.45:0/1438556509 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f32b0016030 con 0x7f32c01008e0 2024-06-06T04:34:39.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.133+0000 7f32c4ef5640 1 -- 172.21.15.45:0/1438556509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32c00625d0 con 0x7f32c01008e0 2024-06-06T04:34:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.135+0000 7f32af7fe640 1 -- 172.21.15.45:0/1438556509 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24789 (secure 0 0 0) 0x7f32b009c540 con 0x7f32c01008e0 2024-06-06T04:34:39.135 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:34:39.135 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":100,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","created":"2024-06-06T04:19:02.626916+0000","modified":"2024-06-06T04:29:05.901370+0000","last_up_change":"2024-06-06T04:26:17.390429+0000","last_in_change":"2024-06-06T04:25:56.938388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T04:22:28.797137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T04:27:17.750625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T04:27:21.486917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T04:27:31.029818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d3cee4a-5025-45b3-b217-082af37e79db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6803","nonce":2111759250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6805","nonce":2111759250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6809","nonce":2111759250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6807","nonce":2111759250}]},"public_addr":"172.21.15.45:6803/2111759250","cluster_addr":"172.21.15.45:6805/2111759250","heartbeat_back_addr":"172.21.15.45:6809/2111759250","heartbeat_front_addr":"172.21.15.45:6807/2111759250","state":["exists","up"]},{"osd":1,"uuid":"75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6801","nonce":3732225287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6803","nonce":3732225287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6807","nonce":3732225287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6805","nonce":3732225287}]},"public_addr":"172.21.15.86:6801/3732225287","cluster_addr":"172.21.15.86:6803/3732225287","heartbeat_back_addr":"172.21.15.86:6807/3732225287","heartbeat_front_addr":"172.21.15.86:6805/3732225287","state":["exists","up"]},{"osd":2,"uuid":"bb20565f-f9ae-47e3-9604-c69f737cff4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6801","nonce":2898783039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6803","nonce":2898783039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6807","nonce":2898783039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6805","nonce":2898783039}]},"public_addr":"172.21.15.154:6801/2898783039","cluster_addr":"172.21.15.154:6803/2898783039","heartbeat_back_addr":"172.21.15.154:6807/2898783039","heartbeat_front_addr":"172.21.15.154:6805/2898783039","state":["exists","up"]},{"osd":3,"uuid":"e1160799-7811-4326-9792-7e6c7d189c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6811","nonce":3436753135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6813","nonce":3436753135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6817","nonce":3436753135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6815","nonce":3436753135}]},"public_addr":"172.21.15.45:6811/3436753135","cluster_addr":"172.21.15.45:6813/3436753135","heartbeat_back_addr":"172.21.15.45:6817/3436753135","heartbeat_front_addr":"172.21.15.45:6815/3436753135","state":["exists","up"]},{"osd":4,"uuid":"6b389bc8-e19a-47f3-83ff-7b782d146d0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6809","nonce":3030235796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6811","nonce":3030235796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6815","nonce":3030235796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6813","nonce":3030235796}]},"public_addr":"172.21.15.86:6809/3030235796","cluster_addr":"172.21.15.86:6811/3030235796","heartbeat_back_addr":"172.21.15.86:6815/3030235796","heartbeat_front_addr":"172.21.15.86:6813/3030235796","state":["exists","up"]},{"osd":5,"uuid":"069de7b2-89bf-4150-bed7-7b8832573dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6809","nonce":2222268646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6811","nonce":2222268646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6815","nonce":2222268646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6813","nonce":2222268646}]},"public_addr":"172.21.15.154:6809/2222268646","cluster_addr":"172.21.15.154:6811/2222268646","heartbeat_back_addr":"172.21.15.154:6815/2222268646","heartbeat_front_addr":"172.21.15.154:6813/2222268646","state":["exists","up"]},{"osd":6,"uuid":"4c70b627-eafa-4e27-ba2a-4d8e1e335962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6819","nonce":384734939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6821","nonce":384734939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6825","nonce":384734939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6823","nonce":384734939}]},"public_addr":"172.21.15.45:6819/384734939","cluster_addr":"172.21.15.45:6821/384734939","heartbeat_back_addr":"172.21.15.45:6825/384734939","heartbeat_front_addr":"172.21.15.45:6823/384734939","state":["exists","up"]},{"osd":7,"uuid":"3b09b875-be52-4be0-b116-7fd9b0c26705","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6817","nonce":1119791005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6819","nonce":1119791005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6823","nonce":1119791005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6821","nonce":1119791005}]},"public_addr":"172.21.15.86:6817/1119791005","cluster_addr":"172.21.15.86:6819/1119791005","heartbeat_back_addr":"172.21.15.86:6823/1119791005","heartbeat_front_addr":"172.21.15.86:6821/1119791005","state":["exists","up"]},{"osd":8,"uuid":"cdf1dd1b-5392-4836-8e65-4c7c5b688d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6817","nonce":3054812496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6819","nonce":3054812496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6823","nonce":3054812496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6821","nonce":3054812496}]},"public_addr":"172.21.15.154:6817/3054812496","cluster_addr":"172.21.15.154:6819/3054812496","heartbeat_back_addr":"172.21.15.154:6823/3054812496","heartbeat_front_addr":"172.21.15.154:6821/3054812496","state":["exists","up"]},{"osd":9,"uuid":"acede92d-2cc1-4967-a357-7e5e201e99d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6827","nonce":1360655510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6829","nonce":1360655510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6833","nonce":1360655510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6831","nonce":1360655510}]},"public_addr":"172.21.15.45:6827/1360655510","cluster_addr":"172.21.15.45:6829/1360655510","heartbeat_back_addr":"172.21.15.45:6833/1360655510","heartbeat_front_addr":"172.21.15.45:6831/1360655510","state":["exists","up"]},{"osd":10,"uuid":"a2d0c488-1892-4da4-adbf-952c22df0ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6825","nonce":1701691877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6827","nonce":1701691877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6831","nonce":1701691877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6829","nonce":1701691877}]},"public_addr":"172.21.15.86:6825/1701691877","cluster_addr":"172.21.15.86:6827/1701691877","heartbeat_back_addr":"172.21.15.86:6831/1701691877","heartbeat_front_addr":"172.21.15.86:6829/1701691877","state":["exists","up"]},{"osd":11,"uuid":"6b349c4c-b672-480b-ac6a-8594dab0eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6825","nonce":2065648007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6827","nonce":2065648007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6831","nonce":2065648007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6829","nonce":2065648007}]},"public_addr":"172.21.15.154:6825/2065648007","cluster_addr":"172.21.15.154:6827/2065648007","heartbeat_back_addr":"172.21.15.154:6831/2065648007","heartbeat_front_addr":"172.21.15.154:6829/2065648007","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:21:37.147394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:00.776617+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:24.786849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:50.487455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:14.869272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:39.815158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:05.972619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:31.410063+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:56.795119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:23.114884+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:49.256504+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:26:15.251967+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/36732849":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/1500318775":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6801/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6800/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/2041380391":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/2504003341":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/3798552599":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/498952822":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6800/1562511056":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6801/1562511056":"2024-06-07T04:19:37.026199+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T04:34:39.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.137+0000 7f32c4ef5640 1 -- 172.21.15.45:0/1438556509 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f32a80a2b90 msgr2=0x7f32a80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:39.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.137+0000 7f32c4ef5640 1 --2- 172.21.15.45:0/1438556509 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f32a80a2b90 0x7f32a80a5050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f32c0069410 tx=0x7f32b400b390 comp rx=0 tx=0).stop 2024-06-06T04:34:39.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.137+0000 7f32c4ef5640 1 -- 172.21.15.45:0/1438556509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c01008e0 msgr2=0x7f32c00696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:39.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.137+0000 7f32c4ef5640 1 --2- 172.21.15.45:0/1438556509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c01008e0 0x7f32c00696a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009fd0 tx=0x7f32b000c270 comp rx=0 tx=0).stop 2024-06-06T04:34:39.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.137+0000 7f32c4ef5640 1 -- 172.21.15.45:0/1438556509 shutdown_connections 2024-06-06T04:34:39.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.137+0000 7f32c4ef5640 1 --2- 172.21.15.45:0/1438556509 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f32a80a2b90 0x7f32a80a5050 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.137+0000 7f32c4ef5640 1 --2- 172.21.15.45:0/1438556509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0102490 0x7f32c019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.137+0000 7f32c4ef5640 1 --2- 172.21.15.45:0/1438556509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32c0101ad0 0x7f32c019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.137+0000 7f32c4ef5640 1 --2- 172.21.15.45:0/1438556509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32c01008e0 0x7f32c00696a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.137+0000 7f32c4ef5640 1 -- 172.21.15.45:0/1438556509 >> 172.21.15.45:0/1438556509 conn(0x7f32c00fbe50 msgr2=0x7f32c0104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.137+0000 7f32c4ef5640 1 -- 172.21.15.45:0/1438556509 shutdown_connections 2024-06-06T04:34:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.137+0000 7f32c4ef5640 1 -- 172.21.15.45:0/1438556509 wait complete. 2024-06-06T04:34:39.149 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-06T04:22:28.797137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-06T04:27:17.750625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-06T04:27:21.486917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-06T04:27:31.029818+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-06T04:34:39.150 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-06T04:34:39.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.257+0000 7f5b44bd5640 1 -- 172.21.15.45:0/1420127704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b40101a00 msgr2=0x7f5b40101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:39.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.257+0000 7f5b44bd5640 1 --2- 172.21.15.45:0/1420127704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b40101a00 0x7f5b40101e20 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009900 tx=0x7f5b3002a410 comp rx=0 tx=0).stop 2024-06-06T04:34:39.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.257+0000 7f5b44bd5640 1 -- 172.21.15.45:0/1420127704 shutdown_connections 2024-06-06T04:34:39.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.257+0000 7f5b44bd5640 1 --2- 172.21.15.45:0/1420127704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b401035b0 0x7f5b40109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.257+0000 7f5b44bd5640 1 --2- 172.21.15.45:0/1420127704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b40102bf0 0x7f5b40103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.257+0000 7f5b44bd5640 1 --2- 172.21.15.45:0/1420127704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b40101a00 0x7f5b40101e20 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.257+0000 7f5b44bd5640 1 -- 172.21.15.45:0/1420127704 >> 172.21.15.45:0/1420127704 conn(0x7f5b400fcf70 msgr2=0x7f5b400ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:39.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.257+0000 7f5b44bd5640 1 -- 172.21.15.45:0/1420127704 shutdown_connections 2024-06-06T04:34:39.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.257+0000 7f5b44bd5640 1 -- 172.21.15.45:0/1420127704 wait complete. 2024-06-06T04:34:39.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.258+0000 7f5b44bd5640 1 Processor -- start 2024-06-06T04:34:39.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.258+0000 7f5b44bd5640 1 -- start start 2024-06-06T04:34:39.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.258+0000 7f5b44bd5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b40101a00 0x7f5b40197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:39.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.258+0000 7f5b44bd5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b40102bf0 0x7f5b401982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:39.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.258+0000 7f5b44bd5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b401035b0 0x7f5b4019f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:39.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.258+0000 7f5b44bd5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4010d580 con 0x7f5b40101a00 2024-06-06T04:34:39.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.258+0000 7f5b44bd5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4010d400 con 0x7f5b40102bf0 2024-06-06T04:34:39.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.258+0000 7f5b44bd5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4010d700 con 0x7f5b401035b0 2024-06-06T04:34:39.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.258+0000 7f5b3e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b40101a00 0x7f5b40197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:39.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.258+0000 7f5b3e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b40101a00 0x7f5b40197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44654/0 (socket says 172.21.15.45:44654) 2024-06-06T04:34:39.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.259+0000 7f5b3e59c640 1 -- 172.21.15.45:0/106892133 learned_addr learned my addr 172.21.15.45:0/106892133 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:39.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.259+0000 7f5b3dd9b640 1 --2- 172.21.15.45:0/106892133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b40102bf0 0x7f5b401982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:39.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.259+0000 7f5b3e59c640 1 -- 172.21.15.45:0/106892133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b401035b0 msgr2=0x7f5b4019f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:39.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.259+0000 7f5b3ed9d640 1 --2- 172.21.15.45:0/106892133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b401035b0 0x7f5b4019f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:39.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.259+0000 7f5b3e59c640 1 --2- 172.21.15.45:0/106892133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b401035b0 0x7f5b4019f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.259+0000 7f5b3e59c640 1 -- 172.21.15.45:0/106892133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b40102bf0 msgr2=0x7f5b401982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:39.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.259+0000 7f5b3e59c640 1 --2- 172.21.15.45:0/106892133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b40102bf0 0x7f5b401982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.259+0000 7f5b3e59c640 1 -- 172.21.15.45:0/106892133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4019f920 con 0x7f5b40101a00 2024-06-06T04:34:39.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.259+0000 7f5b3dd9b640 1 --2- 172.21.15.45:0/106892133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b40102bf0 0x7f5b401982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:34:39.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.259+0000 7f5b3ed9d640 1 --2- 172.21.15.45:0/106892133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b401035b0 0x7f5b4019f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T04:34:39.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.259+0000 7f5b3e59c640 1 --2- 172.21.15.45:0/106892133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b40101a00 0x7f5b40197d60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009230 tx=0x7f5b30004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:39.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.259+0000 7f5b2f7fe640 1 -- 172.21.15.45:0/106892133 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3003f030 con 0x7f5b40101a00 2024-06-06T04:34:39.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.259+0000 7f5b2f7fe640 1 -- 172.21.15.45:0/106892133 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b30003cb0 con 0x7f5b40101a00 2024-06-06T04:34:39.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.259+0000 7f5b44bd5640 1 -- 172.21.15.45:0/106892133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4019fbb0 con 0x7f5b40101a00 2024-06-06T04:34:39.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.259+0000 7f5b2f7fe640 1 -- 172.21.15.45:0/106892133 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30035070 con 0x7f5b40101a00 2024-06-06T04:34:39.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.259+0000 7f5b44bd5640 1 -- 172.21.15.45:0/106892133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b401a00a0 con 0x7f5b40101a00 2024-06-06T04:34:39.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.260+0000 7f5b44bd5640 1 -- 172.21.15.45:0/106892133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b40108e60 con 0x7f5b40101a00 2024-06-06T04:34:39.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.261+0000 7f5b2f7fe640 1 -- 172.21.15.45:0/106892133 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5b30042030 con 0x7f5b40101a00 2024-06-06T04:34:39.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.262+0000 7f5b2f7fe640 1 --2- 172.21.15.45:0/106892133 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5b280a2b90 0x7f5b280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:39.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.262+0000 7f5b3dd9b640 1 --2- 172.21.15.45:0/106892133 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5b280a2b90 0x7f5b280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:39.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.262+0000 7f5b2f7fe640 1 -- 172.21.15.45:0/106892133 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f5b300efbd0 con 0x7f5b40101a00 2024-06-06T04:34:39.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.263+0000 7f5b3dd9b640 1 --2- 172.21.15.45:0/106892133 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5b280a2b90 0x7f5b280a5050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5b34006fd0 tx=0x7f5b34008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:39.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.266+0000 7f5b2f7fe640 1 -- 172.21.15.45:0/106892133 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b300bc770 con 0x7f5b40101a00 2024-06-06T04:34:39.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.459+0000 7f5b44bd5640 1 -- 172.21.15.45:0/106892133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5b40062640 con 0x7f5b40101a00 2024-06-06T04:34:39.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.459+0000 7f5b2f7fe640 1 -- 172.21.15.45:0/106892133 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f5b300bc070 con 0x7f5b40101a00 2024-06-06T04:34:39.460 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-06-06T04:34:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.462+0000 7f5b44bd5640 1 -- 172.21.15.45:0/106892133 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5b280a2b90 msgr2=0x7f5b280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.462+0000 7f5b44bd5640 1 --2- 172.21.15.45:0/106892133 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5b280a2b90 0x7f5b280a5050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5b34006fd0 tx=0x7f5b34008040 comp rx=0 tx=0).stop 2024-06-06T04:34:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.462+0000 7f5b44bd5640 1 -- 172.21.15.45:0/106892133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b40101a00 msgr2=0x7f5b40197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.462+0000 7f5b44bd5640 1 --2- 172.21.15.45:0/106892133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b40101a00 0x7f5b40197d60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009230 tx=0x7f5b30004340 comp rx=0 tx=0).stop 2024-06-06T04:34:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.462+0000 7f5b44bd5640 1 -- 172.21.15.45:0/106892133 shutdown_connections 2024-06-06T04:34:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.462+0000 7f5b44bd5640 1 --2- 172.21.15.45:0/106892133 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f5b280a2b90 0x7f5b280a5050 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.462+0000 7f5b44bd5640 1 --2- 172.21.15.45:0/106892133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b40102bf0 0x7f5b401982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.462+0000 7f5b44bd5640 1 --2- 172.21.15.45:0/106892133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b401035b0 0x7f5b4019f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.462+0000 7f5b44bd5640 1 --2- 172.21.15.45:0/106892133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b40101a00 0x7f5b40197d60 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.462+0000 7f5b44bd5640 1 -- 172.21.15.45:0/106892133 >> 172.21.15.45:0/106892133 conn(0x7f5b400fcf70 msgr2=0x7f5b40106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:39.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.462+0000 7f5b44bd5640 1 -- 172.21.15.45:0/106892133 shutdown_connections 2024-06-06T04:34:39.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.462+0000 7f5b44bd5640 1 -- 172.21.15.45:0/106892133 wait complete. 2024-06-06T04:34:39.472 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:39 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1438556509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:34:39.473 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-06T04:34:39.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.585+0000 7fd586b5a640 1 -- 172.21.15.45:0/2407433822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd580102bf0 msgr2=0x7fd580103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:39.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.585+0000 7fd586b5a640 1 --2- 172.21.15.45:0/2407433822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd580102bf0 0x7fd580103070 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd570009900 tx=0x7fd57002a410 comp rx=0 tx=0).stop 2024-06-06T04:34:39.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.585+0000 7fd586b5a640 1 -- 172.21.15.45:0/2407433822 shutdown_connections 2024-06-06T04:34:39.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.585+0000 7fd586b5a640 1 --2- 172.21.15.45:0/2407433822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5801035b0 0x7fd580109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.585+0000 7fd586b5a640 1 --2- 172.21.15.45:0/2407433822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd580102bf0 0x7fd580103070 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.585+0000 7fd586b5a640 1 --2- 172.21.15.45:0/2407433822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd580101a00 0x7fd580101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.585+0000 7fd586b5a640 1 -- 172.21.15.45:0/2407433822 >> 172.21.15.45:0/2407433822 conn(0x7fd5800fcf70 msgr2=0x7fd5800ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.585+0000 7fd586b5a640 1 -- 172.21.15.45:0/2407433822 shutdown_connections 2024-06-06T04:34:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.585+0000 7fd586b5a640 1 -- 172.21.15.45:0/2407433822 wait complete. 2024-06-06T04:34:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd586b5a640 1 Processor -- start 2024-06-06T04:34:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd586b5a640 1 -- start start 2024-06-06T04:34:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd586b5a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd580101a00 0x7fd580197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd586b5a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd580102bf0 0x7fd580198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd586b5a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5801035b0 0x7fd58019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:39.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd586b5a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd58010d510 con 0x7fd5801035b0 2024-06-06T04:34:39.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd586b5a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd58010d390 con 0x7fd580102bf0 2024-06-06T04:34:39.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd586b5a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd58010d690 con 0x7fd580101a00 2024-06-06T04:34:39.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd5850f7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5801035b0 0x7fd58019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:39.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd5850f7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5801035b0 0x7fd58019f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44666/0 (socket says 172.21.15.45:44666) 2024-06-06T04:34:39.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd5850f7640 1 -- 172.21.15.45:0/3057741266 learned_addr learned my addr 172.21.15.45:0/3057741266 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:39.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd577fff640 1 --2- 172.21.15.45:0/3057741266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd580102bf0 0x7fd580198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:39.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd5848f6640 1 --2- 172.21.15.45:0/3057741266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd580101a00 0x7fd580197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:39.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd5850f7640 1 -- 172.21.15.45:0/3057741266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd580101a00 msgr2=0x7fd580197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:39.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd5850f7640 1 --2- 172.21.15.45:0/3057741266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd580101a00 0x7fd580197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd5850f7640 1 -- 172.21.15.45:0/3057741266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd580102bf0 msgr2=0x7fd580198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:39.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd5850f7640 1 --2- 172.21.15.45:0/3057741266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd580102bf0 0x7fd580198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.586+0000 7fd5850f7640 1 -- 172.21.15.45:0/3057741266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58019f7f0 con 0x7fd5801035b0 2024-06-06T04:34:39.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.587+0000 7fd5850f7640 1 --2- 172.21.15.45:0/3057741266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5801035b0 0x7fd58019f180 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd57800cce0 tx=0x7fd57800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:39.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.587+0000 7fd575ffb640 1 -- 172.21.15.45:0/3057741266 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57800f450 con 0x7fd5801035b0 2024-06-06T04:34:39.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.587+0000 7fd575ffb640 1 -- 172.21.15.45:0/3057741266 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd57800faa0 con 0x7fd5801035b0 2024-06-06T04:34:39.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.587+0000 7fd586b5a640 1 -- 172.21.15.45:0/3057741266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58019fae0 con 0x7fd5801035b0 2024-06-06T04:34:39.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.587+0000 7fd575ffb640 1 -- 172.21.15.45:0/3057741266 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd578018b70 con 0x7fd5801035b0 2024-06-06T04:34:39.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.587+0000 7fd586b5a640 1 -- 172.21.15.45:0/3057741266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58019ffb0 con 0x7fd5801035b0 2024-06-06T04:34:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.588+0000 7fd586b5a640 1 -- 172.21.15.45:0/3057741266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd580108df0 con 0x7fd5801035b0 2024-06-06T04:34:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.589+0000 7fd575ffb640 1 -- 172.21.15.45:0/3057741266 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd578018d10 con 0x7fd5801035b0 2024-06-06T04:34:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.590+0000 7fd575ffb640 1 --2- 172.21.15.45:0/3057741266 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd5680a2b90 0x7fd5680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.590+0000 7fd575ffb640 1 -- 172.21.15.45:0/3057741266 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fd5780cf780 con 0x7fd5801035b0 2024-06-06T04:34:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.590+0000 7fd5848f6640 1 --2- 172.21.15.45:0/3057741266 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd5680a2b90 0x7fd5680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:39.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.590+0000 7fd5848f6640 1 --2- 172.21.15.45:0/3057741266 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd5680a2b90 0x7fd5680a5050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd57c005fd0 tx=0x7fd57c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:39.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.594+0000 7fd575ffb640 1 -- 172.21.15.45:0/3057741266 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd578014030 con 0x7fd5801035b0 2024-06-06T04:34:39.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:39 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1438556509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:34:39.683 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:39 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1438556509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:34:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.777+0000 7fd586b5a640 1 -- 172.21.15.45:0/3057741266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd58002d2e0 con 0x7fd5801035b0 2024-06-06T04:34:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.777+0000 7fd575ffb640 1 -- 172.21.15.45:0/3057741266 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7fd57809c2c0 con 0x7fd5801035b0 2024-06-06T04:34:39.778 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-06-06T04:34:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.779+0000 7fd586b5a640 1 -- 172.21.15.45:0/3057741266 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd5680a2b90 msgr2=0x7fd5680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.779+0000 7fd586b5a640 1 --2- 172.21.15.45:0/3057741266 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd5680a2b90 0x7fd5680a5050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd57c005fd0 tx=0x7fd57c00b410 comp rx=0 tx=0).stop 2024-06-06T04:34:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.779+0000 7fd586b5a640 1 -- 172.21.15.45:0/3057741266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5801035b0 msgr2=0x7fd58019f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.779+0000 7fd586b5a640 1 --2- 172.21.15.45:0/3057741266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5801035b0 0x7fd58019f180 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd57800cce0 tx=0x7fd57800a620 comp rx=0 tx=0).stop 2024-06-06T04:34:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.779+0000 7fd586b5a640 1 -- 172.21.15.45:0/3057741266 shutdown_connections 2024-06-06T04:34:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.779+0000 7fd586b5a640 1 --2- 172.21.15.45:0/3057741266 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fd5680a2b90 0x7fd5680a5050 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.779+0000 7fd586b5a640 1 --2- 172.21.15.45:0/3057741266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd580102bf0 0x7fd580198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.779+0000 7fd586b5a640 1 --2- 172.21.15.45:0/3057741266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5801035b0 0x7fd58019f180 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.779+0000 7fd586b5a640 1 --2- 172.21.15.45:0/3057741266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd580101a00 0x7fd580197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.779+0000 7fd586b5a640 1 -- 172.21.15.45:0/3057741266 >> 172.21.15.45:0/3057741266 conn(0x7fd5800fcf70 msgr2=0x7fd580105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.779+0000 7fd586b5a640 1 -- 172.21.15.45:0/3057741266 shutdown_connections 2024-06-06T04:34:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.779+0000 7fd586b5a640 1 -- 172.21.15.45:0/3057741266 wait complete. 2024-06-06T04:34:39.791 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-06T04:34:39.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.898+0000 7f3e2c49a640 1 -- 172.21.15.45:0/4058315864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e24101a00 msgr2=0x7f3e24101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:39.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.898+0000 7f3e2c49a640 1 --2- 172.21.15.45:0/4058315864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e24101a00 0x7f3e24101e20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3e20009900 tx=0x7f3e2002a410 comp rx=0 tx=0).stop 2024-06-06T04:34:39.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.898+0000 7f3e2c49a640 1 -- 172.21.15.45:0/4058315864 shutdown_connections 2024-06-06T04:34:39.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.898+0000 7f3e2c49a640 1 --2- 172.21.15.45:0/4058315864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e24102bf0 0x7f3e24103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.898+0000 7f3e2c49a640 1 --2- 172.21.15.45:0/4058315864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e241035b0 0x7f3e24109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.898+0000 7f3e2c49a640 1 --2- 172.21.15.45:0/4058315864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e24101a00 0x7f3e24101e20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.898+0000 7f3e2c49a640 1 -- 172.21.15.45:0/4058315864 >> 172.21.15.45:0/4058315864 conn(0x7f3e240fcf70 msgr2=0x7f3e240ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:39.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.898+0000 7f3e2c49a640 1 -- 172.21.15.45:0/4058315864 shutdown_connections 2024-06-06T04:34:39.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.898+0000 7f3e2c49a640 1 -- 172.21.15.45:0/4058315864 wait complete. 2024-06-06T04:34:39.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.898+0000 7f3e2c49a640 1 Processor -- start 2024-06-06T04:34:39.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2c49a640 1 -- start start 2024-06-06T04:34:39.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2c49a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e24101a00 0x7f3e24197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:39.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2c49a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e24102bf0 0x7f3e241981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:39.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2c49a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e241035b0 0x7f3e2419f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:39.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2c49a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2410d530 con 0x7f3e24101a00 2024-06-06T04:34:39.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2c49a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2410d3b0 con 0x7f3e241035b0 2024-06-06T04:34:39.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2c49a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2410d6b0 con 0x7f3e24102bf0 2024-06-06T04:34:39.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2a236640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e24101a00 0x7f3e24197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:39.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2a236640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e24101a00 0x7f3e24197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44694/0 (socket says 172.21.15.45:44694) 2024-06-06T04:34:39.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2a236640 1 -- 172.21.15.45:0/1974977978 learned_addr learned my addr 172.21.15.45:0/1974977978 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:39.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2aa37640 1 --2- 172.21.15.45:0/1974977978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e241035b0 0x7f3e2419f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:39.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e29a35640 1 --2- 172.21.15.45:0/1974977978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e24102bf0 0x7f3e241981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:39.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2a236640 1 -- 172.21.15.45:0/1974977978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e24102bf0 msgr2=0x7f3e241981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:39.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2a236640 1 --2- 172.21.15.45:0/1974977978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e24102bf0 0x7f3e241981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2a236640 1 -- 172.21.15.45:0/1974977978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e241035b0 msgr2=0x7f3e2419f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:39.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2a236640 1 --2- 172.21.15.45:0/1974977978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e241035b0 0x7f3e2419f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:39.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2a236640 1 -- 172.21.15.45:0/1974977978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2419f880 con 0x7f3e24101a00 2024-06-06T04:34:39.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.899+0000 7f3e2a236640 1 --2- 172.21.15.45:0/1974977978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e24101a00 0x7f3e24197c70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3e200314e0 tx=0x7f3e20031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:39.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.900+0000 7f3e1b7fe640 1 -- 172.21.15.45:0/1974977978 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2003f030 con 0x7f3e24101a00 2024-06-06T04:34:39.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.900+0000 7f3e1b7fe640 1 -- 172.21.15.45:0/1974977978 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e200044b0 con 0x7f3e24101a00 2024-06-06T04:34:39.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.900+0000 7f3e2c49a640 1 -- 172.21.15.45:0/1974977978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2419fb10 con 0x7f3e24101a00 2024-06-06T04:34:39.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.900+0000 7f3e1b7fe640 1 -- 172.21.15.45:0/1974977978 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e20035070 con 0x7f3e24101a00 2024-06-06T04:34:39.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.900+0000 7f3e2c49a640 1 -- 172.21.15.45:0/1974977978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e241a0000 con 0x7f3e24101a00 2024-06-06T04:34:39.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.901+0000 7f3e2c49a640 1 -- 172.21.15.45:0/1974977978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e24108e40 con 0x7f3e24101a00 2024-06-06T04:34:39.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.902+0000 7f3e1b7fe640 1 -- 172.21.15.45:0/1974977978 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3e20042030 con 0x7f3e24101a00 2024-06-06T04:34:39.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.902+0000 7f3e1b7fe640 1 --2- 172.21.15.45:0/1974977978 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3e100a2b90 0x7f3e100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:39.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.902+0000 7f3e29a35640 1 --2- 172.21.15.45:0/1974977978 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3e100a2b90 0x7f3e100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:39.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.902+0000 7f3e1b7fe640 1 -- 172.21.15.45:0/1974977978 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f3e200efd20 con 0x7f3e24101a00 2024-06-06T04:34:39.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.903+0000 7f3e29a35640 1 --2- 172.21.15.45:0/1974977978 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3e100a2b90 0x7f3e100a5050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400bbe0 tx=0x7f3e1400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:39.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:39.906+0000 7f3e1b7fe640 1 -- 172.21.15.45:0/1974977978 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3e200bc270 con 0x7f3e24101a00 2024-06-06T04:34:40.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.090+0000 7f3e2c49a640 1 -- 172.21.15.45:0/1974977978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3e2402d530 con 0x7f3e24101a00 2024-06-06T04:34:40.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.090+0000 7f3e1b7fe640 1 -- 172.21.15.45:0/1974977978 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7f3e200c0570 con 0x7f3e24101a00 2024-06-06T04:34:40.095 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-06-06T04:34:40.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.092+0000 7f3e2c49a640 1 -- 172.21.15.45:0/1974977978 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3e100a2b90 msgr2=0x7f3e100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:40.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.092+0000 7f3e2c49a640 1 --2- 172.21.15.45:0/1974977978 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3e100a2b90 0x7f3e100a5050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400bbe0 tx=0x7f3e1400b390 comp rx=0 tx=0).stop 2024-06-06T04:34:40.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.092+0000 7f3e2c49a640 1 -- 172.21.15.45:0/1974977978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e24101a00 msgr2=0x7f3e24197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:40.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.092+0000 7f3e2c49a640 1 --2- 172.21.15.45:0/1974977978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e24101a00 0x7f3e24197c70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3e200314e0 tx=0x7f3e20031a30 comp rx=0 tx=0).stop 2024-06-06T04:34:40.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.092+0000 7f3e2c49a640 1 -- 172.21.15.45:0/1974977978 shutdown_connections 2024-06-06T04:34:40.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.092+0000 7f3e2c49a640 1 --2- 172.21.15.45:0/1974977978 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f3e100a2b90 0x7f3e100a5050 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.092+0000 7f3e2c49a640 1 --2- 172.21.15.45:0/1974977978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e241035b0 0x7f3e2419f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.092+0000 7f3e2c49a640 1 --2- 172.21.15.45:0/1974977978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e24102bf0 0x7f3e241981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.092+0000 7f3e2c49a640 1 --2- 172.21.15.45:0/1974977978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e24101a00 0x7f3e24197c70 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.092+0000 7f3e2c49a640 1 -- 172.21.15.45:0/1974977978 >> 172.21.15.45:0/1974977978 conn(0x7f3e240fcf70 msgr2=0x7f3e24106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:40.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.092+0000 7f3e2c49a640 1 -- 172.21.15.45:0/1974977978 shutdown_connections 2024-06-06T04:34:40.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.092+0000 7f3e2c49a640 1 -- 172.21.15.45:0/1974977978 wait complete. 2024-06-06T04:34:40.104 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-06T04:34:40.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.211+0000 7f97b91c1640 1 -- 172.21.15.45:0/3311089919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97b41076c0 msgr2=0x7f97b4109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:40.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.211+0000 7f97b91c1640 1 --2- 172.21.15.45:0/3311089919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97b41076c0 0x7f97b4109b10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009ca0 tx=0x7f97ac02a8c0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.211+0000 7f97b91c1640 1 -- 172.21.15.45:0/3311089919 shutdown_connections 2024-06-06T04:34:40.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.211+0000 7f97b91c1640 1 --2- 172.21.15.45:0/3311089919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97b41076c0 0x7f97b4109b10 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.211+0000 7f97b91c1640 1 --2- 172.21.15.45:0/3311089919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97b4104d90 0x7f97b4107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.211+0000 7f97b91c1640 1 --2- 172.21.15.45:0/3311089919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97b4102460 0x7f97b4104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.211+0000 7f97b91c1640 1 -- 172.21.15.45:0/3311089919 >> 172.21.15.45:0/3311089919 conn(0x7f97b40fbe50 msgr2=0x7f97b40fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:40.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.211+0000 7f97b91c1640 1 -- 172.21.15.45:0/3311089919 shutdown_connections 2024-06-06T04:34:40.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.211+0000 7f97b91c1640 1 -- 172.21.15.45:0/3311089919 wait complete. 2024-06-06T04:34:40.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.212+0000 7f97b91c1640 1 Processor -- start 2024-06-06T04:34:40.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.212+0000 7f97b91c1640 1 -- start start 2024-06-06T04:34:40.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b91c1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97b4102460 0x7f97b4101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:40.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b91c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97b4104d90 0x7f97b4100410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:40.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b91c1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97b41076c0 0x7f97b4100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:40.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b91c1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97b410d510 con 0x7f97b4104d90 2024-06-06T04:34:40.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b91c1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97b410d390 con 0x7f97b41076c0 2024-06-06T04:34:40.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b91c1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f97b410d690 con 0x7f97b4102460 2024-06-06T04:34:40.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97b4104d90 0x7f97b4100410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:40.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97b4104d90 0x7f97b4100410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44698/0 (socket says 172.21.15.45:44698) 2024-06-06T04:34:40.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b259c640 1 -- 172.21.15.45:0/38394639 learned_addr learned my addr 172.21.15.45:0/38394639 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b2d9d640 1 --2- 172.21.15.45:0/38394639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97b4102460 0x7f97b4101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b359e640 1 --2- 172.21.15.45:0/38394639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97b41076c0 0x7f97b4100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b259c640 1 -- 172.21.15.45:0/38394639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97b4102460 msgr2=0x7f97b4101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b259c640 1 --2- 172.21.15.45:0/38394639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97b4102460 0x7f97b4101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b259c640 1 -- 172.21.15.45:0/38394639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97b41076c0 msgr2=0x7f97b4100950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b259c640 1 --2- 172.21.15.45:0/38394639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97b41076c0 0x7f97b4100950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b259c640 1 -- 172.21.15.45:0/38394639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b41011e0 con 0x7f97b4104d90 2024-06-06T04:34:40.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b259c640 1 --2- 172.21.15.45:0/38394639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97b4104d90 0x7f97b4100410 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f97a800c970 tx=0x7f97a800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:40.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f979bfff640 1 -- 172.21.15.45:0/38394639 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a8007bb0 con 0x7f97b4104d90 2024-06-06T04:34:40.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f97b91c1640 1 -- 172.21.15.45:0/38394639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b41a1d40 con 0x7f97b4104d90 2024-06-06T04:34:40.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f979bfff640 1 -- 172.21.15.45:0/38394639 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97a8007d50 con 0x7f97b4104d90 2024-06-06T04:34:40.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.213+0000 7f979bfff640 1 -- 172.21.15.45:0/38394639 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a800f450 con 0x7f97b4104d90 2024-06-06T04:34:40.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.214+0000 7f97b91c1640 1 -- 172.21.15.45:0/38394639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b41a2220 con 0x7f97b4104d90 2024-06-06T04:34:40.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.214+0000 7f97b91c1640 1 -- 172.21.15.45:0/38394639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b40fd940 con 0x7f97b4104d90 2024-06-06T04:34:40.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.216+0000 7f979bfff640 1 -- 172.21.15.45:0/38394639 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f97a8004eb0 con 0x7f97b4104d90 2024-06-06T04:34:40.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.216+0000 7f979bfff640 1 --2- 172.21.15.45:0/38394639 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f979c0a2b90 0x7f979c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:40.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.216+0000 7f97b2d9d640 1 --2- 172.21.15.45:0/38394639 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f979c0a2b90 0x7f979c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:40.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.216+0000 7f979bfff640 1 -- 172.21.15.45:0/38394639 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f97a80cf010 con 0x7f97b4104d90 2024-06-06T04:34:40.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.217+0000 7f97b2d9d640 1 --2- 172.21.15.45:0/38394639 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f979c0a2b90 0x7f979c0a5050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f97a4005d00 tx=0x7f97a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:40.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.221+0000 7f979bfff640 1 -- 172.21.15.45:0/38394639 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f97a8014030 con 0x7f97b4104d90 2024-06-06T04:34:40.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.404+0000 7f97b91c1640 1 -- 172.21.15.45:0/38394639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f97b40625a0 con 0x7f97b4104d90 2024-06-06T04:34:40.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.405+0000 7f979bfff640 1 -- 172.21.15.45:0/38394639 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v100) ==== 103+0+11 (secure 0 0 0) 0x7f97a809b560 con 0x7f97b4104d90 2024-06-06T04:34:40.405 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-06-06T04:34:40.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.407+0000 7f97b91c1640 1 -- 172.21.15.45:0/38394639 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f979c0a2b90 msgr2=0x7f979c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:40.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.407+0000 7f97b91c1640 1 --2- 172.21.15.45:0/38394639 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f979c0a2b90 0x7f979c0a5050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f97a4005d00 tx=0x7f97a400b410 comp rx=0 tx=0).stop 2024-06-06T04:34:40.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.407+0000 7f97b91c1640 1 -- 172.21.15.45:0/38394639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97b4104d90 msgr2=0x7f97b4100410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.407+0000 7f97b91c1640 1 --2- 172.21.15.45:0/38394639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97b4104d90 0x7f97b4100410 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f97a800c970 tx=0x7f97a800ce30 comp rx=0 tx=0).stop 2024-06-06T04:34:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.407+0000 7f97b91c1640 1 -- 172.21.15.45:0/38394639 shutdown_connections 2024-06-06T04:34:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.407+0000 7f97b91c1640 1 --2- 172.21.15.45:0/38394639 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f979c0a2b90 0x7f979c0a5050 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.407+0000 7f97b91c1640 1 --2- 172.21.15.45:0/38394639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97b41076c0 0x7f97b4100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.407+0000 7f97b91c1640 1 --2- 172.21.15.45:0/38394639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97b4104d90 0x7f97b4100410 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.407+0000 7f97b91c1640 1 --2- 172.21.15.45:0/38394639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97b4102460 0x7f97b4101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.407+0000 7f97b91c1640 1 -- 172.21.15.45:0/38394639 >> 172.21.15.45:0/38394639 conn(0x7f97b40fbe50 msgr2=0x7f97b4102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:40.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.407+0000 7f97b91c1640 1 -- 172.21.15.45:0/38394639 shutdown_connections 2024-06-06T04:34:40.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.407+0000 7f97b91c1640 1 -- 172.21.15.45:0/38394639 wait complete. 2024-06-06T04:34:40.417 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:40 smithi045 ceph-mon[32299]: pgmap v527: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T04:34:40.417 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:40 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/106892133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T04:34:40.418 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:40 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3057741266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T04:34:40.418 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:40 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1974977978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T04:34:40.419 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-06-06T04:34:40.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.523+0000 7f4debdd5640 1 -- 172.21.15.45:0/3586234303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de4102490 msgr2=0x7f4de4108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:40.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.523+0000 7f4debdd5640 1 --2- 172.21.15.45:0/3586234303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de4102490 0x7f4de4108d20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009ca0 tx=0x7f4ddc02a8c0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.523+0000 7f4debdd5640 1 -- 172.21.15.45:0/3586234303 shutdown_connections 2024-06-06T04:34:40.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.523+0000 7f4debdd5640 1 --2- 172.21.15.45:0/3586234303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de4102490 0x7f4de4108d20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.523+0000 7f4debdd5640 1 --2- 172.21.15.45:0/3586234303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4de4101ad0 0x7f4de4101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.523+0000 7f4debdd5640 1 --2- 172.21.15.45:0/3586234303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4de41008e0 0x7f4de4100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.523+0000 7f4debdd5640 1 -- 172.21.15.45:0/3586234303 >> 172.21.15.45:0/3586234303 conn(0x7f4de40fbe50 msgr2=0x7f4de40fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:40.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.523+0000 7f4debdd5640 1 -- 172.21.15.45:0/3586234303 shutdown_connections 2024-06-06T04:34:40.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.523+0000 7f4debdd5640 1 -- 172.21.15.45:0/3586234303 wait complete. 2024-06-06T04:34:40.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.523+0000 7f4debdd5640 1 Processor -- start 2024-06-06T04:34:40.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.523+0000 7f4debdd5640 1 -- start start 2024-06-06T04:34:40.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4debdd5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de41008e0 0x7f4de4197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:40.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4debdd5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4de4101ad0 0x7f4de41981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:40.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4debdd5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4de4102490 0x7f4de419f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:40.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4debdd5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4de410c410 con 0x7f4de41008e0 2024-06-06T04:34:40.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4debdd5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4de410c290 con 0x7f4de4102490 2024-06-06T04:34:40.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4debdd5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4de410c590 con 0x7f4de4101ad0 2024-06-06T04:34:40.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4de9b71640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de41008e0 0x7f4de4197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:40.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4de9b71640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de41008e0 0x7f4de4197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44722/0 (socket says 172.21.15.45:44722) 2024-06-06T04:34:40.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4de9b71640 1 -- 172.21.15.45:0/4166876004 learned_addr learned my addr 172.21.15.45:0/4166876004 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:34:40.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4de9370640 1 --2- 172.21.15.45:0/4166876004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4de4101ad0 0x7f4de41981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:40.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4dea372640 1 --2- 172.21.15.45:0/4166876004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4de4102490 0x7f4de419f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:40.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4de9b71640 1 -- 172.21.15.45:0/4166876004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4de4101ad0 msgr2=0x7f4de41981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:40.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4de9b71640 1 --2- 172.21.15.45:0/4166876004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4de4101ad0 0x7f4de41981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4de9b71640 1 -- 172.21.15.45:0/4166876004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4de4102490 msgr2=0x7f4de419f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:40.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4de9b71640 1 --2- 172.21.15.45:0/4166876004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4de4102490 0x7f4de419f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:40.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4de9b71640 1 -- 172.21.15.45:0/4166876004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de419f8d0 con 0x7f4de41008e0 2024-06-06T04:34:40.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4de9b71640 1 --2- 172.21.15.45:0/4166876004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de41008e0 0x7f4de4197c90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400e9c0 tx=0x7f4dd400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:40.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4de2ffd640 1 -- 172.21.15.45:0/4166876004 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd400cd80 con 0x7f4de41008e0 2024-06-06T04:34:40.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4debdd5640 1 -- 172.21.15.45:0/4166876004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de419fbc0 con 0x7f4de41008e0 2024-06-06T04:34:40.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4de2ffd640 1 -- 172.21.15.45:0/4166876004 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dd40035f0 con 0x7f4de41008e0 2024-06-06T04:34:40.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.524+0000 7f4de2ffd640 1 -- 172.21.15.45:0/4166876004 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4010690 con 0x7f4de41008e0 2024-06-06T04:34:40.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.525+0000 7f4debdd5640 1 -- 172.21.15.45:0/4166876004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de41a0160 con 0x7f4de41008e0 2024-06-06T04:34:40.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.525+0000 7f4debdd5640 1 -- 172.21.15.45:0/4166876004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de41a0330 con 0x7f4de41008e0 2024-06-06T04:34:40.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.527+0000 7f4de2ffd640 1 -- 172.21.15.45:0/4166876004 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4dd40037a0 con 0x7f4de41008e0 2024-06-06T04:34:40.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.527+0000 7f4de2ffd640 1 --2- 172.21.15.45:0/4166876004 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4dcc0a2b90 0x7f4dcc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:34:40.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.527+0000 7f4de9370640 1 --2- 172.21.15.45:0/4166876004 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4dcc0a2b90 0x7f4dcc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:34:40.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.527+0000 7f4de2ffd640 1 -- 172.21.15.45:0/4166876004 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f4dd40cf7a0 con 0x7f4de41008e0 2024-06-06T04:34:40.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.528+0000 7f4de9370640 1 --2- 172.21.15.45:0/4166876004 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4dcc0a2b90 0x7f4dcc0a5050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4de4199190 tx=0x7f4dd800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:34:40.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.532+0000 7f4de2ffd640 1 -- 172.21.15.45:0/4166876004 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4dd409c340 con 0x7f4de41008e0 2024-06-06T04:34:40.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:40 smithi086 ceph-mon[36053]: pgmap v527: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T04:34:40.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:40 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/106892133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T04:34:40.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:40 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3057741266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T04:34:40.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:40 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1974977978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T04:34:40.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:40 smithi154 ceph-mon[36428]: pgmap v527: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-06T04:34:40.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:40 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/106892133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T04:34:40.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:40 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3057741266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T04:34:40.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:40 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1974977978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T04:34:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.769+0000 7f4debdd5640 1 -- 172.21.15.45:0/4166876004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f4de41a05a0 con 0x7f4de41008e0 2024-06-06T04:34:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.778+0000 7f4de2ffd640 1 -- 172.21.15.45:0/4166876004 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740371 (secure 0 0 0) 0x7f4dd409bc40 con 0x7f4de41008e0 2024-06-06T04:34:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:34:40.788 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:34:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 20, 2024-06-06T04:34:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: "active_gid": 14152, 2024-06-06T04:34:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: "active_name": "x", 2024-06-06T04:34:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addrs": { 2024-06-06T04:34:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-06-06T04:34:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-06-06T04:34:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6800", 2024-06-06T04:34:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1718508550 2024-06-06T04:34:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-06-06T04:34:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6801", 2024-06-06T04:34:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1718508550 2024-06-06T04:34:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-06-06T04:34:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addr": "172.21.15.45:6801/1718508550", 2024-06-06T04:34:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: "active_change": "2024-06-06T04:20:04.267379+0000", 2024-06-06T04:34:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: "active_mgr_features": 4540701547738038271, 2024-06-06T04:34:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: "available": true, 2024-06-06T04:34:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: "standbys": [ 2024-06-06T04:34:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 24109, 2024-06-06T04:34:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "z", 2024-06-06T04:34:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540701547738038271, 2024-06-06T04:34:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-06-06T04:34:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-06-06T04:34:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-06-06T04:34:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-06-06T04:34:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-06-06T04:34:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-06-06T04:34:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-06-06T04:34:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-06-06T04:34:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-06-06T04:34:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-06-06T04:34:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-06-06T04:34:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-06-06T04:34:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-06-06T04:34:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-06-06T04:34:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-06-06T04:34:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-06-06T04:34:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-06-06T04:34:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-06-06T04:34:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-06-06T04:34:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-06-06T04:34:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-06-06T04:34:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-06-06T04:34:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-06-06T04:34:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-06-06T04:34:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-06-06T04:34:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-06-06T04:34:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-06-06T04:34:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-06-06T04:34:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-06T04:34:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-06-06T04:34:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-06-06T04:34:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-06-06T04:34:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-06-06T04:34:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-06-06T04:34:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-06-06T04:34:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-06-06T04:34:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-06-06T04:34:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-06-06T04:34:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-06-06T04:34:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-06-06T04:34:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T04:34:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-06-06T04:34:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-06-06T04:34:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-06-06T04:34:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-06-06T04:34:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-06-06T04:34:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-06-06T04:34:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-06T04:34:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T04:34:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-06-06T04:34:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-06-06T04:34:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-06-06T04:34:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-06-06T04:34:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-06-06T04:34:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-06-06T04:34:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-06T04:34:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-06-06T04:34:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-06-06T04:34:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-06-06T04:34:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-06T04:34:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-06T04:34:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-06-06T04:34:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-06-06T04:34:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-06-06T04:34:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-06-06T04:34:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-06-06T04:34:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-06-06T04:34:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-06T04:34:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-06-06T04:34:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-06-06T04:34:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-06-06T04:34:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-06-06T04:34:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T04:34:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-06-06T04:34:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-06-06T04:34:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-06-06T04:34:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-06-06T04:34:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-06-06T04:34:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-06-06T04:34:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-06T04:34:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T04:34:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-06-06T04:34:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-06-06T04:34:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-06-06T04:34:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-06T04:34:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-06-06T04:34:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-06-06T04:34:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-06-06T04:34:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-06-06T04:34:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-06-06T04:34:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-06-06T04:34:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-06-06T04:34:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-06-06T04:34:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-06-06T04:34:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-06-06T04:34:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-06-06T04:34:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-06T04:34:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-06-06T04:34:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-06-06T04:34:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-06-06T04:34:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-06T04:34:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-06-06T04:34:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-06-06T04:34:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-06-06T04:34:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-06-06T04:34:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-06T04:34:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-06T04:34:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-06-06T04:34:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-06-06T04:34:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-06-06T04:34:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-06T04:34:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-06-06T04:34:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-06-06T04:34:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-06-06T04:34:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-06-06T04:34:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-06T04:34:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-06-06T04:34:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-06-06T04:34:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-06-06T04:34:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-06T04:34:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-06-06T04:34:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-06-06T04:34:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-06-06T04:34:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-06T04:34:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-06-06T04:34:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-06-06T04:34:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-06T04:34:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-06T04:34:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-06-06T04:34:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-06-06T04:34:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-06-06T04:34:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-06T04:34:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-06-06T04:34:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-06-06T04:34:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-06-06T04:34:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-06T04:34:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-06-06T04:34:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-06-06T04:34:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-06-06T04:34:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-06-06T04:34:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-06-06T04:34:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-06T04:34:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-06-06T04:34:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-06-06T04:34:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-06T04:34:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-06-06T04:34:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-06-06T04:34:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-06T04:34:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-06-06T04:34:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-06-06T04:34:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-06T04:34:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-06-06T04:34:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-06-06T04:34:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-06T04:34:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-06-06T04:34:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-06-06T04:34:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-06-06T04:34:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-06T04:34:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-06-06T04:34:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-06-06T04:34:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-06-06T04:34:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-06T04:34:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-06-06T04:34:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-06-06T04:34:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-06-06T04:34:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-06T04:34:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-06-06T04:34:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-06-06T04:34:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-06-06T04:34:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-06T04:34:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-06-06T04:34:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-06-06T04:34:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-06-06T04:34:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-06T04:34:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-06-06T04:34:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-06-06T04:34:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-06-06T04:34:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-06T04:34:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-06-06T04:34:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-06-06T04:34:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-06-06T04:34:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-06T04:34:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-06-06T04:34:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-06-06T04:34:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-06-06T04:34:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-06T04:34:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-06-06T04:34:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-06-06T04:34:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-06-06T04:34:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-06T04:34:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-06-06T04:34:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-06-06T04:34:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-06-06T04:34:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-06T04:34:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-06-06T04:34:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-06-06T04:34:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-06-06T04:34:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-06T04:34:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-06-06T04:34:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-06-06T04:34:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-06-06T04:34:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-06T04:34:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-06-06T04:34:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-06-06T04:34:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-06-06T04:34:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-06T04:34:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-06-06T04:34:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-06-06T04:34:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-06-06T04:34:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-06T04:34:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-06-06T04:34:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-06-06T04:34:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-06-06T04:34:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-06T04:34:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-06-06T04:34:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-06-06T04:34:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-06-06T04:34:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-06-06T04:34:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-06-06T04:34:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-06-06T04:34:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-06-06T04:34:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-06T04:34:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-06-06T04:34:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-06-06T04:34:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-06-06T04:34:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-06T04:34:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-06-06T04:34:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-06-06T04:34:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-06-06T04:34:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-06T04:34:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-06-06T04:34:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-06-06T04:34:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-06-06T04:34:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-06-06T04:34:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-06-06T04:34:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-06-06T04:34:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-06T04:34:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-06-06T04:34:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-06-06T04:34:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-06-06T04:34:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-06-06T04:34:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-06-06T04:34:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-06-06T04:34:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-06T04:34:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-06T04:34:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-06-06T04:34:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-06-06T04:34:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-06-06T04:34:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-06-06T04:34:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-06-06T04:34:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-06-06T04:34:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-06T04:34:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-06-06T04:34:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-06-06T04:34:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-06T04:34:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:40.924 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:40.924 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.924 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.924 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.924 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.924 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.924 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.924 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:40.925 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:40.925 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:40.925 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:40.925 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:40.925 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:40.925 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:40.925 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:40.925 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.926 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.926 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.926 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.926 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-06-06T04:34:40.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-06-06T04:34:40.926 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.926 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.927 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.927 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.927 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.927 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.927 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.927 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-06T04:34:40.927 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.928 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.928 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.928 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:40.928 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:40.928 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.928 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.928 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.929 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.929 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.929 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.929 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.929 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-06T04:34:40.929 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.929 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.929 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.930 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:40.930 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:40.930 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.930 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.930 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.930 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:40.930 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.931 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.931 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:40.931 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:40.931 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:40.931 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:40.931 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:40.931 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:40.931 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:40.932 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:40.932 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.932 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.932 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.932 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.932 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:40.932 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:40.933 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.933 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.933 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.933 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.933 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.933 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.933 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.933 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.933 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.934 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.934 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.934 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-06T04:34:40.934 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-06T04:34:40.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.934 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.934 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.935 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.935 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.935 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.935 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.935 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-06T04:34:40.935 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.935 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.935 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.936 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-06T04:34:40.936 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-06T04:34:40.936 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.936 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.936 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.936 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-06-06T04:34:40.936 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.936 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.937 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.937 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-06T04:34:40.937 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.937 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.937 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.937 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-06-06T04:34:40.937 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-06-06T04:34:40.938 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:40.938 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.938 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.938 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:40.938 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.938 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.938 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.938 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-06-06T04:34:40.939 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.939 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.939 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.939 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-06-06T04:34:40.939 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-06-06T04:34:40.939 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:40.939 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.940 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.940 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:40.940 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.940 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.940 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.940 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-06T04:34:40.940 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.940 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.941 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.941 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-06-06T04:34:40.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-06-06T04:34:40.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:40.941 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.941 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.942 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.942 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.942 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.942 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-06-06T04:34:40.942 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.942 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.942 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.943 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-06-06T04:34:40.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-06-06T04:34:40.943 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.943 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.943 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.943 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-06-06T04:34:40.943 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.943 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.943 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:40.944 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-06-06T04:34:40.944 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-06-06T04:34:40.944 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:40.944 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-06-06T04:34:40.944 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.944 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.944 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.945 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-06-06T04:34:40.945 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-06-06T04:34:40.945 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.945 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.945 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.945 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-06-06T04:34:40.945 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.945 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.946 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.946 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-06T04:34:40.946 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.946 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.946 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.946 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-06-06T04:34:40.946 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-06-06T04:34:40.946 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.947 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.947 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.947 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-06T04:34:40.947 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.947 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.947 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.947 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-06T04:34:40.947 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.948 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.948 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.948 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-06-06T04:34:40.948 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-06-06T04:34:40.948 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.948 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.949 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.949 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.949 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.949 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.949 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.949 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-06T04:34:40.949 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.949 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.950 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.950 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-06-06T04:34:40.950 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-06-06T04:34:40.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.950 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.950 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.950 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.951 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.951 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.951 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.951 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-06T04:34:40.951 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.951 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.952 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-06-06T04:34:40.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-06-06T04:34:40.952 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.952 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.952 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.952 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.952 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.952 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.952 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.953 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-06T04:34:40.953 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.953 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.953 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.953 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-06-06T04:34:40.954 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-06-06T04:34:40.954 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.954 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.954 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.954 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.954 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.954 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.954 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.955 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-06T04:34:40.955 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.955 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.955 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.955 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-06-06T04:34:40.955 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-06-06T04:34:40.955 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.956 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.956 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.956 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.956 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.956 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.956 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.956 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-06T04:34:40.956 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-06-06T04:34:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-06-06T04:34:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.957 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-06-06T04:34:40.958 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.958 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.958 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.958 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-06-06T04:34:40.958 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.958 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.958 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.959 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-06-06T04:34:40.959 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-06-06T04:34:40.959 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.959 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.959 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.959 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.959 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.959 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.960 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.960 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-06T04:34:40.960 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.960 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.960 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.960 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-06-06T04:34:40.960 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-06-06T04:34:40.961 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.961 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.961 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.961 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.961 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.961 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.961 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.961 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-06T04:34:40.961 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.962 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.962 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.962 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-06-06T04:34:40.962 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-06-06T04:34:40.962 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.962 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.962 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.963 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.963 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.963 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.963 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.963 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-06T04:34:40.963 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.963 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.964 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-06-06T04:34:40.964 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-06-06T04:34:40.964 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.964 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.964 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.964 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.964 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.965 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.965 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.965 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-06-06T04:34:40.965 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.965 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.965 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.965 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-06-06T04:34:40.966 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-06-06T04:34:40.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.966 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.966 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.966 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.966 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.966 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.966 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.967 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-06T04:34:40.967 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.967 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.967 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.967 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-06-06T04:34:40.967 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-06-06T04:34:40.967 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.968 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.968 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.968 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.968 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.968 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.968 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.968 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-06T04:34:40.968 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.969 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.969 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.969 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:40.969 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:40.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:40.969 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-06-06T04:34:40.969 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:40.970 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:40.970 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:40.970 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:40.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:40.970 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.970 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.970 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.970 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.971 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.971 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.971 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:40.971 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:40.971 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:40.971 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:40.971 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:40.971 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:40.972 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:40.972 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:40.972 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.972 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.972 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.972 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.972 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:40.973 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:40.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.973 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.973 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.973 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.973 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.973 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.974 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.974 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.974 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.974 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.974 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:40.974 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:40.974 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.975 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.975 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:40.975 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.975 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.975 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:40.975 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:40.975 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:40.976 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:40.976 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:40.976 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:40.976 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:40.976 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:40.976 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.976 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.976 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.976 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.977 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:40.977 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:40.977 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.977 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.977 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.977 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.977 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.978 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.978 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.978 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.978 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.978 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.978 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.978 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-06-06T04:34:40.979 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-06-06T04:34:40.979 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:40.979 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.979 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.979 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-06-06T04:34:40.979 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.979 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.979 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.980 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-06T04:34:40.980 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.980 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.980 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.980 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-06-06T04:34:40.980 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-06-06T04:34:40.980 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:40.981 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.981 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:40.981 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-06-06T04:34:40.981 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.981 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.981 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.981 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-06T04:34:40.981 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.981 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.982 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.982 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:40.982 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:40.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:40.982 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-06-06T04:34:40.982 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:40.982 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:40.983 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:40.983 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-06T04:34:40.983 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-06T04:34:40.983 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:40.983 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.983 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.983 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:40.983 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.984 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.984 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.984 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.984 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.984 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.984 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.984 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-06-06T04:34:40.985 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-06T04:34:40.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:40.985 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.985 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.985 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:40.985 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.985 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.985 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.986 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.986 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.986 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.986 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.986 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-06T04:34:40.986 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-06T04:34:40.986 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.986 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.987 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.987 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.987 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.987 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.987 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.987 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.987 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.987 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.988 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.988 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-06-06T04:34:40.988 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-06-06T04:34:40.988 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.988 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.988 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.988 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:40.989 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.989 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.989 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.989 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.989 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.989 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.989 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.990 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-06T04:34:40.990 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-06T04:34:40.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.990 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.990 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.990 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.990 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.990 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.991 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.991 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.991 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.991 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.991 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.991 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-06-06T04:34:40.991 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-06T04:34:40.992 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.992 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.992 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.992 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.992 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.992 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.992 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.992 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.992 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.993 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.993 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.993 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-06T04:34:40.993 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-06T04:34:40.993 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.993 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.994 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.994 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.994 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.994 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.994 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.994 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.994 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.994 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.995 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.995 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-06T04:34:40.995 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-06T04:34:40.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.995 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.995 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.995 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.996 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.996 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.996 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.996 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.996 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.996 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.996 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.997 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-06T04:34:40.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-06T04:34:40.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.997 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.997 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.997 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.997 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.997 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.998 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.998 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:40.998 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:40.998 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:40.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:40.998 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-06T04:34:40.998 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-06T04:34:40.998 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:40.999 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:40.999 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:40.999 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:40.999 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:40.999 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:40.999 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:40.999 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:40.999 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.000 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.000 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.000 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-06T04:34:41.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-06T04:34:41.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.000 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.000 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.001 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.001 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.001 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.001 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.001 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.001 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.001 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.001 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.002 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-06T04:34:41.002 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-06T04:34:41.002 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.002 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.002 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.002 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.002 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.003 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.003 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.003 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.003 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.003 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.003 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.003 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-06T04:34:41.004 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-06T04:34:41.004 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.004 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.004 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.004 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.004 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.004 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.004 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.004 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.005 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.005 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.005 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.005 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-06T04:34:41.005 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-06T04:34:41.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.005 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.006 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.006 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.006 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.006 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.006 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.006 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.006 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.006 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.007 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.007 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-06-06T04:34:41.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-06T04:34:41.007 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.007 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.007 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.007 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-06-06T04:34:41.008 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.008 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.008 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.008 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.008 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.008 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.008 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.009 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-06T04:34:41.009 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-06T04:34:41.009 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.009 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.009 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.009 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.009 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.009 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.009 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.010 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.010 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.010 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.010 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.010 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-06-06T04:34:41.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-06-06T04:34:41.010 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.011 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.011 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.011 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.011 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.011 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.011 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.011 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.011 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.012 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.012 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.012 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-06-06T04:34:41.012 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-06T04:34:41.012 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.012 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.012 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.013 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-06-06T04:34:41.013 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.013 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.013 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.013 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.013 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.013 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.013 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.014 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-06T04:34:41.014 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-06T04:34:41.014 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.014 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.014 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.014 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.014 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.015 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.015 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.015 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.015 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.015 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.015 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.015 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-06T04:34:41.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-06T04:34:41.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.016 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.016 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.016 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.016 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.016 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.016 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.017 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.017 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.017 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.017 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-06T04:34:41.017 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-06T04:34:41.017 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.018 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.018 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.018 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.018 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.018 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.018 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.018 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.018 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.019 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.019 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.019 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-06T04:34:41.019 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-06T04:34:41.019 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.019 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.019 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.020 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.020 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.020 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.020 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.020 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.020 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.020 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.020 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.021 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-06-06T04:34:41.021 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-06T04:34:41.021 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.021 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.021 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.021 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.021 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.022 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.022 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.022 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.022 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.022 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.022 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.022 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-06T04:34:41.023 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-06T04:34:41.023 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.023 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.023 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.023 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.023 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.023 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.023 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.024 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.024 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.024 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.024 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.024 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-06T04:34:41.024 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-06T04:34:41.024 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.024 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.025 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.025 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.025 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.025 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.025 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.025 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.025 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.025 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.026 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.026 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-06T04:34:41.026 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-06T04:34:41.026 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.026 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.026 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.026 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.027 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.027 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.027 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.027 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.027 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.027 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.027 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.028 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-06T04:34:41.028 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-06T04:34:41.028 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.028 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.028 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.028 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.028 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.028 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.029 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.029 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.029 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.029 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.029 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.029 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-06T04:34:41.029 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-06T04:34:41.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.030 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.030 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.030 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.030 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.030 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.030 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.030 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.030 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.031 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.031 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.031 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-06T04:34:41.031 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-06T04:34:41.031 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.031 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.031 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.032 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.032 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.032 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.032 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.032 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.032 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.032 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.032 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.033 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-06T04:34:41.033 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-06T04:34:41.033 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.033 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.033 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.033 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.033 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.034 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.034 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.034 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.034 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.034 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.034 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.034 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-06T04:34:41.035 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-06T04:34:41.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.035 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.035 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.035 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.035 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.035 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.035 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.036 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.036 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.036 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.036 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.036 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-06-06T04:34:41.036 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-06T04:34:41.036 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.036 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.037 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.037 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.037 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.037 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.037 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.037 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.037 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.037 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.038 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.038 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-06T04:34:41.038 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-06T04:34:41.038 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.038 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.038 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.039 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-06T04:34:41.039 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.039 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.039 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.039 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.039 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.039 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.039 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.040 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-06T04:34:41.040 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-06T04:34:41.040 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.040 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.040 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.040 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:41.040 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.040 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.041 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.041 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.041 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.041 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.041 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.041 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-06T04:34:41.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-06T04:34:41.042 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.042 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.042 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.042 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-06-06T04:34:41.042 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.042 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.042 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.042 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.043 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.043 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.043 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.043 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-06T04:34:41.043 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-06T04:34:41.043 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.043 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.044 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.044 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-06-06T04:34:41.044 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.044 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.044 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.044 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.044 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.044 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.045 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.045 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-06-06T04:34:41.045 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-06T04:34:41.045 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.045 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.045 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.045 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.046 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.046 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.046 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.046 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.046 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.046 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.046 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.047 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-06T04:34:41.047 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-06T04:34:41.047 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.047 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.047 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.047 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-06-06T04:34:41.047 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.047 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.048 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.048 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.048 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.048 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.048 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.048 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-06-06T04:34:41.048 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-06T04:34:41.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.049 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.049 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.049 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.049 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.049 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.049 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.049 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.049 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.050 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.050 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.050 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-06-06T04:34:41.050 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-06T04:34:41.050 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.050 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.050 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.051 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.051 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.051 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.051 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.051 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.051 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.051 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.051 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.052 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-06T04:34:41.052 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-06T04:34:41.052 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.052 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.052 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.052 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-06-06T04:34:41.052 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.053 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.053 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.053 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.053 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.053 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.053 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.053 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-06T04:34:41.054 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-06T04:34:41.054 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.054 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.054 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.054 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:41.054 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.054 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.054 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.055 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.055 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.055 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.055 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.055 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-06T04:34:41.055 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-06T04:34:41.055 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.056 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.056 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.056 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-06-06T04:34:41.056 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.056 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.056 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.056 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.056 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.056 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.057 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.057 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-06-06T04:34:41.057 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-06-06T04:34:41.057 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.057 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.057 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.057 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.058 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.058 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.058 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.058 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.058 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.058 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.058 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.059 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-06-06T04:34:41.059 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-06-06T04:34:41.059 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.059 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.059 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.059 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.059 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.059 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.060 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.060 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.060 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.060 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.060 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.060 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-06-06T04:34:41.060 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-06-06T04:34:41.061 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.061 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.061 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.061 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.061 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.061 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.061 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.062 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-06-06T04:34:41.062 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.062 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.062 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.062 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-06-06T04:34:41.062 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-06-06T04:34:41.062 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.063 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.063 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.063 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-06-06T04:34:41.063 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.063 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.063 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.063 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.063 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.064 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.064 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.064 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-06-06T04:34:41.064 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-06-06T04:34:41.064 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.064 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.064 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.064 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.065 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.065 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.065 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.065 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.065 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.065 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.065 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.066 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.066 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.066 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.066 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.066 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.066 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.066 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.066 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.067 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.067 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.067 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.067 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.067 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.067 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.067 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.067 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.068 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.068 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.068 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.068 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.068 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.068 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.068 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.069 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.069 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.069 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.069 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.069 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.069 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.069 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.069 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.069 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.070 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.070 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.070 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.070 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.070 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.070 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.070 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.071 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.071 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.071 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.071 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.071 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.071 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.072 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.072 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.072 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.072 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.072 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.072 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.072 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.072 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.073 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.073 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.073 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.073 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.073 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.073 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.073 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.074 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.074 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.074 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.074 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.074 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.074 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.074 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-06-06T04:34:41.075 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-06-06T04:34:41.075 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.075 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.075 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.075 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.075 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.075 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.075 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.076 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-06-06T04:34:41.076 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.076 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.076 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.076 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-06-06T04:34:41.076 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-06-06T04:34:41.076 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.077 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.077 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.077 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.077 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.077 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.077 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.077 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.077 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.077 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.078 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.078 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-06-06T04:34:41.078 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-06-06T04:34:41.078 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.078 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.078 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.078 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-06-06T04:34:41.079 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.079 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.079 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.079 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.079 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.079 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.079 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.080 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-06-06T04:34:41.080 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-06-06T04:34:41.080 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.080 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.080 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.080 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-06-06T04:34:41.080 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.080 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.081 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.081 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.081 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.081 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.081 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.081 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-06-06T04:34:41.081 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-06-06T04:34:41.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.082 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.082 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.082 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.082 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.082 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.082 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.082 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.083 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.083 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.083 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.083 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-06-06T04:34:41.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-06-06T04:34:41.083 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.083 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.084 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.084 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-06-06T04:34:41.084 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.084 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.084 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.084 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.084 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.084 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.085 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.085 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-06-06T04:34:41.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-06-06T04:34:41.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.085 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.085 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.085 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-06-06T04:34:41.086 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.086 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.086 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.086 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.086 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-06-06T04:34:41.086 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.086 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.086 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.087 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.087 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.087 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-06-06T04:34:41.087 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-06-06T04:34:41.087 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.087 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.087 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.088 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-06-06T04:34:41.088 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-06-06T04:34:41.088 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-06-06T04:34:41.088 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.088 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.088 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.088 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.088 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.089 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-06-06T04:34:41.089 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-06-06T04:34:41.089 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.089 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.089 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.089 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.089 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.090 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.090 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.090 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.090 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.090 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.090 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.090 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.090 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.091 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-06-06T04:34:41.091 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.091 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.091 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.091 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-06-06T04:34:41.091 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-06-06T04:34:41.092 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.092 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.092 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.092 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.092 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.092 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.092 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.092 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-06-06T04:34:41.093 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.093 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.093 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.093 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.093 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.093 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.093 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.094 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.094 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.094 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.094 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.094 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.094 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.094 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.094 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.095 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.095 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.095 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.095 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.095 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.095 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.095 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.095 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.096 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.096 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.096 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.096 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.096 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.096 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.096 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.096 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.097 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.097 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.097 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.097 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.097 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.097 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.097 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.098 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.098 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.098 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.098 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.098 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.098 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.098 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.098 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.099 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.099 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.099 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.099 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.099 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.099 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.099 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.099 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.100 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.100 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.100 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.100 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.100 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.100 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.101 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.101 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.101 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.101 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.101 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.101 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.101 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.101 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.102 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-06-06T04:34:41.102 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-06-06T04:34:41.102 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.102 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.102 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.102 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-06-06T04:34:41.102 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.102 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.103 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.103 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-06T04:34:41.103 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.103 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.103 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.103 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-06-06T04:34:41.103 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-06-06T04:34:41.104 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.104 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.104 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.104 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-06-06T04:34:41.104 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.104 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.104 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.104 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-06T04:34:41.105 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.105 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.105 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.105 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-06-06T04:34:41.105 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-06-06T04:34:41.105 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.105 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.106 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.106 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-06-06T04:34:41.106 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.106 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.106 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.106 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-06-06T04:34:41.106 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.106 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.107 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.107 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-06-06T04:34:41.107 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-06-06T04:34:41.107 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.107 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.107 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.107 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-06-06T04:34:41.108 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.108 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.108 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.108 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-06T04:34:41.108 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.108 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.108 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.109 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-06-06T04:34:41.109 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-06-06T04:34:41.109 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.109 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.109 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.109 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.109 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.109 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.110 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.110 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-06T04:34:41.110 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.110 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.110 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.110 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-06-06T04:34:41.114 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-06-06T04:34:41.114 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.114 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.114 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.114 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-06-06T04:34:41.114 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.115 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.115 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.115 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-06-06T04:34:41.115 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.115 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.115 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.115 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-06-06T04:34:41.116 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-06-06T04:34:41.116 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.116 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.116 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.116 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-06-06T04:34:41.116 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.116 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.116 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.116 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-06T04:34:41.117 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.117 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.117 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.117 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.117 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.117 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-06-06T04:34:41.118 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.118 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.118 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.118 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.118 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.118 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.118 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.119 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.119 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.119 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.119 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.119 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.119 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.119 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.119 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.120 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.120 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.120 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.120 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.120 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.120 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.120 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.121 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.121 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.121 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.121 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.121 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.121 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.121 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.122 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.122 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.122 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.122 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.122 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.122 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.122 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.123 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.123 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.123 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.123 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.123 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.123 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.123 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.123 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.124 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.124 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.124 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.124 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.124 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.124 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.124 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.124 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.125 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.125 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.125 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.125 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.125 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.125 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.125 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.125 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.126 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.126 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.126 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.126 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.126 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.126 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.126 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.127 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.127 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-06-06T04:34:41.127 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-06-06T04:34:41.127 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.127 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.127 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.127 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-06-06T04:34:41.127 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.128 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.128 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.128 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.128 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.128 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.128 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.128 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-06-06T04:34:41.129 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-06-06T04:34:41.129 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.129 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.129 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.129 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-06-06T04:34:41.129 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.129 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.129 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.130 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.130 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.130 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.130 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.130 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-06-06T04:34:41.130 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-06-06T04:34:41.130 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.131 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.131 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.131 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-06-06T04:34:41.131 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.131 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.131 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.131 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.131 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.132 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.132 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.132 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.132 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.132 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.132 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-06-06T04:34:41.132 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-06-06T04:34:41.133 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-06-06T04:34:41.133 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.133 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-06-06T04:34:41.133 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-06-06T04:34:41.133 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.133 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.133 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.133 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-06-06T04:34:41.134 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.134 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.134 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.134 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-06T04:34:41.134 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.134 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.134 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.135 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-06-06T04:34:41.135 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-06-06T04:34:41.135 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.135 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.135 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.135 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-06-06T04:34:41.135 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.136 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.136 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.136 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-06T04:34:41.136 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.136 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.136 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.136 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-06-06T04:34:41.137 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-06-06T04:34:41.137 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.137 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.137 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.137 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.137 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.137 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.137 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.138 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-06-06T04:34:41.138 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.138 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.138 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.138 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-06-06T04:34:41.138 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-06-06T04:34:41.138 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.139 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.139 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.139 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-06-06T04:34:41.139 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-06-06T04:34:41.139 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.139 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.139 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-06T04:34:41.139 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.140 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.140 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.140 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.140 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.140 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.140 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.140 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.141 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.141 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.141 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.141 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.141 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.141 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.141 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.141 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.141 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.142 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.142 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.142 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.142 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.142 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.142 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.143 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.143 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.143 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.143 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.143 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.143 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.143 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.143 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.144 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.144 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.144 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.144 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.144 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.144 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.144 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.144 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.145 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.145 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.145 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.145 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.145 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.145 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.145 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.145 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.146 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.146 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.146 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.146 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.146 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.146 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.146 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.146 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.147 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.147 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.147 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.147 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.147 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.147 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.147 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.148 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.148 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.148 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.148 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.148 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.148 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.148 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.149 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-06-06T04:34:41.149 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-06-06T04:34:41.149 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.149 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.149 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.149 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.149 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.149 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.149 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.150 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-06-06T04:34:41.150 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.150 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.150 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.150 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-06-06T04:34:41.150 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-06-06T04:34:41.150 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.151 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.151 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.151 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-06-06T04:34:41.151 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.151 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.151 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.151 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-06-06T04:34:41.151 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.152 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.152 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.152 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-06-06T04:34:41.152 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-06-06T04:34:41.152 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.152 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.152 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.153 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-06-06T04:34:41.153 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.153 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.153 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.153 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T04:34:41.153 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.153 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.153 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.154 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-06-06T04:34:41.154 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-06-06T04:34:41.154 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.154 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.154 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.154 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-06-06T04:34:41.154 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-06-06T04:34:41.155 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-06-06T04:34:41.155 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.155 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-06T04:34:41.155 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.155 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.155 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.155 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-06-06T04:34:41.155 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-06-06T04:34:41.156 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.156 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.156 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.156 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.156 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.156 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.156 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.156 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-06-06T04:34:41.157 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.157 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.157 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.157 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-06-06T04:34:41.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-06-06T04:34:41.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.158 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.158 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.158 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-06-06T04:34:41.158 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.158 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.158 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.158 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T04:34:41.158 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.159 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.159 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.159 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.159 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-06-06T04:34:41.159 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.160 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.160 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.160 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.160 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.160 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.160 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.160 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.160 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.161 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.161 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.161 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.161 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.161 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.161 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.161 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.161 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.162 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.162 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.162 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.162 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.162 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.162 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.163 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.163 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.163 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.163 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.163 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.163 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.163 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.163 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.164 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.164 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.164 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.164 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.164 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.164 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.164 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.165 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.165 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.165 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.165 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.165 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.165 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.165 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.166 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.166 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.166 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.166 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.166 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.166 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.166 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.166 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.167 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.167 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.167 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.167 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.167 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.167 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.167 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.168 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.168 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.168 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.168 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.168 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.168 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.168 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.168 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.169 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.169 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.169 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-06-06T04:34:41.169 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.169 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.169 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.169 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.170 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.170 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.170 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.170 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.170 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.170 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.170 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.171 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.171 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.171 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.171 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.171 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.171 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.171 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.171 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.172 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.172 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.172 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.172 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.172 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.172 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.172 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.173 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.173 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.173 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.173 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.173 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.173 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.173 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.173 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.173 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.174 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.174 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.174 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.174 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.174 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.174 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.175 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.175 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.175 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.175 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.175 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.175 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.175 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.175 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.176 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.176 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.176 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.176 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.176 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.176 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.176 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.177 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.177 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.177 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.177 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.177 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.177 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.177 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.177 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.177 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.178 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.178 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.178 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.178 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.178 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.178 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.179 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-06-06T04:34:41.179 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.179 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.179 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.179 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-06-06T04:34:41.179 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-06-06T04:34:41.179 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.179 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.180 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.180 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-06-06T04:34:41.180 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.180 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.180 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.180 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-06T04:34:41.180 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.180 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.181 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.181 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-06-06T04:34:41.181 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-06-06T04:34:41.181 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.181 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.181 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.181 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:41.182 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-06-06T04:34:41.182 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.182 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.182 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-06T04:34:41.182 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.182 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.182 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.183 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.183 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.183 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.183 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.183 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.183 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.183 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.184 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.184 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.184 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.184 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.184 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.184 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.184 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.184 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.185 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.185 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.185 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.185 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.185 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.185 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.185 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.186 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.186 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.186 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.186 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.186 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.186 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.186 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.186 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.187 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.187 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.187 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.187 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.187 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.187 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.188 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.188 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.188 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.188 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.188 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.188 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.188 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.188 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.189 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.189 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.189 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.189 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.189 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.189 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.189 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.189 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.190 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.190 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.190 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.190 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.190 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.190 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.190 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.191 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.191 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.191 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.191 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.191 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.191 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.191 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.191 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.192 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.192 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-06-06T04:34:41.192 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.192 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.192 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.192 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-06-06T04:34:41.193 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-06-06T04:34:41.193 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.193 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.193 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.193 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-06-06T04:34:41.193 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.193 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.193 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.194 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-06-06T04:34:41.194 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-06T04:34:41.194 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.194 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.194 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.194 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.195 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.195 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.195 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.195 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.195 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.195 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.195 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.195 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.196 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.196 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.196 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.196 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.196 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.196 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.196 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.196 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.197 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.197 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.197 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.197 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.197 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.197 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.198 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.198 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.198 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.198 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.198 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.198 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.198 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.199 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.199 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.199 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.199 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.199 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.199 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.199 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.199 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.200 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.200 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.200 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.200 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.200 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.200 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.200 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.200 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.201 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.201 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.201 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.201 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.201 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.201 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.201 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.202 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.202 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.202 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.202 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.202 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.202 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.202 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.202 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.203 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.203 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-06-06T04:34:41.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-06-06T04:34:41.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.203 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.203 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.203 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.204 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.204 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.204 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.204 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-06-06T04:34:41.204 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-06T04:34:41.204 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.204 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.205 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-06-06T04:34:41.205 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-06-06T04:34:41.205 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.205 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.205 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.205 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-06-06T04:34:41.205 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.205 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.206 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.206 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-06-06T04:34:41.206 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.206 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.206 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.206 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-06-06T04:34:41.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-06-06T04:34:41.207 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.207 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.207 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.207 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-06-06T04:34:41.207 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.207 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.207 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.207 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-06-06T04:34:41.208 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.208 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.208 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.208 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-06-06T04:34:41.208 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-06-06T04:34:41.208 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.208 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.209 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.209 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.209 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.209 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.209 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.209 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-06-06T04:34:41.209 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.209 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.209 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.210 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-06-06T04:34:41.210 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-06-06T04:34:41.210 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.210 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.210 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.210 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-06-06T04:34:41.210 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.211 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.211 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.211 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-06T04:34:41.211 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-06T04:34:41.211 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.211 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-06-06T04:34:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.213 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.213 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.213 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.213 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.213 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.213 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.213 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.213 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.214 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.214 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.214 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.214 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.214 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.214 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.214 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.214 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.215 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.215 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.215 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.215 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.215 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.215 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.216 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.216 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.216 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.216 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.216 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.216 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.216 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.217 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.217 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.217 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.217 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.217 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.217 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.217 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.218 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.218 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.218 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.218 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.218 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.218 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.218 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.218 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.219 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.219 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.219 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.219 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.219 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.219 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.219 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.220 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.220 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.220 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.220 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.220 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.220 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.220 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.221 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.221 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.221 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.221 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.222 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-06-06T04:34:41.222 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.222 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.222 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.222 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.222 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.222 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.222 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.223 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.223 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.223 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.223 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.223 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.223 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.223 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.223 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.224 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.224 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.224 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.224 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.224 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.224 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.224 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.224 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.225 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.225 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.225 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.225 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.225 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.225 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.225 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.226 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.226 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.226 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.226 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.226 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.226 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.226 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.227 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.227 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.227 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.227 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.227 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.227 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.227 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.227 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.228 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.228 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.228 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.228 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.228 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.228 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.228 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.228 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.229 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.229 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.229 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.229 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.229 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.229 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.229 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.230 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.230 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.230 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.230 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.230 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.230 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.230 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.231 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.231 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.231 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.231 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.231 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-06-06T04:34:41.231 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.232 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.232 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.232 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.232 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.232 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.232 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.232 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.232 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.233 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.233 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.233 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.233 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.233 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.233 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.233 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.233 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.233 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.234 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.234 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.234 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.234 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.234 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.234 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.234 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.235 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.235 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.235 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.235 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.235 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.235 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.235 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.236 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.236 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.236 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.236 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.236 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.236 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.236 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.237 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.237 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.237 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.237 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.237 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.237 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.237 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.237 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.238 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.238 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.238 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.238 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.238 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.238 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.238 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.239 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.239 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.239 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.239 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.239 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.239 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.240 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.240 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.240 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.240 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.240 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.240 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.240 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.240 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.241 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.241 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.241 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-06-06T04:34:41.241 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.241 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.241 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.241 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-06-06T04:34:41.242 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-06-06T04:34:41.242 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.242 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.242 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.242 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.242 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.242 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.243 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.243 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-06T04:34:41.243 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.243 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.243 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.243 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.243 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.244 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.244 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.244 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.244 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.244 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.244 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.244 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.244 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.245 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.245 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.245 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.245 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.245 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.245 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.245 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.245 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.246 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.246 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.246 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.246 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.246 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.246 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.246 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.247 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.247 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.247 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.247 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.247 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.247 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.247 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.247 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.248 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.248 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.248 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.248 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.248 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.248 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.248 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.249 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.249 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.249 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.249 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.249 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.249 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.249 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.249 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-06-06T04:34:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-06-06T04:34:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.253 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.253 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.253 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.253 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.253 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.253 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-06-06T04:34:41.253 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-06-06T04:34:41.253 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-06-06T04:34:41.254 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.254 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-06-06T04:34:41.254 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.254 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.254 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.254 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.254 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.255 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-06-06T04:34:41.255 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.255 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.255 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.255 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.256 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.256 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.256 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.256 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.256 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.256 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.257 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.257 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.257 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.257 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.257 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.257 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.257 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.257 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.258 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.258 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.258 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.258 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.258 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.258 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.259 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.259 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.259 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.259 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.259 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.259 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.259 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.259 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.260 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.260 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.260 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.260 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.260 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.260 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.260 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.261 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.261 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.261 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.261 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.261 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.261 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.261 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.261 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.262 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.262 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.262 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.262 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.262 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.262 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.262 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.263 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.263 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.263 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.263 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.263 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.263 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.263 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.264 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.264 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.264 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.264 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.264 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.264 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.264 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.265 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-06-06T04:34:41.265 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.265 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.265 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.265 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.265 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.266 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.266 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.266 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.266 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.266 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.266 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.266 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.266 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.267 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.267 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.267 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.267 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.267 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.267 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.267 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.267 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.268 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.268 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.268 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.268 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.268 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-06-06T04:34:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.277 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.277 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.277 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.277 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.277 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.277 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.277 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.278 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.278 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.278 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.278 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.278 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.278 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.280 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.280 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.280 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.280 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.280 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.280 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.281 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.281 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.281 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.281 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.281 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.281 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.281 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.281 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-06-06T04:34:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-06-06T04:34:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-06-06T04:34:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-06-06T04:34:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-06-06T04:34:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-06-06T04:34:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-06-06T04:34:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-06-06T04:34:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-06T04:34:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-06-06T04:34:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-06-06T04:34:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-06-06T04:34:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-06T04:34:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-06-06T04:34:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-06-06T04:34:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-06-06T04:34:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-06-06T04:34:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-06-06T04:34:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-06-06T04:34:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-06-06T04:34:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-06-06T04:34:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-06-06T04:34:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.304 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.304 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-06-06T04:34:41.304 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.304 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.304 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.304 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.304 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-06-06T04:34:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-06-06T04:34:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-06-06T04:34:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-06-06T04:34:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-06-06T04:34:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-06T04:34:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-06T04:34:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-06-06T04:34:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-06-06T04:34:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-06T04:34:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-06T04:34:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-06-06T04:34:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-06-06T04:34:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-06-06T04:34:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-06-06T04:34:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-06-06T04:34:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-06-06T04:34:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-06-06T04:34:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-06T04:34:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-06-06T04:34:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-06-06T04:34:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-06-06T04:34:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-06T04:34:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-06-06T04:34:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-06-06T04:34:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-06-06T04:34:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-06-06T04:34:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-06-06T04:34:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-06-06T04:34:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-06-06T04:34:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-06-06T04:34:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-06-06T04:34:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-06-06T04:34:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-06-06T04:34:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-06-06T04:34:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-06-06T04:34:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-06-06T04:34:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.336 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.336 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.336 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.336 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.336 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.336 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.336 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.336 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-06-06T04:34:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-06-06T04:34:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-06-06T04:34:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-06-06T04:34:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-06-06T04:34:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-06-06T04:34:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.346 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.346 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.346 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.346 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.346 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.346 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.346 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.346 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-06-06T04:34:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-06-06T04:34:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-06-06T04:34:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-06-06T04:34:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-06-06T04:34:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.357 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.357 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.357 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.357 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.357 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.357 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.357 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.357 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.358 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.358 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.358 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.358 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.358 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.358 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.358 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.359 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.359 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.359 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.359 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.359 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.359 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.360 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.360 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.360 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.360 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.360 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-06-06T04:34:41.360 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-06-06T04:34:41.361 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.361 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.361 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.361 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.361 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.361 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.361 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.361 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.362 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.362 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.362 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.362 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-06-06T04:34:41.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-06-06T04:34:41.362 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.362 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.363 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.363 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.363 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.363 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.363 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.363 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.363 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.363 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.364 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.364 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.364 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.364 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-06-06T04:34:41.364 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.364 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.365 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.365 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.365 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.365 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.365 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.365 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.365 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.365 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.366 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.366 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.366 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.366 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.366 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.366 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.366 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.366 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.367 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.367 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.367 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.367 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.367 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.368 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.368 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.368 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.368 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.368 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.368 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.368 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.368 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.369 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.369 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.369 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.369 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.369 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.369 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.369 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.370 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.370 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.370 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.370 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.370 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.370 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.370 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.370 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.371 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.371 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.371 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.371 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.371 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.371 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.371 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.372 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.372 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.372 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.372 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.372 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.372 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.372 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.372 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.373 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.373 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.373 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.373 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.373 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.373 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.373 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-06-06T04:34:41.374 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.374 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.374 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.374 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.374 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.375 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.375 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.375 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.375 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.375 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.375 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.375 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.375 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.375 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.376 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.376 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.376 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.376 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.376 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.376 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.376 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.377 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.377 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.377 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.377 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.377 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.378 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.378 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.378 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.378 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.378 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.378 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.378 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.379 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.379 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.379 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.379 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.379 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.379 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.379 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.379 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.380 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.380 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.380 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.380 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.380 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.380 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.380 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.380 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.381 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.381 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.381 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.381 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.381 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.381 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.382 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.382 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.382 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.382 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.382 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.382 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.382 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.382 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.383 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.383 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-06-06T04:34:41.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-06-06T04:34:41.383 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.383 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.383 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.383 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-06-06T04:34:41.384 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.384 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.384 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.384 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-06T04:34:41.384 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.384 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.384 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.384 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.385 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.385 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.385 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-06-06T04:34:41.385 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.385 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.385 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.385 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-06-06T04:34:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-06-06T04:34:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-06-06T04:34:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-06-06T04:34:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-06-06T04:34:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-06-06T04:34:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-06-06T04:34:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-06-06T04:34:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-06-06T04:34:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-06-06T04:34:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-06-06T04:34:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:41.402 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.402 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.402 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.402 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.402 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.402 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.402 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.402 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-06-06T04:34:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-06-06T04:34:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-06-06T04:34:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-06-06T04:34:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-06-06T04:34:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-06-06T04:34:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-06-06T04:34:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-06-06T04:34:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-06-06T04:34:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-06-06T04:34:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-06-06T04:34:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-06-06T04:34:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-06-06T04:34:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-06-06T04:34:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-06-06T04:34:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-06-06T04:34:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-06-06T04:34:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-06-06T04:34:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-06-06T04:34:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-06-06T04:34:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-06-06T04:34:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-06-06T04:34:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-06-06T04:34:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-06-06T04:34:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-06-06T04:34:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-06-06T04:34:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-06-06T04:34:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-06-06T04:34:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-06T04:34:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-06-06T04:34:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-06-06T04:34:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-06-06T04:34:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-06-06T04:34:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-06-06T04:34:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-06-06T04:34:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-06T04:34:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-06-06T04:34:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-06-06T04:34:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-06T04:34:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-06-06T04:34:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-06-06T04:34:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-06T04:34:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-06-06T04:34:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-06-06T04:34:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-06T04:34:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-06-06T04:34:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-06-06T04:34:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-06T04:34:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-06-06T04:34:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-06-06T04:34:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-06-06T04:34:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-06-06T04:34:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-06-06T04:34:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-06-06T04:34:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-06T04:34:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-06-06T04:34:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-06-06T04:34:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-06-06T04:34:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-06-06T04:34:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-06-06T04:34:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-06-06T04:34:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-06-06T04:34:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-06-06T04:34:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-06-06T04:34:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-06-06T04:34:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-06-06T04:34:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-06-06T04:34:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-06-06T04:34:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-06-06T04:34:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-06-06T04:34:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-06-06T04:34:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-06-06T04:34:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-06-06T04:34:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-06-06T04:34:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-06T04:34:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-06-06T04:34:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-06-06T04:34:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-06-06T04:34:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-06-06T04:34:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-06-06T04:34:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-06T04:34:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-06-06T04:34:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-06-06T04:34:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-06-06T04:34:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-06-06T04:34:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-06-06T04:34:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-06-06T04:34:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-06T04:34:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-06-06T04:34:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-06-06T04:34:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-06T04:34:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-06-06T04:34:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-06-06T04:34:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-06-06T04:34:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-06-06T04:34:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-06-06T04:34:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-06-06T04:34:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-06-06T04:34:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-06-06T04:34:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-06-06T04:34:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-06-06T04:34:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-06-06T04:34:41.539 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-06-06T04:34:41.539 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.539 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.539 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.539 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.539 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.539 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.539 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.539 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.540 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.540 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.540 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.540 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-06-06T04:34:41.540 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-06-06T04:34:41.540 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.540 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.541 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.541 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-06-06T04:34:41.541 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.541 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.541 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.541 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.541 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.541 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.542 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.542 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-06-06T04:34:41.542 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-06-06T04:34:41.542 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.542 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.542 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.542 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-06T04:34:41.543 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.543 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.543 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.543 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.543 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.543 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.543 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.543 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.544 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.544 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.544 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-06-06T04:34:41.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.544 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 24119, 2024-06-06T04:34:41.544 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "y", 2024-06-06T04:34:41.544 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540701547738038271, 2024-06-06T04:34:41.545 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-06-06T04:34:41.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.545 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-06-06T04:34:41.545 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.545 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.545 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.545 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-06-06T04:34:41.545 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-06-06T04:34:41.546 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.546 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.546 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.546 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-06-06T04:34:41.546 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.546 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.546 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.546 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-06-06T04:34:41.547 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.547 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.547 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.547 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.547 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.547 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.547 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.547 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.548 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.548 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.548 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.548 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.548 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.548 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.548 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.548 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.549 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.549 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.549 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.549 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.549 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.549 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.549 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.550 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.550 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.550 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.550 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.550 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.550 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.550 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.551 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.551 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.551 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.551 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.551 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.551 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.551 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.552 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.552 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.552 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.552 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.552 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.552 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.552 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.552 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.553 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.553 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.553 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.553 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.553 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.553 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.553 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.553 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.554 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.554 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.554 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.554 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.554 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.554 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.554 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.555 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.555 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.555 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.555 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.555 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.555 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.555 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.555 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.556 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-06-06T04:34:41.556 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-06-06T04:34:41.556 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.556 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.556 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.556 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.556 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.556 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.557 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.557 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-06-06T04:34:41.557 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.557 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.557 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.557 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-06-06T04:34:41.557 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-06-06T04:34:41.558 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.558 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.558 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.558 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-06-06T04:34:41.558 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.558 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.558 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.558 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-06-06T04:34:41.559 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.559 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.559 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.559 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-06-06T04:34:41.559 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-06-06T04:34:41.559 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.559 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.560 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.560 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.560 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.560 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.560 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.560 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-06-06T04:34:41.560 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.560 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.561 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.561 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-06-06T04:34:41.561 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-06-06T04:34:41.561 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.561 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.561 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.561 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.561 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.562 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.562 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.562 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-06-06T04:34:41.562 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.562 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.562 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-06-06T04:34:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-06-06T04:34:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-06-06T04:34:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-06-06T04:34:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-06-06T04:34:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-06-06T04:34:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.565 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.565 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.565 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.565 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.565 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.565 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.565 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-06-06T04:34:41.565 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.566 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.566 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.566 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-06-06T04:34:41.566 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-06-06T04:34:41.566 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.566 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.567 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.567 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.567 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.567 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.567 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.567 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-06T04:34:41.567 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.567 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.568 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.568 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-06-06T04:34:41.568 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-06-06T04:34:41.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.568 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.568 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.568 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.569 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.569 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.569 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.569 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-06-06T04:34:41.569 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.569 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.569 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.569 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.570 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.570 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-06-06T04:34:41.570 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.570 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.570 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.570 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-06-06T04:34:41.571 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-06-06T04:34:41.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.571 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.571 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.571 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.571 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.571 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.572 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-06-06T04:34:41.572 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.572 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.572 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-06-06T04:34:41.572 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-06-06T04:34:41.572 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.573 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.573 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-06-06T04:34:41.573 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.573 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.573 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-06-06T04:34:41.573 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T04:34:41.573 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.574 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.574 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-06-06T04:34:41.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-06-06T04:34:41.574 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-06-06T04:34:41.574 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.574 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.574 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-06-06T04:34:41.575 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-06-06T04:34:41.575 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-06-06T04:34:41.575 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.575 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-06T04:34:41.575 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T04:34:41.575 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.575 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.576 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-06-06T04:34:41.576 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-06-06T04:34:41.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-06-06T04:34:41.576 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.576 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.576 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-06-06T04:34:41.576 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-06-06T04:34:41.576 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-06-06T04:34:41.577 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.577 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-06T04:34:41.577 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.577 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.577 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.577 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-06-06T04:34:41.577 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-06-06T04:34:41.578 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.578 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.578 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.578 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-06-06T04:34:41.578 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.578 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.578 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.578 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-06T04:34:41.579 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-06T04:34:41.579 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.579 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.579 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-06-06T04:34:41.579 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-06-06T04:34:41.579 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:41.579 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.580 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.580 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-06-06T04:34:41.580 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-06-06T04:34:41.580 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-06-06T04:34:41.580 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.580 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-06-06T04:34:41.580 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-06T04:34:41.580 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.581 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.581 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-06-06T04:34:41.581 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-06-06T04:34:41.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.581 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.581 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.581 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-06-06T04:34:41.582 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.582 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.582 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.582 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-06-06T04:34:41.582 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T04:34:41.582 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.582 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.583 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-06-06T04:34:41.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-06-06T04:34:41.583 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-06-06T04:34:41.583 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.583 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.583 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-06-06T04:34:41.583 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-06-06T04:34:41.583 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-06-06T04:34:41.584 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.584 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-06T04:34:41.584 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T04:34:41.584 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.584 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.584 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.584 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.585 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.585 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.585 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.585 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.585 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.585 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.585 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.585 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.586 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.586 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.586 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.586 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.586 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.586 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.586 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.586 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.587 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.587 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.587 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.587 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.587 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.587 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.587 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.588 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.588 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.588 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.588 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.588 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.588 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.588 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.589 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.589 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.589 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.589 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.589 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.589 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.589 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.589 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.590 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.590 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.590 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.590 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.590 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.590 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.590 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.590 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.591 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.591 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.591 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.591 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.591 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.591 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.591 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.592 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.592 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.592 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.592 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.592 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.592 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.592 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.592 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.592 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.593 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.593 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-06-06T04:34:41.593 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-06-06T04:34:41.593 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:41.593 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.593 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.593 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-06-06T04:34:41.594 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.594 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.594 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.594 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-06T04:34:41.594 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.594 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.594 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.595 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-06-06T04:34:41.595 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-06-06T04:34:41.595 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.595 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.595 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.595 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-06-06T04:34:41.595 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.595 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.596 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.596 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-06-06T04:34:41.596 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-06-06T04:34:41.596 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-06-06T04:34:41.596 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-06-06T04:34:41.596 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-06-06T04:34:41.596 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.596 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-06-06T04:34:41.597 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.597 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.597 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.597 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-06-06T04:34:41.597 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-06-06T04:34:41.597 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.597 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.598 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.598 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.598 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.598 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.598 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.598 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-06T04:34:41.598 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.598 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.598 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.599 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-06-06T04:34:41.599 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-06-06T04:34:41.599 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.599 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.599 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.599 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-06-06T04:34:41.599 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.600 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.600 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.600 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-06T04:34:41.600 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.600 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.600 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.600 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-06-06T04:34:41.601 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-06-06T04:34:41.601 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.601 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.601 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.601 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-06-06T04:34:41.601 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-06-06T04:34:41.601 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.601 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.601 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-06T04:34:41.602 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-06T04:34:41.602 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.602 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.602 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-06-06T04:34:41.602 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-06-06T04:34:41.602 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-06-06T04:34:41.602 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-06T04:34:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.603 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-06-06T04:34:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.604 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.605 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.605 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-06-06T04:34:41.605 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-06-06T04:34:41.605 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:41.605 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.605 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.605 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-06-06T04:34:41.605 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.606 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.606 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.606 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-06T04:34:41.606 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.606 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.606 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.606 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-06-06T04:34:41.607 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-06-06T04:34:41.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.607 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.607 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.607 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-06-06T04:34:41.607 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.607 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.607 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.608 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-06T04:34:41.608 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.608 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.608 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.608 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-06-06T04:34:41.608 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-06-06T04:34:41.608 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.609 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.609 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.609 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-06-06T04:34:41.609 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.609 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.609 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.609 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-06T04:34:41.609 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.610 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.610 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.610 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-06-06T04:34:41.610 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-06-06T04:34:41.610 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.610 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.610 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.611 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.611 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.611 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.611 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.611 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-06T04:34:41.611 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-06T04:34:41.611 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.611 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.612 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-06-06T04:34:41.612 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-06-06T04:34:41.612 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.612 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.612 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.612 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-06-06T04:34:41.612 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.613 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.613 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.613 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-06T04:34:41.613 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.613 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.613 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.613 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-06-06T04:34:41.614 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-06-06T04:34:41.614 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:41.614 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.614 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.614 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-06-06T04:34:41.614 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.614 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.614 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.615 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-06T04:34:41.615 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.615 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.615 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.615 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-06-06T04:34:41.615 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-06-06T04:34:41.615 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.616 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.616 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.616 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.616 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.616 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.616 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.616 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-06-06T04:34:41.616 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-06-06T04:34:41.617 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-06-06T04:34:41.617 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.617 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-06T04:34:41.617 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.617 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.617 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.617 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-06-06T04:34:41.618 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-06-06T04:34:41.618 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.618 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.618 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.618 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.618 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.618 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.618 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.619 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-06T04:34:41.619 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.619 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.619 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.619 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-06-06T04:34:41.619 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-06-06T04:34:41.619 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.620 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.620 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.620 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.620 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.620 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.620 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.620 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-06T04:34:41.620 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.620 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.621 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.621 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-06-06T04:34:41.621 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-06-06T04:34:41.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.621 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.621 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.622 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.622 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.622 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.622 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-06T04:34:41.622 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.622 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.622 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.623 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-06-06T04:34:41.623 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-06-06T04:34:41.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.623 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.623 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.623 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-06T04:34:41.623 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.623 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.624 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.624 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-06-06T04:34:41.624 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.624 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.624 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.624 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-06-06T04:34:41.624 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-06-06T04:34:41.625 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.625 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.625 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.625 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-06T04:34:41.625 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.625 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.625 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.625 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-06-06T04:34:41.625 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.626 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.626 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.626 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-06-06T04:34:41.626 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-06-06T04:34:41.626 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.626 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.626 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.627 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-06T04:34:41.627 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.627 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.627 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.627 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-06-06T04:34:41.627 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.627 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.627 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.628 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-06-06T04:34:41.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-06-06T04:34:41.628 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.628 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.628 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.628 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-06T04:34:41.628 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.629 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.629 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.629 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-06-06T04:34:41.629 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.629 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.629 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.629 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-06-06T04:34:41.630 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-06-06T04:34:41.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.630 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.630 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.630 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-06T04:34:41.630 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.630 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.630 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.631 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-06-06T04:34:41.631 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.631 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.631 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.631 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-06-06T04:34:41.631 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-06-06T04:34:41.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.632 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.632 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.632 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-06T04:34:41.632 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.632 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.632 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.632 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-06-06T04:34:41.632 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.633 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.633 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.633 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-06-06T04:34:41.633 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-06-06T04:34:41.633 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.633 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.633 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.633 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-06T04:34:41.634 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.634 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.634 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.634 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-06-06T04:34:41.634 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.634 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.634 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.635 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-06-06T04:34:41.635 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-06-06T04:34:41.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.635 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.635 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.635 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-06T04:34:41.635 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.635 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.636 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.636 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-06-06T04:34:41.636 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.636 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.636 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.636 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-06-06T04:34:41.637 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-06-06T04:34:41.637 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.637 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.637 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.637 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-06T04:34:41.637 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.637 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.637 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.637 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-06-06T04:34:41.638 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.638 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.638 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.638 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-06-06T04:34:41.638 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-06-06T04:34:41.638 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.638 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.639 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.639 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-06T04:34:41.639 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.639 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.639 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.639 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-06-06T04:34:41.639 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.639 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.640 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.640 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-06-06T04:34:41.640 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-06-06T04:34:41.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.640 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.640 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.640 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-06T04:34:41.641 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.641 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.641 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.641 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-06-06T04:34:41.641 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.641 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.641 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.642 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-06-06T04:34:41.642 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-06-06T04:34:41.642 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.642 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.642 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.642 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-06T04:34:41.642 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.642 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.643 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.643 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-06-06T04:34:41.643 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.643 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.643 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.643 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-06-06T04:34:41.643 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-06-06T04:34:41.644 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.644 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.644 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.644 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-06T04:34:41.644 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.644 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.644 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.644 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-06-06T04:34:41.645 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.645 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.645 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.645 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-06-06T04:34:41.645 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-06-06T04:34:41.645 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.645 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.646 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.646 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-06T04:34:41.646 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.646 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.646 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.646 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-06-06T04:34:41.646 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.647 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.647 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.647 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-06-06T04:34:41.647 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-06-06T04:34:41.647 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.647 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.647 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.648 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-06T04:34:41.648 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.648 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.648 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.648 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-06-06T04:34:41.648 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.648 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.648 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.649 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-06-06T04:34:41.649 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-06-06T04:34:41.649 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.649 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.649 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.649 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-06T04:34:41.649 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.650 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.650 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.650 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-06-06T04:34:41.650 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.650 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.650 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.650 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-06-06T04:34:41.651 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-06-06T04:34:41.651 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.651 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.651 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.651 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.651 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.651 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.651 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.652 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-06-06T04:34:41.652 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.652 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.652 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.652 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-06-06T04:34:41.652 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-06-06T04:34:41.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.653 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.653 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.653 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-06-06T04:34:41.653 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.653 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.653 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.653 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-06T04:34:41.653 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.654 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.654 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.654 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-06-06T04:34:41.654 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-06-06T04:34:41.654 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.654 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.654 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.655 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-06-06T04:34:41.655 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.655 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.655 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.655 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-06T04:34:41.655 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.655 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.655 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.656 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-06-06T04:34:41.656 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-06-06T04:34:41.656 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.656 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.656 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.656 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-06-06T04:34:41.656 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.657 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.657 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.657 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-06T04:34:41.657 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.657 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.657 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.657 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-06-06T04:34:41.658 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-06-06T04:34:41.658 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.658 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.658 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.658 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-06-06T04:34:41.658 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.658 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.658 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.659 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-06-06T04:34:41.659 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.659 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.659 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.659 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-06-06T04:34:41.659 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-06-06T04:34:41.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.660 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.660 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.660 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.660 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.660 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.660 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.660 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-06T04:34:41.661 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.661 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.661 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.661 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-06-06T04:34:41.661 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-06-06T04:34:41.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.661 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.662 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-06-06T04:34:41.662 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.662 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.662 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.662 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-06-06T04:34:41.662 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.662 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.663 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.663 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-06-06T04:34:41.663 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-06-06T04:34:41.663 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.663 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.663 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.664 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-06T04:34:41.664 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.664 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.664 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.664 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-06T04:34:41.664 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.664 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.664 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.665 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-06-06T04:34:41.665 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-06-06T04:34:41.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.665 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.665 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.665 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-06-06T04:34:41.665 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.666 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.666 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.666 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-06-06T04:34:41.666 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.666 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.666 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.666 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-06-06T04:34:41.667 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-06-06T04:34:41.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.667 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.667 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.667 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.667 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.667 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.667 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.668 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-06T04:34:41.668 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.668 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.668 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.668 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-06-06T04:34:41.668 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-06-06T04:34:41.668 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.669 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.669 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.669 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.669 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.669 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.669 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.669 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-06T04:34:41.669 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.670 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.670 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.670 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.670 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.670 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.671 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.671 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.671 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.671 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.671 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.671 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.671 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.671 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.671 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.672 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.672 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.672 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.672 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.672 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.672 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.672 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.673 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-06-06T04:34:41.673 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-06-06T04:34:41.673 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.673 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.673 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.673 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.673 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.673 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.674 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.674 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-06T04:34:41.674 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.674 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.674 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.674 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.675 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.675 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.675 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.675 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.675 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.675 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.676 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-06T04:34:41.676 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.676 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.676 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.676 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.676 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.676 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.677 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.677 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.677 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.677 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.677 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.677 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.677 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.677 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.678 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.678 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.678 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.678 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.678 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.678 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.678 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.678 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.679 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.679 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.679 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.679 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.679 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.679 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.680 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.680 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.680 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.680 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.680 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.680 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.680 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.681 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-06T04:34:41.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-06T04:34:41.681 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.681 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.681 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.681 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.681 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.681 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-06T04:34:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-06T04:34:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-06T04:34:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-06-06T04:34:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-06T04:34:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-06-06T04:34:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-06-06T04:34:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-06-06T04:34:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-06-06T04:34:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-06-06T04:34:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-06T04:34:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-06-06T04:34:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-06-06T04:34:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-06-06T04:34:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-06-06T04:34:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-06-06T04:34:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-06-06T04:34:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-06-06T04:34:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-06-06T04:34:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-06-06T04:34:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.692 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-06-06T04:34:41.692 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-06-06T04:34:41.692 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.692 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.692 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.692 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-06-06T04:34:41.692 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.692 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.693 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.693 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-06T04:34:41.693 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.693 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.693 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.693 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-06-06T04:34:41.693 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-06-06T04:34:41.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.694 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.694 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.694 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-06T04:34:41.694 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.694 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.694 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.694 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-06T04:34:41.694 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.695 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.695 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.695 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-06-06T04:34:41.695 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-06-06T04:34:41.695 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.695 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.695 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.696 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.696 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.696 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.696 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.696 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-06T04:34:41.696 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.696 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.696 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.697 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-06-06T04:34:41.697 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-06-06T04:34:41.697 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.697 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.697 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.697 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.697 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.698 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.698 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.698 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-06T04:34:41.698 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.698 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.698 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.698 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-06-06T04:34:41.699 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-06-06T04:34:41.699 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.699 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.699 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.699 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.699 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.699 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.699 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.700 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-06T04:34:41.700 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.700 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.700 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.700 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-06-06T04:34:41.700 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-06-06T04:34:41.700 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.701 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.701 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.701 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.701 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.701 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.701 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.701 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-06T04:34:41.701 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.702 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.702 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.702 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-06-06T04:34:41.702 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-06-06T04:34:41.702 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.702 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.702 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.703 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.703 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.703 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.703 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.703 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-06T04:34:41.703 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.703 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.703 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.704 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-06-06T04:34:41.704 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-06-06T04:34:41.704 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.704 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.704 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.704 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-06-06T04:34:41.704 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.705 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.705 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.705 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-06-06T04:34:41.705 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.705 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.705 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.705 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-06-06T04:34:41.706 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-06-06T04:34:41.706 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.706 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.706 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.706 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.706 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.706 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.706 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.707 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-06T04:34:41.707 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.707 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.707 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.707 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-06-06T04:34:41.707 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-06-06T04:34:41.707 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.708 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.708 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.708 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.708 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.708 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.708 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.708 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-06T04:34:41.708 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.709 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.709 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.709 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-06-06T04:34:41.709 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-06-06T04:34:41.709 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.709 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.709 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.710 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.710 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.710 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.710 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.710 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-06T04:34:41.710 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.710 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.710 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.711 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-06-06T04:34:41.711 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-06-06T04:34:41.711 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.711 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.711 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.711 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.711 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.712 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.712 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.712 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-06-06T04:34:41.712 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.712 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.712 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.712 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-06-06T04:34:41.713 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-06-06T04:34:41.713 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.713 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.713 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.713 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.713 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.713 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.713 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.714 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-06T04:34:41.714 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.714 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.714 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.714 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-06-06T04:34:41.714 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-06-06T04:34:41.714 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.715 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.715 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.715 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.715 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.715 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.715 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.715 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-06T04:34:41.715 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.716 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.716 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.716 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.716 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.716 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.716 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-06-06T04:34:41.716 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.717 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.717 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.717 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.717 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.717 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.717 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.717 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.717 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.718 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.718 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.718 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.718 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.718 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.718 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.718 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.718 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.719 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.719 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.719 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.719 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.719 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.719 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.719 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.720 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.720 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.720 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.720 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.720 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.720 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.720 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.720 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.721 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.721 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.721 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.721 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.721 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.721 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.721 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.722 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.722 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.722 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.722 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.722 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.722 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.722 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.722 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.723 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.723 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.723 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.723 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.723 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.723 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.723 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.723 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.724 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.724 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.724 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.724 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.724 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.724 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.724 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.725 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.725 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.725 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.725 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.725 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.725 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.725 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.726 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-06-06T04:34:41.726 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-06-06T04:34:41.726 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.726 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.726 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.726 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-06-06T04:34:41.726 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.726 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.727 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.727 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-06T04:34:41.727 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.727 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.727 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.730 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-06-06T04:34:41.730 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-06-06T04:34:41.730 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.730 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.730 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.730 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-06-06T04:34:41.731 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.731 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.731 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.731 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-06T04:34:41.731 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.731 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.731 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.731 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.732 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.732 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-06-06T04:34:41.732 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.732 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.732 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.732 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-06T04:34:41.732 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-06T04:34:41.733 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.733 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.733 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.733 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:41.733 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.733 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.733 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.734 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.734 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.734 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.734 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.734 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-06-06T04:34:41.734 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-06T04:34:41.734 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.735 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.735 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.735 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.735 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.735 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.735 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.735 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.735 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.735 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.736 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.736 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-06T04:34:41.736 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-06T04:34:41.736 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.736 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.736 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.736 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.737 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.737 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.737 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.737 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.737 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.737 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.737 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.738 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-06-06T04:34:41.738 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-06-06T04:34:41.738 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.738 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.738 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.738 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.738 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.739 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.739 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.739 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.739 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.739 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.739 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.739 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-06T04:34:41.740 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-06T04:34:41.740 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.740 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.740 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.740 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.740 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.740 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.740 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.740 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.741 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.741 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.741 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.741 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-06-06T04:34:41.741 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-06T04:34:41.741 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.741 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.742 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.742 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.742 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.742 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.742 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.742 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.742 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.742 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.743 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.743 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-06T04:34:41.743 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-06T04:34:41.743 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.743 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.743 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.744 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.744 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.744 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.744 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.744 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.744 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.744 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.745 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.745 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-06T04:34:41.745 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-06T04:34:41.745 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.745 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.745 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.745 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.746 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.746 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.746 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.746 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.746 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.746 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.746 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.747 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-06T04:34:41.747 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-06T04:34:41.747 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.747 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.747 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.747 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.747 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.747 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.748 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.748 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.748 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.748 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.748 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.748 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-06T04:34:41.748 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-06T04:34:41.749 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.749 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.749 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.749 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.749 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.749 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.749 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.749 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.750 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.750 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.750 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.750 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-06T04:34:41.750 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-06T04:34:41.750 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.750 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.751 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.751 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.751 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.751 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.751 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.751 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.751 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.751 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.752 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.752 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-06T04:34:41.752 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-06T04:34:41.752 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.752 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.752 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.752 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.753 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.753 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.753 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.753 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.753 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.753 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.753 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.754 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-06T04:34:41.754 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-06T04:34:41.754 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.754 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.754 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.754 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.754 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.754 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.755 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.755 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.755 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.755 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.755 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.755 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-06T04:34:41.756 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-06T04:34:41.756 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.756 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.756 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.756 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.756 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.756 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.756 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.757 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.757 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.757 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.757 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.757 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-06-06T04:34:41.757 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-06T04:34:41.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.757 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.758 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.758 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-06-06T04:34:41.758 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.758 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.758 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.758 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.758 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.759 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.759 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.759 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-06T04:34:41.759 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-06T04:34:41.759 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.759 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.759 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.760 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.760 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.760 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.760 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.760 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.760 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.760 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.760 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.761 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-06-06T04:34:41.761 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-06-06T04:34:41.761 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.761 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.761 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.761 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.761 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.762 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.762 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.762 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.762 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.762 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.762 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.762 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-06-06T04:34:41.763 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-06T04:34:41.763 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.763 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.763 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.763 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-06-06T04:34:41.763 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.763 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.763 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.764 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.764 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.764 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.764 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.764 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-06T04:34:41.764 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-06T04:34:41.764 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.765 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.765 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.765 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.765 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.765 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.765 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.765 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.765 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.766 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.766 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.766 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-06T04:34:41.766 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-06T04:34:41.766 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.766 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.766 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.767 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.767 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.767 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.767 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.767 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.767 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.767 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.767 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.768 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-06T04:34:41.768 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-06T04:34:41.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.768 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.768 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.768 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.768 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.769 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.769 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.769 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.769 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.769 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.769 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.769 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-06T04:34:41.770 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-06T04:34:41.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.770 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.770 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.770 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.770 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.770 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.770 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.771 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.771 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.771 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.771 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.771 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-06-06T04:34:41.771 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-06T04:34:41.771 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.772 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.772 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.772 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.772 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.772 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.772 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.772 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.772 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.773 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.773 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.773 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-06T04:34:41.773 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-06T04:34:41.773 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.773 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.773 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.774 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.774 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.774 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.774 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.774 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.774 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.774 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.774 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.775 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-06T04:34:41.775 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-06T04:34:41.775 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.775 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.775 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.775 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.775 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.776 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.776 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.776 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.776 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.776 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.776 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.776 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-06T04:34:41.777 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-06T04:34:41.777 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.777 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.777 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.777 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.777 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.777 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.777 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.778 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.778 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.778 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.778 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.778 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-06T04:34:41.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-06T04:34:41.779 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.779 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.779 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.779 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.779 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.779 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.779 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.779 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.780 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.780 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.780 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.780 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-06T04:34:41.780 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-06T04:34:41.780 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.780 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.781 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.781 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.781 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.781 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.781 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.781 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.781 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.781 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.782 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.782 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-06T04:34:41.782 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-06T04:34:41.782 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.782 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.782 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.782 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.783 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.783 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.783 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.783 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.783 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.783 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.784 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-06T04:34:41.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-06T04:34:41.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.784 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.784 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.784 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.784 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.784 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.785 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.785 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.785 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.785 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.785 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.785 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-06T04:34:41.785 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-06T04:34:41.786 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.786 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.786 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.786 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.786 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.786 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.786 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.786 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.786 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.787 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.787 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.787 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-06-06T04:34:41.787 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-06T04:34:41.787 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.787 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.787 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.788 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.788 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.788 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.788 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.788 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.788 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.788 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.788 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.789 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-06T04:34:41.789 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-06T04:34:41.789 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.789 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.789 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.789 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-06T04:34:41.789 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.790 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.790 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.790 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.790 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.790 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.790 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.790 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-06T04:34:41.791 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-06T04:34:41.791 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.791 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.791 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.791 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:41.791 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.791 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.791 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.792 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.792 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.792 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.792 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.792 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-06T04:34:41.792 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-06T04:34:41.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.793 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.793 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.793 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-06-06T04:34:41.793 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.793 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.793 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.793 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.793 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.794 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.794 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.794 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-06T04:34:41.794 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-06T04:34:41.794 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.794 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.794 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.795 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-06-06T04:34:41.795 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.795 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.795 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.795 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.795 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.795 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.795 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.796 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-06-06T04:34:41.796 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-06T04:34:41.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.796 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.796 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.796 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.796 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.796 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.797 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.797 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.797 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.797 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.797 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.797 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-06T04:34:41.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-06T04:34:41.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.798 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.798 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.798 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-06-06T04:34:41.798 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.798 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.798 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.798 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.799 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.799 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.799 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.799 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-06-06T04:34:41.799 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-06T04:34:41.799 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.799 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.800 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.800 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.800 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.800 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.800 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.800 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.800 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.800 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.801 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.801 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-06-06T04:34:41.801 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-06T04:34:41.801 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.801 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.801 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.801 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.801 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.802 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.802 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.802 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.802 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.802 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.802 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.802 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-06T04:34:41.803 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-06T04:34:41.803 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.803 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.803 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.803 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-06-06T04:34:41.803 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.803 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.803 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.804 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.804 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.804 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.804 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.804 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-06T04:34:41.804 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-06T04:34:41.804 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.805 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.805 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.805 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:41.805 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.805 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.805 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.805 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.805 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.806 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.806 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.806 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-06T04:34:41.806 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-06T04:34:41.806 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.806 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.806 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.807 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-06-06T04:34:41.807 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.807 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.807 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.807 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.807 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.807 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.807 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.808 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-06-06T04:34:41.808 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-06-06T04:34:41.808 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.808 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.808 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.808 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.809 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.809 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.809 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.809 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.809 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.809 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.809 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-06-06T04:34:41.810 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-06-06T04:34:41.810 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.810 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.810 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.810 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.810 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.810 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.810 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.811 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.811 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.811 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.811 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.811 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-06-06T04:34:41.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-06-06T04:34:41.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.812 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.812 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.812 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.812 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.812 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.812 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.812 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-06-06T04:34:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-06-06T04:34:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-06-06T04:34:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-06-06T04:34:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.815 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.815 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-06-06T04:34:41.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-06-06T04:34:41.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.815 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.815 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.816 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.816 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.816 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.816 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.816 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.816 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.816 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.817 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.817 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.817 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.817 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.817 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.817 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.817 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.817 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.818 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.818 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.818 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.818 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.818 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.818 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.818 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.818 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.819 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.819 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.819 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.819 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.819 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.819 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.820 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.820 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.820 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.820 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.820 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.820 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.820 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.820 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.821 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.821 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.821 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.821 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.821 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.821 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.821 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.822 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.822 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.822 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.822 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.822 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.822 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.822 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.822 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.823 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.823 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.823 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.823 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.823 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.823 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.823 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.824 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.824 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.824 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.824 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.824 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.824 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.824 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.824 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.824 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.825 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.825 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.825 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.825 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-06-06T04:34:41.825 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-06-06T04:34:41.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.825 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.826 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.826 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.826 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.826 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.826 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.826 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-06-06T04:34:41.826 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.827 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.827 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-06-06T04:34:41.827 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-06-06T04:34:41.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.827 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.827 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.827 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.828 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.828 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.828 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.828 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.828 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.828 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.828 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.829 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-06-06T04:34:41.829 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-06-06T04:34:41.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.829 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.829 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.829 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-06-06T04:34:41.829 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.829 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.830 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.830 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.830 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.830 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.830 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.830 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-06-06T04:34:41.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-06-06T04:34:41.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.831 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.831 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-06-06T04:34:41.831 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.831 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.831 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.831 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.832 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.832 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.832 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.832 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-06-06T04:34:41.832 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-06-06T04:34:41.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.832 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.833 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.833 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.833 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.833 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.833 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.833 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.833 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.833 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.833 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.834 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-06-06T04:34:41.834 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-06-06T04:34:41.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.834 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.834 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.834 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-06-06T04:34:41.834 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.835 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.835 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.835 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.835 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.835 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.835 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.835 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-06-06T04:34:41.836 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-06-06T04:34:41.836 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.836 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.836 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.836 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-06-06T04:34:41.836 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.836 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.836 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.837 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.837 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-06-06T04:34:41.837 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.837 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.837 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.837 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.837 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.838 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-06-06T04:34:41.838 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-06-06T04:34:41.838 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.838 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.838 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.838 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-06-06T04:34:41.838 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-06-06T04:34:41.838 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-06-06T04:34:41.839 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.839 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.839 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.839 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.839 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.839 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-06-06T04:34:41.839 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-06-06T04:34:41.840 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.840 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.840 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.840 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.840 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.840 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.840 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.841 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.841 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.841 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.841 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-06-06T04:34:41.842 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.842 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.842 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.842 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-06-06T04:34:41.842 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-06-06T04:34:41.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.842 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.842 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.843 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.843 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.843 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.843 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.843 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-06-06T04:34:41.843 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.843 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.843 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.844 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.844 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.844 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.844 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.844 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.844 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.844 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.844 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.845 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.845 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.845 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.845 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.845 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.845 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.845 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.845 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.846 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.846 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.846 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.846 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.846 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.846 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.847 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.847 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.847 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.847 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.847 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.847 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.847 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.847 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.848 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.848 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.848 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.848 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.848 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.848 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.848 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.848 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.849 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.849 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.849 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.849 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.849 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.849 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.849 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.849 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.850 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.850 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.850 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.850 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.850 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.850 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.851 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.851 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.851 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.851 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.851 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.851 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.851 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.851 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.852 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.852 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.852 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.852 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.852 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-06-06T04:34:41.852 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-06-06T04:34:41.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.853 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.853 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.853 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-06-06T04:34:41.853 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.853 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.853 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.853 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-06T04:34:41.853 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.854 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.854 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.854 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-06-06T04:34:41.854 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-06-06T04:34:41.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.854 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.855 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-06-06T04:34:41.855 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.855 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.855 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.855 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-06T04:34:41.855 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.855 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.855 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.856 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-06-06T04:34:41.856 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-06-06T04:34:41.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.856 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.856 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.856 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-06-06T04:34:41.856 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.857 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.857 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.857 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-06-06T04:34:41.857 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.857 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.857 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.857 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-06-06T04:34:41.857 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-06-06T04:34:41.858 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.858 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.858 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-06-06T04:34:41.858 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.858 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.858 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.858 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-06T04:34:41.859 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.859 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.859 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.859 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-06-06T04:34:41.859 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-06-06T04:34:41.859 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.859 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.860 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.860 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:41.860 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.860 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.860 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.860 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-06T04:34:41.860 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.861 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.861 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.861 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-06-06T04:34:41.861 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-06-06T04:34:41.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.861 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.861 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.861 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-06-06T04:34:41.862 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.862 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.862 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.862 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-06-06T04:34:41.862 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.862 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.863 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.863 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-06-06T04:34:41.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-06-06T04:34:41.863 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.863 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.863 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.863 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-06-06T04:34:41.864 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.864 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.864 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.864 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-06T04:34:41.864 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.864 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.864 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.864 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.865 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.865 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-06-06T04:34:41.865 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.865 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.865 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.865 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.866 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.866 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.866 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.866 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.866 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.866 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.866 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.866 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.867 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.867 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.867 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.867 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.867 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.867 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.867 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.867 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.868 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.868 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.868 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.868 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.868 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.868 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.868 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.868 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.869 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.869 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.869 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.869 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.869 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.869 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.869 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.869 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.870 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.870 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.870 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.870 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.870 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.870 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.871 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.871 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.871 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.871 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.871 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.871 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.871 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.871 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.872 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.872 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.872 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.872 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.872 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.872 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.872 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.873 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.873 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.873 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.873 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.873 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.873 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.874 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.874 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.874 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.874 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-06-06T04:34:41.874 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-06-06T04:34:41.874 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.874 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.875 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.875 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-06-06T04:34:41.875 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.875 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.875 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.875 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.875 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.875 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.876 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.876 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-06-06T04:34:41.876 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-06-06T04:34:41.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.876 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.876 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.876 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-06-06T04:34:41.876 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.877 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.877 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.877 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.877 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.877 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.877 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.877 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-06-06T04:34:41.878 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-06-06T04:34:41.878 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.878 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.878 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.878 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-06-06T04:34:41.878 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.878 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.878 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.879 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.879 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.879 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.879 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.879 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.879 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.880 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-06-06T04:34:41.880 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-06-06T04:34:41.880 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-06-06T04:34:41.880 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.880 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-06-06T04:34:41.880 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-06-06T04:34:41.880 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.880 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.881 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.881 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-06-06T04:34:41.881 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.881 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.881 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.881 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-06T04:34:41.881 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.881 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.882 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.882 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-06-06T04:34:41.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-06-06T04:34:41.882 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.882 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.882 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.882 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-06-06T04:34:41.883 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.883 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.883 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.883 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-06T04:34:41.883 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.883 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.883 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.884 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-06-06T04:34:41.884 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-06-06T04:34:41.884 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.884 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.884 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.884 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.884 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.884 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.885 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.885 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-06-06T04:34:41.885 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.885 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.885 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.885 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-06-06T04:34:41.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-06-06T04:34:41.886 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:41.886 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.886 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.886 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-06-06T04:34:41.886 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-06-06T04:34:41.886 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.886 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.886 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-06T04:34:41.887 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.887 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.887 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.887 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.887 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.887 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.888 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.888 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.888 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.888 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.888 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.888 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.888 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.888 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.888 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.889 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.889 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.889 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.889 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.889 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.889 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.889 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.890 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.890 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.890 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.890 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.890 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.890 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.890 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.891 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.891 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.891 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.891 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.891 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.891 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.891 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.892 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.892 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.892 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.892 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.892 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.892 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.892 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.892 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.893 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.893 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.893 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.893 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.893 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.893 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.893 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.893 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.894 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.894 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.894 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.894 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.894 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.894 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.894 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.895 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.895 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.895 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.895 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.895 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.895 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.895 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.896 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.896 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-06-06T04:34:41.896 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-06-06T04:34:41.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.896 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.896 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.897 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.897 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.897 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.897 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-06-06T04:34:41.897 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.897 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.897 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.898 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-06-06T04:34:41.898 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-06-06T04:34:41.898 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.898 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.898 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.898 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-06-06T04:34:41.898 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.898 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.899 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.899 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-06-06T04:34:41.899 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.899 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.899 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.899 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-06-06T04:34:41.899 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-06-06T04:34:41.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.900 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.900 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-06-06T04:34:41.900 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.900 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.900 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.900 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T04:34:41.901 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.901 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.901 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.901 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-06-06T04:34:41.901 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-06-06T04:34:41.901 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.901 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.901 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.902 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-06-06T04:34:41.902 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-06-06T04:34:41.902 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-06-06T04:34:41.902 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.902 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-06T04:34:41.902 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.902 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.902 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.903 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-06-06T04:34:41.903 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-06-06T04:34:41.903 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.903 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.903 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.903 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.903 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.904 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.904 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.904 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-06-06T04:34:41.904 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.904 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.904 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.904 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-06-06T04:34:41.905 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-06-06T04:34:41.905 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.905 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.905 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.905 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-06-06T04:34:41.905 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.905 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.905 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.906 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T04:34:41.906 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.906 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.906 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.906 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.906 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.907 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-06-06T04:34:41.907 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.907 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.907 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.907 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.907 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.907 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.908 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.908 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.908 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.908 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.908 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.908 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.908 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.908 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.908 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.909 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.909 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.909 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.909 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.909 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.909 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.909 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.909 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.910 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.910 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.910 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.910 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.910 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.911 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.911 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.911 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.911 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.911 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.911 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.911 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.911 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.912 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.912 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.912 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.912 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.912 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.912 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.912 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.912 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.913 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.913 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.913 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.913 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.913 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.913 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.913 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.913 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.914 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.914 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.914 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.914 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.914 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.914 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.915 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.915 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.915 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.915 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.915 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.915 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.915 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.915 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.916 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.916 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.916 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.916 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-06-06T04:34:41.916 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.916 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.916 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.917 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.917 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.917 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.917 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.917 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.917 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.917 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.918 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.918 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.918 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.918 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.918 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.918 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.918 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.918 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.919 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.919 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.919 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.919 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.919 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.919 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.919 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.920 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.920 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.920 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.920 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.920 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.920 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.920 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.920 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.921 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.921 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.921 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.921 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.921 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.921 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.921 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.921 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.922 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.922 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.922 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.922 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.922 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.922 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.922 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.922 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.923 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.923 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.923 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.923 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.923 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.923 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.923 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.924 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.924 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.924 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.924 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.924 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.924 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.924 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.924 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.925 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.925 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.925 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.925 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.925 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.925 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-06-06T04:34:41.926 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.926 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.926 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.926 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-06-06T04:34:41.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-06-06T04:34:41.926 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.926 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.926 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.927 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-06-06T04:34:41.927 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.927 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.927 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.927 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-06T04:34:41.927 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.927 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.927 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.928 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-06-06T04:34:41.928 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-06-06T04:34:41.928 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.928 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.928 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.928 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:41.928 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-06-06T04:34:41.929 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.929 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.929 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-06T04:34:41.929 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.929 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.929 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.929 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.930 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.930 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.930 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.930 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.930 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.930 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.930 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.930 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.931 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.931 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.931 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.931 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.931 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.931 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.931 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.931 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.932 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.932 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.932 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.932 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.932 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.932 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.932 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.933 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.933 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.933 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.933 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.933 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.933 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.933 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.933 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.934 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.934 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.934 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.934 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.934 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.934 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.935 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.935 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.935 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.935 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.935 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.935 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.935 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.935 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.936 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.936 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.936 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.936 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.936 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.936 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.936 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.937 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.937 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.937 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.937 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.937 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.937 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.937 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.938 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.938 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.938 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.938 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.938 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.938 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.939 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-06-06T04:34:41.939 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.939 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.939 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.939 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-06-06T04:34:41.939 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-06-06T04:34:41.939 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.939 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.939 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.940 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-06-06T04:34:41.940 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.940 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.940 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.940 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-06-06T04:34:41.940 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-06T04:34:41.940 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.940 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.941 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.941 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.941 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.941 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.942 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.942 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.942 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.942 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.942 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.942 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.942 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.942 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.943 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.943 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.943 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.943 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.943 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.943 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.944 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.944 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.944 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.944 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.944 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.944 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.944 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.944 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.945 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.945 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.945 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.945 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.945 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.945 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.945 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.946 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.946 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.946 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.946 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.946 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.946 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.946 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.946 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.947 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.947 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.947 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.947 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.947 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.947 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.947 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.947 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.948 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.948 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.948 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.948 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.948 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.948 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.948 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.948 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.949 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.949 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.949 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.949 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.949 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.949 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-06-06T04:34:41.949 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-06-06T04:34:41.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.950 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.950 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.950 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.950 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.950 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.950 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.950 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-06-06T04:34:41.951 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-06T04:34:41.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.951 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.951 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-06-06T04:34:41.951 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-06-06T04:34:41.951 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.951 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.952 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.952 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-06-06T04:34:41.952 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.952 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.952 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.952 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-06-06T04:34:41.952 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.952 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.952 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.953 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-06-06T04:34:41.953 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-06-06T04:34:41.953 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:41.953 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.953 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.954 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-06-06T04:34:41.954 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.954 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.954 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.954 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-06-06T04:34:41.954 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.954 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.954 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.955 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-06-06T04:34:41.955 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-06-06T04:34:41.955 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.955 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.955 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.955 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.955 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.956 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.956 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.956 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-06-06T04:34:41.956 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.956 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.956 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.956 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-06-06T04:34:41.957 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-06-06T04:34:41.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.957 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.957 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-06-06T04:34:41.957 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.957 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.957 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.958 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-06T04:34:41.958 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-06T04:34:41.958 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.958 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.958 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.958 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.959 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-06-06T04:34:41.959 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.959 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.959 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.959 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.959 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.959 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.959 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.960 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.960 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.960 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.960 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.960 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.960 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.960 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.961 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.961 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.961 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.961 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.961 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.961 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.961 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.961 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.962 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.962 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.962 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.962 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.962 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.962 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.962 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.962 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.963 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.963 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.963 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.963 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.963 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.963 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.964 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.964 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.964 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.964 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.964 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.964 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.964 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.964 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.965 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.965 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.965 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.965 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.965 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.965 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.965 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.965 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.965 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.966 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.966 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.966 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.966 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.966 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.966 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.967 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.967 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.967 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.967 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.967 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.967 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.967 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.967 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.968 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.968 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-06-06T04:34:41.968 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.968 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.968 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.969 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.969 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.969 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.969 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.969 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.969 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.969 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.969 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.969 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.970 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.970 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.970 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.970 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.970 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.970 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.970 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.970 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.971 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.971 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.971 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.971 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.971 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.971 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.971 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.972 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.972 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.972 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.972 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.972 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.972 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.972 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.972 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.973 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.973 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.973 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.973 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.973 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.974 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.974 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.974 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.974 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.974 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.974 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.974 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.974 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.975 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.975 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.975 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.975 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.975 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.975 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.975 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.976 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.976 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.976 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.976 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.976 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.976 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.976 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.976 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.977 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.977 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.977 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.977 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.977 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.977 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.978 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-06-06T04:34:41.978 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.978 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.978 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.978 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.978 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.978 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.979 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.979 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.979 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.979 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.979 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.979 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.979 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.979 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.980 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.980 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.980 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.980 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.980 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.980 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.980 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.980 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.981 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.981 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.981 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.981 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.981 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.981 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.981 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.982 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.982 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.982 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.982 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.982 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.982 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.982 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.983 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.983 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.983 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.983 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.983 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.983 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.983 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.983 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.984 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.984 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.984 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.984 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.984 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.984 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.984 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.984 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.985 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.985 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.985 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.985 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.985 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.985 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.986 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.986 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.986 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.986 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.986 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.986 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.986 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.986 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.987 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.987 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.987 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:41.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:41.987 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-06-06T04:34:41.987 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:41.987 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:41.987 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:41.988 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-06-06T04:34:41.988 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-06-06T04:34:41.988 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.988 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.988 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:41.988 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.988 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.988 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.989 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.989 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-06T04:34:41.989 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.989 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.989 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.989 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:41.989 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:41.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.990 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.990 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.990 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.990 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.990 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.990 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.990 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.991 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.991 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.991 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.991 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.991 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.991 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.991 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.991 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.992 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.992 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.992 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:41.992 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:41.992 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.992 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.992 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.992 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.993 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.993 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.993 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.993 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.993 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.993 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.993 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.994 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:41.994 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:41.994 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.994 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.994 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.994 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:41.994 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.994 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.995 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.995 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:41.995 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:41.995 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:41.995 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:41.995 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:41.995 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:41.995 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.996 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.996 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.996 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.996 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.996 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:41.996 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:41.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:41.997 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.997 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:41.997 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.997 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.997 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:41.997 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:41.997 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:41.997 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:41.998 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:41.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:41.998 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-06-06T04:34:41.998 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-06-06T04:34:41.998 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:41.998 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:41.998 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:41.999 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:41.999 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:41.999 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:41.999 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:41.999 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-06-06T04:34:41.999 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-06-06T04:34:41.999 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-06-06T04:34:41.999 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:41.999 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-06-06T04:34:42.000 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.000 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.000 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.000 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.000 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-06-06T04:34:42.001 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.001 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.001 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.001 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.001 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.001 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.001 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.002 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.002 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.002 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.002 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.002 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.002 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.002 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.002 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.003 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.003 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.003 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.003 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.003 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.003 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.003 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.004 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.004 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.004 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.004 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.004 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.004 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.004 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.004 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.005 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.005 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.005 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.005 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.005 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.005 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.005 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.006 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.006 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.006 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.006 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.006 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.006 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.006 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.006 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.007 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.007 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.007 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.007 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.007 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.007 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.007 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.007 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.008 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.008 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.008 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.008 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.008 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.008 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.009 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.009 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.009 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.009 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.009 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.009 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.009 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.009 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-06-06T04:34:42.010 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.010 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.010 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.011 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.011 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.011 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.011 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.011 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.011 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.012 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.012 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.012 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.012 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.012 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.012 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.012 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.012 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.013 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.013 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.013 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.013 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.013 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.013 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.013 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.014 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.014 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.014 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.014 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.014 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.014 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.014 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.014 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.015 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.015 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.015 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.015 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.015 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.015 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.015 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.016 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.016 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.016 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.016 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.016 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.016 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.016 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.016 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.017 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.017 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.017 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.017 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.017 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.017 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.017 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.018 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.018 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.018 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.018 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.018 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.018 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.019 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.019 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.019 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.019 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-06-06T04:34:42.020 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.020 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.020 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.020 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.020 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.021 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.021 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.021 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.021 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.021 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.021 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.021 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.021 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.022 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.022 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.022 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.022 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.022 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.022 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.022 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.023 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.023 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.023 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.023 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.023 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.023 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.023 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.023 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.024 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.024 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.024 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.024 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.024 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.024 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.024 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.025 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.025 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.025 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.025 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.025 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.025 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.025 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.026 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.026 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.026 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.026 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.026 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.026 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.026 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.026 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.027 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.027 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.027 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.027 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.027 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.027 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.027 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.028 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.028 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.028 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.028 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.028 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.028 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.028 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.028 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.029 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.029 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-06-06T04:34:42.029 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-06-06T04:34:42.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.029 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.029 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.029 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-06-06T04:34:42.030 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.030 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.030 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.030 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.030 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.030 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.030 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.031 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-06-06T04:34:42.031 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-06-06T04:34:42.031 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:42.031 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.031 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.031 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-06-06T04:34:42.031 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-06-06T04:34:42.031 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.032 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.032 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-06-06T04:34:42.032 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-06T04:34:42.032 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.032 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.032 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.032 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.033 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-06-06T04:34:42.033 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.033 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.033 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.033 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-06-06T04:34:42.033 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-06-06T04:34:42.033 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.034 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.034 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.034 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.034 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.034 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.034 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.034 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-06T04:34:42.034 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.035 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.035 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.035 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-06-06T04:34:42.035 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-06-06T04:34:42.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.035 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.035 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.036 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.036 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.036 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.036 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.036 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.036 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.036 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.036 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.037 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.037 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.037 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.037 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.037 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.037 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.037 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.037 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.038 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.038 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.038 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.038 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.038 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.038 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.038 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.039 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.039 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.039 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.039 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.039 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.039 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.040 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.040 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.040 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.040 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.040 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.040 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.040 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.040 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.041 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.041 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.041 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.041 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.041 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.041 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.041 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.042 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.042 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.042 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.042 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.042 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.042 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.042 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.042 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.043 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.043 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.043 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.043 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.043 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.043 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.043 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.044 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.044 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.044 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.044 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.044 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.044 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.044 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.044 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.045 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.045 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.045 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.045 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.045 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.045 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-06-06T04:34:42.045 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-06-06T04:34:42.046 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.046 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.046 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.046 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-06-06T04:34:42.046 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.046 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.046 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.046 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-06-06T04:34:42.047 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.047 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.047 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.047 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-06-06T04:34:42.047 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-06-06T04:34:42.047 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.047 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.048 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.048 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-06-06T04:34:42.048 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.048 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.048 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.048 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-06-06T04:34:42.048 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.048 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.048 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.049 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.049 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.049 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-06-06T04:34:42.049 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.049 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.049 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.050 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-06-06T04:34:42.050 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-06-06T04:34:42.050 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.050 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.050 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.050 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.050 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.051 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.051 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.051 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.051 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.051 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.051 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.051 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-06-06T04:34:42.051 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-06-06T04:34:42.052 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.052 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.052 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.052 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.052 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.052 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.052 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.052 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-06T04:34:42.053 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-06T04:34:42.053 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.053 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.053 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.053 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.053 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.053 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.054 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.054 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.054 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.054 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.054 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.054 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.054 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.054 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.055 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.055 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.055 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.055 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.055 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.055 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.055 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.055 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.056 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.056 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.056 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.056 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.056 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.056 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.056 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.057 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.057 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.057 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.057 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.057 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.057 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.057 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.058 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.058 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.058 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.058 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.058 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.058 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.058 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.058 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.059 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.059 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.059 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.059 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.059 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.059 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.059 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.059 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.060 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.060 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.060 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.060 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.060 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.060 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.060 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.060 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.061 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.061 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.061 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.061 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.061 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.061 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.061 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.061 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.062 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-06-06T04:34:42.062 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-06-06T04:34:42.062 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.062 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.062 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.062 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.062 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.063 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.063 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.063 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.063 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.063 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.063 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.063 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-06T04:34:42.064 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-06T04:34:42.064 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.064 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.064 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.064 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-06-06T04:34:42.064 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.064 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.064 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.065 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.065 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.065 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.065 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.065 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-06-06T04:34:42.065 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-06-06T04:34:42.065 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:42.066 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.066 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.066 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-06-06T04:34:42.066 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.066 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.066 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.066 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.066 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.067 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.067 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.067 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-06-06T04:34:42.067 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-06-06T04:34:42.067 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.067 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.067 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.067 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-06-06T04:34:42.068 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.068 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.068 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.068 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-06T04:34:42.068 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.068 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.068 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.069 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-06-06T04:34:42.069 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-06-06T04:34:42.069 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.069 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.069 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.069 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-06-06T04:34:42.069 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.069 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.070 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.070 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-06T04:34:42.070 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.070 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.070 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.070 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-06-06T04:34:42.070 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-06-06T04:34:42.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.071 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.071 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.071 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-06-06T04:34:42.071 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.071 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.071 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.071 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.072 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.072 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.072 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.072 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-06-06T04:34:42.072 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-06-06T04:34:42.072 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.072 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.073 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.073 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-06-06T04:34:42.073 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.073 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.073 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.073 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-06-06T04:34:42.073 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-06-06T04:34:42.073 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.074 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.074 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.074 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.074 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.074 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-06-06T04:34:42.074 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-06-06T04:34:42.074 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.075 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.075 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.075 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-06-06T04:34:42.075 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-06-06T04:34:42.075 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-06-06T04:34:42.075 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.075 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.075 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.076 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.076 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.076 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-06-06T04:34:42.076 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.077 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.077 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.077 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.077 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.077 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.077 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.077 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.078 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.078 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.078 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.078 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.078 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.078 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.078 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.078 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.079 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.079 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.079 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.079 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.079 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.079 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.079 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.080 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.080 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.080 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.080 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.080 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.080 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.080 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.080 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.081 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.081 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.081 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.081 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.081 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.081 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.081 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.081 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.082 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.082 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.082 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.082 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.082 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.082 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.082 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.082 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.083 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.083 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.083 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.083 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.083 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.083 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.083 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.083 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.084 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.084 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.084 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.084 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.084 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.084 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.084 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.085 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.085 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.085 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.085 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.085 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.085 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.085 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-06-06T04:34:42.086 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-06-06T04:34:42.086 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.086 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.086 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.086 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:42.086 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.086 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.086 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.087 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.087 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.087 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.087 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.087 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-06-06T04:34:42.087 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-06-06T04:34:42.087 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.088 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.088 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.088 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.088 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.088 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.088 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.088 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.088 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.089 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.089 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.089 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-06-06T04:34:42.089 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-06-06T04:34:42.089 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.089 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.089 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.089 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.090 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.090 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.090 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.090 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.090 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.090 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.090 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.090 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.091 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.091 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-06-06T04:34:42.091 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.091 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.091 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.091 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-06-06T04:34:42.092 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-06-06T04:34:42.092 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.092 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.092 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.092 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.092 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.092 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.092 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.093 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.093 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.093 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.093 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.093 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-06-06T04:34:42.093 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-06-06T04:34:42.093 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.094 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.095 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.095 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.095 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.095 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.095 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.095 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.095 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.095 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.096 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.096 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.096 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.096 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.096 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.096 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.096 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.097 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.097 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.097 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.097 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.097 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.097 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.097 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.097 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.098 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.098 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.098 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.098 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.098 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.098 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.098 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.098 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.099 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.099 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.099 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.099 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.099 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.099 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.099 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.100 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.100 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.100 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.100 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.100 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.100 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.100 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.100 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.101 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.101 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.101 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.101 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.101 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.101 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.101 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.101 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.102 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.102 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.102 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.102 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.102 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.102 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.102 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.103 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.103 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.103 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.103 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.103 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.103 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.103 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.104 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-06-06T04:34:42.104 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-06-06T04:34:42.104 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.104 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.104 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.104 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.104 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.104 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.105 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.105 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.105 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.105 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.105 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.105 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-06-06T04:34:42.105 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-06-06T04:34:42.105 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.106 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.106 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.106 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.106 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.106 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.106 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.106 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.106 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.107 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.107 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.107 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.107 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.107 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-06-06T04:34:42.107 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.108 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.108 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.108 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.108 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.108 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.108 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.108 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.108 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.109 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.109 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.109 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.109 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.109 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.109 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.109 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.109 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.110 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.110 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.110 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.110 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.110 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.110 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.110 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.111 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.111 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.111 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.111 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.111 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.111 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.111 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.111 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.112 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.112 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.112 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.112 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.112 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.112 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.112 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.113 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.113 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.113 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.113 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.113 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.113 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.113 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.113 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.114 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.114 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.114 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.114 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.114 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.114 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.114 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.114 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.115 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.115 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.115 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.115 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.115 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.115 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.115 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.116 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.116 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.116 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.116 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.116 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.116 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.116 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.116 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.117 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-06-06T04:34:42.117 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.117 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.117 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.117 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.117 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.117 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.118 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.118 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.118 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.118 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.118 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.118 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.118 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.118 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.119 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.119 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.119 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.119 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.119 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.119 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.119 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.119 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.120 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.120 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.120 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.120 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.120 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.120 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.120 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.121 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.121 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.121 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.121 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.121 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.121 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.121 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.122 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.122 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.122 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.122 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.122 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.122 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.122 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.122 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.123 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.123 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.123 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.123 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.123 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.123 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.123 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.123 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.124 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.124 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.124 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.124 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.124 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.124 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.124 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.125 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.125 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.125 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.125 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.125 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.125 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.125 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.125 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.126 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.126 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.126 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-06-06T04:34:42.126 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-06-06T04:34:42.126 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.126 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.126 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.127 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-06-06T04:34:42.127 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.127 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.127 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.127 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-06T04:34:42.127 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.127 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.127 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.128 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.128 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.128 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-06-06T04:34:42.128 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.128 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.128 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.128 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.129 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.129 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.129 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.129 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.129 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.129 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.129 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.129 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.130 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.130 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.130 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.130 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.130 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.130 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.130 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.130 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.131 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.131 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.131 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.131 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.131 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.131 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.131 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.132 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.132 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.132 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.132 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.132 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.132 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.132 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.132 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.132 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.133 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.133 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.133 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.133 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.133 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.133 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.133 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.134 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.134 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.134 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.134 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.134 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.134 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.134 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.134 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.135 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.135 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.135 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.135 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.135 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.135 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.135 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.136 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.136 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.136 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.136 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.136 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.136 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.136 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.136 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.137 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.137 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.137 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.137 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-06-06T04:34:42.137 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-06-06T04:34:42.137 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.137 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.138 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.138 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.138 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.138 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.138 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.138 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.138 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.138 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.138 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.139 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-06-06T04:34:42.139 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-06-06T04:34:42.139 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.139 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.139 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.139 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-06-06T04:34:42.139 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.140 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.140 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.140 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.140 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.140 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.140 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.140 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-06-06T04:34:42.141 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-06-06T04:34:42.141 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.141 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.141 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.141 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.141 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.141 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.141 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.142 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.142 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.142 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.142 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.142 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-06-06T04:34:42.142 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-06-06T04:34:42.142 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.143 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.143 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.143 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.143 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.143 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.143 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.143 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.143 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.144 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.144 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.144 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-06-06T04:34:42.144 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-06-06T04:34:42.144 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:42.144 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.144 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.145 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.145 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.145 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.145 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.145 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.145 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.145 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.145 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.146 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-06-06T04:34:42.146 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-06-06T04:34:42.146 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.146 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.146 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.146 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.146 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.147 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.147 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.147 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.147 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.147 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.147 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.147 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-06-06T04:34:42.147 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-06-06T04:34:42.148 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.148 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.148 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.148 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.148 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.148 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.148 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.148 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.149 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.149 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.149 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.149 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-06-06T04:34:42.149 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-06-06T04:34:42.149 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.149 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.150 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.150 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.150 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.150 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.150 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.150 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.150 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.150 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.151 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.151 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-06-06T04:34:42.151 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-06-06T04:34:42.151 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.151 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.151 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.151 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.152 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-06-06T04:34:42.152 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-06-06T04:34:42.152 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.152 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.152 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.152 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.152 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.153 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-06-06T04:34:42.153 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-06-06T04:34:42.153 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.153 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.153 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.153 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.153 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.153 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.154 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.154 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.154 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.154 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.154 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.154 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-06-06T04:34:42.154 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-06-06T04:34:42.155 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.155 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.155 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.155 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.155 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.155 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.155 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.155 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.155 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.156 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.156 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.156 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-06-06T04:34:42.156 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-06-06T04:34:42.156 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.156 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.156 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.157 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.157 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.157 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.157 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.157 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.157 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.157 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.157 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.158 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.158 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.158 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-06-06T04:34:42.158 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.158 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.158 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.159 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-06-06T04:34:42.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-06-06T04:34:42.159 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.159 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.159 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.159 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.159 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.159 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.160 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.160 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-06-06T04:34:42.160 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.160 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.160 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.160 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-06-06T04:34:42.160 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-06-06T04:34:42.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.161 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.161 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.161 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.161 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.161 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.161 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.161 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-06-06T04:34:42.161 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.162 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.162 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.162 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.162 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.162 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.162 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.163 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.163 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.163 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.163 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.163 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.163 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.163 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.163 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.164 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.164 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.164 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.164 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.164 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.164 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.164 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.165 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.165 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.165 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.165 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.165 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.165 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.166 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.166 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.166 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.166 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.166 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.166 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.167 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.167 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.167 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.167 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.167 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.167 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.167 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.167 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.168 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.168 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.168 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.168 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.168 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.168 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.168 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.168 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.169 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.169 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.169 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.169 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.169 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.169 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.169 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.169 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.170 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.170 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.170 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.170 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.170 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.170 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.170 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.170 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.171 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.171 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-06-06T04:34:42.171 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.171 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.171 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.171 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.172 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.172 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.172 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.172 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.172 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.172 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.172 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.172 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.173 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.173 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.173 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.173 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.173 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.173 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.173 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.173 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.174 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.174 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.174 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.174 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.174 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.174 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.174 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.175 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.175 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.175 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.175 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.175 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.175 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.175 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.175 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.176 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.176 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.176 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.176 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.176 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.176 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.176 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.176 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.177 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.177 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.177 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.177 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.177 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.177 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.177 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.177 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.178 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.178 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.178 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.178 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.178 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.178 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.178 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.179 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.179 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.179 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.179 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.179 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.179 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.179 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.179 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.180 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.180 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.180 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.180 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.180 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.180 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-06-06T04:34:42.181 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.181 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.181 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.181 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.181 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.181 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.181 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.181 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.182 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.182 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.182 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.182 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.182 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.182 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.182 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.182 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.183 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.183 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.183 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.183 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.183 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.183 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.183 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.184 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.184 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.184 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.184 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.184 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.184 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.184 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.184 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.184 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.185 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.185 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.185 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.185 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.185 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.185 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.185 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.186 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.186 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.186 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.186 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.186 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.186 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.186 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.186 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.187 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.187 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.187 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.187 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.187 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.187 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.187 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.187 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.188 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.188 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.188 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.188 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.188 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.188 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.188 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.188 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.189 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.189 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.189 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.189 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.189 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.189 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.189 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.190 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-06-06T04:34:42.190 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.190 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.190 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.190 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-06-06T04:34:42.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-06-06T04:34:42.191 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.191 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.191 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.191 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-06T04:34:42.191 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.191 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.191 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.191 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.192 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.192 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.192 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.192 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-06-06T04:34:42.192 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-06-06T04:34:42.192 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.192 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.193 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.193 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-06-06T04:34:42.193 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.193 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.193 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.193 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.193 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.193 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.194 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.194 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.194 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.194 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.194 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.194 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.195 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.195 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.195 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.195 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.195 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.195 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.195 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.195 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.196 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.196 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.196 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.196 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.196 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.196 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.196 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.197 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.197 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.197 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.197 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.197 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.197 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.197 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.198 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.198 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.198 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.198 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.198 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.198 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.198 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.199 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.199 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.199 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.199 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.199 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.199 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.199 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.199 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.200 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.200 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.200 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.200 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.200 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.200 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.200 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.200 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.200 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.201 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.201 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.201 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.201 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.201 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.201 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.201 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.202 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.202 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.202 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.202 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.202 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.202 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.202 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.202 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-06-06T04:34:42.203 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.203 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.203 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.203 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-06-06T04:34:42.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-06-06T04:34:42.204 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.204 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.204 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.204 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.204 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.204 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.204 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.204 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-06T04:34:42.205 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.205 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.205 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.205 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-06-06T04:34:42.205 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-06-06T04:34:42.205 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.205 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.206 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.206 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.206 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.206 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.206 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.206 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-06T04:34:42.206 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.206 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.207 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.207 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-06-06T04:34:42.207 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-06-06T04:34:42.207 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.207 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.207 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.207 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.207 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.208 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.208 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.208 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-06T04:34:42.208 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.208 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.208 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.209 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-06-06T04:34:42.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-06-06T04:34:42.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.209 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.209 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.209 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.209 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.209 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.209 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.210 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-06T04:34:42.210 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.210 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.210 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.210 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-06-06T04:34:42.210 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-06-06T04:34:42.210 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.211 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.211 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.211 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.211 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.211 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.211 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.211 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-06T04:34:42.211 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.212 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.212 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.212 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-06-06T04:34:42.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-06-06T04:34:42.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.212 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.212 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.213 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.213 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.213 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.213 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.213 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.213 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.213 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.213 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.214 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-06-06T04:34:42.214 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-06-06T04:34:42.214 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.214 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.214 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.214 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.214 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.215 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.215 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.215 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.215 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.215 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.215 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.215 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-06-06T04:34:42.216 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-06-06T04:34:42.216 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.216 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.216 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.216 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-06T04:34:42.216 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.216 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.216 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.217 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.217 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.217 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.217 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.217 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-06-06T04:34:42.217 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-06-06T04:34:42.217 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.218 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.218 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.218 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.218 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.218 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.218 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.218 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.218 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.219 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.219 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.219 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-06-06T04:34:42.219 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-06-06T04:34:42.219 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.219 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.219 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.220 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-06-06T04:34:42.220 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-06-06T04:34:42.220 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.220 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.220 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.220 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.220 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.220 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.221 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-06-06T04:34:42.221 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-06-06T04:34:42.221 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.221 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.221 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.221 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-06-06T04:34:42.221 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.222 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.222 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.222 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.222 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.222 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.222 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.222 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-06-06T04:34:42.222 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-06-06T04:34:42.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.223 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.223 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.223 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.223 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.223 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.223 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.223 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.224 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.224 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.224 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.224 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-06-06T04:34:42.224 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-06-06T04:34:42.224 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.224 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.225 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.225 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.225 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.225 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.225 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.225 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.225 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.225 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.226 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.226 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.226 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.226 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.226 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.226 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.226 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.227 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.227 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.227 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.227 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.227 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.227 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.227 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.227 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.227 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.228 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.228 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.228 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.228 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.228 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.228 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.228 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.229 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.229 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.229 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.229 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.229 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.229 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.229 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.229 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.230 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.230 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.230 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.230 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.230 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.230 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.231 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.231 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.231 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.231 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.231 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.231 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.231 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.232 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.232 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.232 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.232 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.232 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.232 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.232 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.232 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.233 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.233 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.233 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.233 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.233 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.233 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.234 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.234 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.234 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.234 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.234 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.234 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.234 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-06-06T04:34:42.235 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-06-06T04:34:42.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.235 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.235 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.235 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.235 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.235 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.235 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.235 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.236 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.236 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.236 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.236 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-06-06T04:34:42.236 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-06-06T04:34:42.236 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.236 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.237 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.237 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.237 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.237 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.237 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.237 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.237 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.237 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.238 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.238 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-06-06T04:34:42.238 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-06-06T04:34:42.238 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.238 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.238 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.238 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-06T04:34:42.239 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.239 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.239 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.239 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.239 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.239 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.239 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-06-06T04:34:42.240 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.240 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.240 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.241 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.241 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.241 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.241 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.241 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.241 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.241 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.241 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.241 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.242 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.242 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.242 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.242 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.242 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.242 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.242 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.242 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.243 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.243 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.243 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.243 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.243 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.243 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.243 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.244 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.244 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.244 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.244 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.244 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.244 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.244 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.244 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.245 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.245 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.245 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.245 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.245 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.245 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.245 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.246 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.246 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.246 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.246 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.246 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.246 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.246 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.246 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.247 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.247 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.247 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.247 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.247 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.247 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.247 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.248 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.248 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.248 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.248 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.248 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.248 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.248 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.248 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.249 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.249 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.249 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.249 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.249 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.249 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.250 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-06-06T04:34:42.250 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.250 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.250 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.250 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.250 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.250 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.250 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.251 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.251 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.251 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.251 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.251 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.251 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.251 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.251 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.251 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.252 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.252 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.252 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.252 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.252 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.252 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.252 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.253 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.253 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.253 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.253 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.253 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.253 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.253 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.253 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.254 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.254 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.254 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.254 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.254 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.254 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.255 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.255 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.255 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.255 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.255 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.255 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.255 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.255 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.256 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.256 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.256 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.256 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.256 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.256 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.256 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.256 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.257 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.257 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.257 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.257 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.257 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.257 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.257 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.258 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.258 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.258 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.258 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.258 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.258 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.258 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.258 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.259 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-06-06T04:34:42.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-06-06T04:34:42.259 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.259 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.259 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.259 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-06-06T04:34:42.259 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.260 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.260 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.260 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-06T04:34:42.260 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.260 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.260 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.260 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-06-06T04:34:42.261 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-06-06T04:34:42.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.261 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.261 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.261 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.261 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.261 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.262 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-06-06T04:34:42.262 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.262 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.262 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.262 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-06-06T04:34:42.262 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-06-06T04:34:42.262 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.263 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.263 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.263 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-06-06T04:34:42.263 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.263 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.263 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.263 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-06T04:34:42.263 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.264 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.264 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.264 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-06-06T04:34:42.264 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-06-06T04:34:42.264 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.264 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.264 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.265 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.265 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.265 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.265 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.265 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-06T04:34:42.265 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.265 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.265 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.266 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.266 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.266 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-06-06T04:34:42.266 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.266 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.266 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.267 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-06-06T04:34:42.267 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-06-06T04:34:42.267 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-06-06T04:34:42.267 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.267 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.267 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-06-06T04:34:42.267 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.267 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.268 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.268 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.268 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.268 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.268 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.268 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-06-06T04:34:42.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-06-06T04:34:42.269 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.269 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.269 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.269 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.269 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.269 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.269 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.269 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.270 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.270 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.270 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.270 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-06-06T04:34:42.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-06-06T04:34:42.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.270 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.271 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.271 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-06-06T04:34:42.271 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.271 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.271 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.271 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.271 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.271 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.272 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.272 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.272 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.272 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.272 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.272 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.272 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.273 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.273 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.273 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.273 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.273 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.273 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.273 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.273 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.274 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.274 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.274 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.274 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.274 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.274 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.274 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.275 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.275 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.275 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.275 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.275 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.275 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.275 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.275 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.276 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.276 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.276 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.276 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.276 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.277 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.277 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.277 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.277 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.277 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.277 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.277 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.277 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.277 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.278 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.278 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.278 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.278 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.278 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.278 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.278 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.278 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.279 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.279 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.279 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.279 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.279 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.279 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.279 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.280 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.280 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.280 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.280 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.280 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.280 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.280 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-06-06T04:34:42.281 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-06-06T04:34:42.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.281 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.281 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.281 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.281 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.281 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.281 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.282 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.282 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.282 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.282 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.282 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-06-06T04:34:42.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-06-06T04:34:42.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.283 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.283 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-06-06T04:34:42.283 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.283 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.283 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.283 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.283 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.284 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.284 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.284 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-06-06T04:34:42.284 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-06-06T04:34:42.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.284 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.284 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.285 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-06T04:34:42.285 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.285 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.285 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.285 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.285 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.285 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.285 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.286 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-06-06T04:34:42.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.286 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.286 INFO:teuthology.orchestra.run.smithi045.stdout: "modules": [ 2024-06-06T04:34:42.286 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-06-06T04:34:42.286 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard", 2024-06-06T04:34:42.287 INFO:teuthology.orchestra.run.smithi045.stdout: "iostat", 2024-06-06T04:34:42.287 INFO:teuthology.orchestra.run.smithi045.stdout: "nfs", 2024-06-06T04:34:42.287 INFO:teuthology.orchestra.run.smithi045.stdout: "restful" 2024-06-06T04:34:42.287 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.287 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-06-06T04:34:42.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.287 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-06-06T04:34:42.287 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.288 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.288 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.288 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-06-06T04:34:42.288 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-06-06T04:34:42.288 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.288 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.288 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.288 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-06-06T04:34:42.289 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.289 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.289 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.289 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-06-06T04:34:42.289 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.289 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.289 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.290 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.290 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.290 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.290 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.290 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.290 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.291 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.291 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.291 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.291 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.291 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.291 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.291 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.291 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.291 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.292 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.292 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.292 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.292 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.292 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.292 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.293 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.293 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.293 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.293 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.293 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.293 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.293 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.294 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.294 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.294 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.294 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.294 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.294 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.295 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.295 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.295 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.295 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.295 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.295 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.295 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.295 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.296 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.296 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.296 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.296 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.296 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.296 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.296 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.297 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.297 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.297 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.297 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.297 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.297 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.297 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.298 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.298 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.298 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.298 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.298 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.298 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-06-06T04:34:42.298 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-06-06T04:34:42.299 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.299 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.299 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.299 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.299 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.299 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.299 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.299 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-06-06T04:34:42.300 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.300 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.300 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.300 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-06-06T04:34:42.300 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-06-06T04:34:42.300 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.300 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.301 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.301 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-06-06T04:34:42.301 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.301 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.301 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.301 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-06-06T04:34:42.301 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.301 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.302 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.302 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-06-06T04:34:42.302 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-06-06T04:34:42.302 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.302 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.302 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.302 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.303 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.303 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.303 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.303 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-06-06T04:34:42.303 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.303 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.303 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.304 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-06-06T04:34:42.304 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-06-06T04:34:42.304 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.304 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.304 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.304 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.304 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.304 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.305 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-06-06T04:34:42.305 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.305 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.305 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.305 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-06-06T04:34:42.305 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-06-06T04:34:42.305 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.306 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.306 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.306 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-06-06T04:34:42.306 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.306 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.306 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-06-06T04:34:42.307 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.307 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.307 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.307 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-06-06T04:34:42.307 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-06-06T04:34:42.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.307 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.307 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.308 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.308 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.308 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.308 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.308 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-06-06T04:34:42.308 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.308 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.308 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.309 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-06-06T04:34:42.309 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-06-06T04:34:42.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.309 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.309 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.309 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.309 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.310 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.310 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.310 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-06T04:34:42.310 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.310 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.310 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.310 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-06-06T04:34:42.311 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-06-06T04:34:42.311 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.311 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.311 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.311 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.311 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.311 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.311 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.312 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-06-06T04:34:42.312 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.312 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.312 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.312 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.312 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.313 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-06-06T04:34:42.313 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.313 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.313 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.313 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-06-06T04:34:42.313 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-06-06T04:34:42.313 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.313 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.314 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.314 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.314 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.314 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.314 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.314 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-06-06T04:34:42.314 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.314 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.315 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.315 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-06-06T04:34:42.315 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-06-06T04:34:42.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.315 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.315 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.315 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-06-06T04:34:42.316 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.316 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.316 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.316 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-06-06T04:34:42.316 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T04:34:42.316 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.316 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.316 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-06-06T04:34:42.317 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-06-06T04:34:42.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-06-06T04:34:42.317 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.317 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.317 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-06-06T04:34:42.317 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-06-06T04:34:42.317 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-06-06T04:34:42.318 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.318 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-06T04:34:42.318 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T04:34:42.318 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.318 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.318 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-06-06T04:34:42.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-06-06T04:34:42.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-06-06T04:34:42.319 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.319 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.319 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-06-06T04:34:42.319 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-06-06T04:34:42.319 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-06-06T04:34:42.319 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.319 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-06T04:34:42.320 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.320 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.320 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.320 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-06-06T04:34:42.320 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-06-06T04:34:42.320 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.320 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.321 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.321 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-06-06T04:34:42.321 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.321 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.321 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.321 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-06T04:34:42.321 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-06T04:34:42.321 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.322 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.322 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-06-06T04:34:42.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-06-06T04:34:42.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:42.322 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.322 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.323 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-06-06T04:34:42.323 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-06-06T04:34:42.323 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-06-06T04:34:42.323 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.323 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-06-06T04:34:42.323 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-06T04:34:42.323 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.323 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.324 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-06-06T04:34:42.324 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-06-06T04:34:42.324 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.324 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.324 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.324 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-06-06T04:34:42.324 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.324 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.325 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.325 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-06-06T04:34:42.325 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T04:34:42.325 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.325 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.325 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-06-06T04:34:42.325 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-06-06T04:34:42.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-06-06T04:34:42.326 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.326 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.326 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-06-06T04:34:42.326 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-06-06T04:34:42.326 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-06-06T04:34:42.326 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.326 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-06T04:34:42.327 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T04:34:42.327 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.327 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.327 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.327 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.328 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.328 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.328 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.328 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.328 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.328 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.328 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.328 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.329 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.329 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.329 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.329 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.329 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.329 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.329 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.329 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.330 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.330 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.330 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.330 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.330 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.330 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.330 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.331 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.331 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.331 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.331 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.331 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.331 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.331 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.332 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.332 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.332 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.332 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.332 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.332 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.332 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.332 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.333 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.333 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.333 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.333 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.333 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.333 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.333 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.333 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.334 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.334 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.334 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.334 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.334 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.334 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.334 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.335 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.335 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.335 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.335 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.335 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.335 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.335 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.335 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.336 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-06-06T04:34:42.336 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-06-06T04:34:42.336 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:42.336 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.336 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.336 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-06-06T04:34:42.336 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.337 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.337 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.337 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-06T04:34:42.337 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.337 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.337 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.337 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-06-06T04:34:42.338 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-06-06T04:34:42.338 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.338 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.338 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.338 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-06-06T04:34:42.338 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.338 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.342 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.343 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-06-06T04:34:42.343 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-06-06T04:34:42.343 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-06-06T04:34:42.343 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-06-06T04:34:42.343 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-06-06T04:34:42.343 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.343 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-06-06T04:34:42.344 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.344 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.344 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.344 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-06-06T04:34:42.344 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-06-06T04:34:42.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.344 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.345 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.345 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.345 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.345 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.345 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.345 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-06T04:34:42.345 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.345 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.346 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.346 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-06-06T04:34:42.346 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-06-06T04:34:42.346 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.346 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.346 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.346 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-06-06T04:34:42.347 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.347 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.347 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.347 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-06T04:34:42.347 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.347 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.347 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.348 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-06-06T04:34:42.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-06-06T04:34:42.348 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.348 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.348 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.348 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-06-06T04:34:42.348 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-06-06T04:34:42.348 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.349 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.349 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-06T04:34:42.349 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-06T04:34:42.349 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.349 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.349 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-06-06T04:34:42.349 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-06-06T04:34:42.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-06-06T04:34:42.350 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.350 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.350 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:42.350 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.350 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.350 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.351 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-06T04:34:42.351 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.351 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.351 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.351 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.351 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.351 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-06-06T04:34:42.352 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.352 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.352 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.352 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-06-06T04:34:42.352 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-06-06T04:34:42.352 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:42.352 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.352 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.353 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-06-06T04:34:42.353 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.353 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.353 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.353 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-06T04:34:42.353 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.353 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.353 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.354 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-06-06T04:34:42.354 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-06-06T04:34:42.354 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.354 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.354 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.354 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-06-06T04:34:42.354 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.355 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.355 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.355 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-06T04:34:42.355 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.355 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.355 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.355 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-06-06T04:34:42.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-06-06T04:34:42.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.356 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.356 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.356 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-06-06T04:34:42.356 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.356 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.356 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.356 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-06T04:34:42.357 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.357 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.357 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.357 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-06-06T04:34:42.357 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-06-06T04:34:42.357 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.357 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.358 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.358 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.358 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.358 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.358 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.358 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-06T04:34:42.358 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-06T04:34:42.358 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.359 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.359 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-06-06T04:34:42.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-06-06T04:34:42.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.359 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.359 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.359 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-06-06T04:34:42.360 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.360 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.360 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.360 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-06T04:34:42.360 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.360 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.361 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-06-06T04:34:42.361 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-06-06T04:34:42.361 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:42.361 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.361 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.361 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-06-06T04:34:42.361 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.361 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.362 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.362 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-06T04:34:42.362 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.362 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.362 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.362 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-06-06T04:34:42.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-06-06T04:34:42.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.363 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.363 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.363 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.363 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.363 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.363 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.363 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-06-06T04:34:42.364 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-06-06T04:34:42.364 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-06-06T04:34:42.364 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.364 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-06T04:34:42.364 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.364 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.364 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.364 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-06-06T04:34:42.365 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-06-06T04:34:42.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.365 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.365 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.365 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.365 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.365 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.365 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.366 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-06T04:34:42.366 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.366 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.366 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.366 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-06-06T04:34:42.366 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-06-06T04:34:42.366 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.367 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.367 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.367 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.367 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.367 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.367 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.367 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-06T04:34:42.367 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.368 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.368 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.368 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-06-06T04:34:42.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-06-06T04:34:42.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.368 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.369 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.369 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.369 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.369 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.369 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.369 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-06T04:34:42.369 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.369 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.370 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.370 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-06-06T04:34:42.370 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-06-06T04:34:42.370 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.370 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.370 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.370 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-06T04:34:42.371 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.371 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.371 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.371 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-06-06T04:34:42.371 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.371 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.371 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.372 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-06-06T04:34:42.372 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-06-06T04:34:42.372 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.372 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.372 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.372 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-06T04:34:42.372 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.372 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.373 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.373 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-06-06T04:34:42.373 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.373 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.373 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.373 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-06-06T04:34:42.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-06-06T04:34:42.374 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.374 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.374 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.374 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-06T04:34:42.374 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.374 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.374 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.374 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-06-06T04:34:42.375 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.375 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.375 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.375 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-06-06T04:34:42.375 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-06-06T04:34:42.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.375 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.376 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.376 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-06T04:34:42.376 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.376 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.376 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.376 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-06-06T04:34:42.376 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.376 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.377 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.377 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-06-06T04:34:42.377 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-06-06T04:34:42.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.377 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.377 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-06T04:34:42.378 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.378 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.378 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.378 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-06-06T04:34:42.378 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.378 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.378 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.379 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-06-06T04:34:42.379 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-06-06T04:34:42.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.379 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.379 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.379 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-06T04:34:42.379 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.379 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.380 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.380 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-06-06T04:34:42.380 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.380 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.380 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.380 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-06-06T04:34:42.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-06-06T04:34:42.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.381 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.381 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.381 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-06T04:34:42.381 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.381 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.381 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.381 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-06-06T04:34:42.382 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.382 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.382 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.382 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-06-06T04:34:42.382 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-06-06T04:34:42.382 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.382 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.383 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.383 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-06T04:34:42.383 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.383 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.383 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.383 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-06-06T04:34:42.383 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.383 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.384 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.384 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-06-06T04:34:42.384 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-06-06T04:34:42.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.384 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.384 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-06T04:34:42.385 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.385 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.385 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.385 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-06-06T04:34:42.385 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.385 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.385 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-06-06T04:34:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-06-06T04:34:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-06T04:34:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.387 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.387 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-06-06T04:34:42.387 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.387 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.387 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.387 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-06-06T04:34:42.387 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-06-06T04:34:42.388 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.388 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.388 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.388 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-06T04:34:42.388 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.388 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.388 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.388 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-06-06T04:34:42.389 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.389 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.389 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.389 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-06-06T04:34:42.389 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-06-06T04:34:42.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.389 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.390 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.390 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-06T04:34:42.390 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.390 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.390 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.390 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-06-06T04:34:42.390 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.390 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.391 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.391 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-06-06T04:34:42.391 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-06-06T04:34:42.391 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.391 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.391 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.391 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-06T04:34:42.391 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.392 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.392 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.392 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-06-06T04:34:42.392 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.392 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.392 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.392 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-06-06T04:34:42.393 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-06-06T04:34:42.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.393 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.393 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.393 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-06T04:34:42.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.393 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.393 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.394 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-06-06T04:34:42.394 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.394 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.394 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.394 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-06-06T04:34:42.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-06-06T04:34:42.394 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.395 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.395 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-06T04:34:42.395 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.395 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.395 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.395 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-06-06T04:34:42.395 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.396 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.396 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.396 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-06-06T04:34:42.396 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-06-06T04:34:42.396 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.396 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.396 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.397 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-06T04:34:42.397 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.397 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.397 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.397 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-06-06T04:34:42.397 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.397 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.397 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.398 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-06-06T04:34:42.398 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-06-06T04:34:42.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.398 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.398 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.398 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.399 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.399 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.399 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.399 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-06-06T04:34:42.399 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.399 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.399 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.400 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-06-06T04:34:42.400 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-06-06T04:34:42.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.400 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.400 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.400 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-06-06T04:34:42.400 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.400 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-06T04:34:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-06-06T04:34:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-06-06T04:34:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-06-06T04:34:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-06T04:34:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-06-06T04:34:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-06-06T04:34:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-06-06T04:34:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-06T04:34:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-06-06T04:34:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-06-06T04:34:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-06-06T04:34:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-06-06T04:34:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-06-06T04:34:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-06-06T04:34:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-06T04:34:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-06-06T04:34:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-06-06T04:34:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-06-06T04:34:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-06-06T04:34:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-06-06T04:34:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-06-06T04:34:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.410 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-06T04:34:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-06T04:34:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.411 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-06-06T04:34:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-06-06T04:34:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-06-06T04:34:42.412 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.413 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.413 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.413 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-06-06T04:34:42.413 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.413 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.413 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.413 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-06-06T04:34:42.413 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-06-06T04:34:42.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.414 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.414 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.414 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.414 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.414 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.414 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.415 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-06T04:34:42.415 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.415 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.415 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.415 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-06-06T04:34:42.415 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-06-06T04:34:42.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.415 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.416 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.416 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.416 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.416 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.416 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.416 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-06T04:34:42.416 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.416 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.417 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.417 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.417 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.417 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.417 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.417 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.417 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.418 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.418 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.418 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.418 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.418 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.418 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.418 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.418 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.419 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.419 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.419 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.419 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.419 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.419 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.419 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-06-06T04:34:42.420 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-06-06T04:34:42.420 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.420 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.420 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.420 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.420 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.420 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.420 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.420 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-06T04:34:42.421 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.421 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.421 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.421 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.421 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.421 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.421 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.422 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.422 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.422 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.422 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.422 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-06T04:34:42.422 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.422 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.423 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.423 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.423 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.423 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.423 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.423 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.423 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.424 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.424 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.424 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.424 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.424 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.424 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.424 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.424 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.425 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.425 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.425 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.425 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.425 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.425 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.425 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.426 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.426 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.426 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.426 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.426 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.426 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.426 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.427 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.427 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.427 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.427 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.427 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-06T04:34:42.427 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-06T04:34:42.427 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.428 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.428 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.428 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.428 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.428 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.428 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.428 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-06T04:34:42.428 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.429 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.429 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.429 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-06T04:34:42.429 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-06T04:34:42.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.429 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.429 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.430 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-06-06T04:34:42.430 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.430 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.430 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.430 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-06T04:34:42.430 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.430 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.430 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.431 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-06-06T04:34:42.431 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-06-06T04:34:42.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.431 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.431 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.431 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:42.431 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.432 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.432 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.432 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-06-06T04:34:42.432 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.432 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.432 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.432 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-06-06T04:34:42.433 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-06-06T04:34:42.433 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.433 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.433 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.433 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:42.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.433 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.433 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.434 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-06T04:34:42.434 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.434 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.434 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-06-06T04:34:42.434 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-06-06T04:34:42.434 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.435 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.435 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.435 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.435 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.435 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.435 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.435 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-06-06T04:34:42.435 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.436 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.436 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-06-06T04:34:42.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-06-06T04:34:42.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.436 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.436 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.437 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-06-06T04:34:42.437 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.437 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.437 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.437 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-06-06T04:34:42.437 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-06-06T04:34:42.437 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.437 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-06-06T04:34:42.437 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.438 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.438 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.438 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-06-06T04:34:42.438 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-06-06T04:34:42.438 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.438 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.438 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.439 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-06-06T04:34:42.439 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.439 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.439 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.439 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-06T04:34:42.439 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.439 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.439 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.440 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-06-06T04:34:42.440 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-06-06T04:34:42.440 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.440 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.440 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.440 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-06T04:34:42.440 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.441 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.441 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.441 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-06T04:34:42.441 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.441 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.441 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.441 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-06-06T04:34:42.442 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-06-06T04:34:42.442 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.442 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.442 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.442 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.442 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.442 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.442 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.443 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-06T04:34:42.443 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.443 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.443 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.443 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-06-06T04:34:42.443 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-06-06T04:34:42.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.443 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.444 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.444 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.444 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.444 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.444 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.444 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-06T04:34:42.444 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.444 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.445 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.445 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-06-06T04:34:42.445 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-06-06T04:34:42.445 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.445 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.445 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.445 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.446 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.446 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.446 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.446 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-06T04:34:42.446 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.446 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.446 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.447 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-06-06T04:34:42.447 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-06-06T04:34:42.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.447 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.447 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.447 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.447 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.447 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.448 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.448 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-06T04:34:42.448 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.448 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.448 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.448 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-06-06T04:34:42.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-06-06T04:34:42.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.449 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.449 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.449 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.449 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.449 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.449 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.449 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-06T04:34:42.450 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.450 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.450 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.450 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-06-06T04:34:42.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-06-06T04:34:42.450 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.450 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.451 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.451 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-06-06T04:34:42.451 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.451 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.451 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.451 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-06-06T04:34:42.451 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.451 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.452 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.452 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-06-06T04:34:42.452 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-06-06T04:34:42.452 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.452 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.452 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.452 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.452 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.453 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.453 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.453 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-06T04:34:42.453 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.453 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.453 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.453 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-06-06T04:34:42.454 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-06-06T04:34:42.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.454 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.454 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.454 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.454 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.454 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.454 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.455 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-06T04:34:42.455 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.455 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.455 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.455 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-06-06T04:34:42.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-06-06T04:34:42.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.456 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.456 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.456 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.456 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.456 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.456 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.456 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-06T04:34:42.456 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.457 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.457 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.457 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-06-06T04:34:42.457 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-06-06T04:34:42.457 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.457 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.457 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.458 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.458 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.458 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.458 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.458 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-06-06T04:34:42.458 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.458 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.459 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-06-06T04:34:42.459 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-06-06T04:34:42.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.459 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.459 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.459 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.459 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.460 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.460 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.460 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-06T04:34:42.460 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.460 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.460 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.460 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-06-06T04:34:42.461 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-06-06T04:34:42.461 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.461 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.461 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.461 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.461 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.461 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.461 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.461 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-06T04:34:42.462 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.462 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.462 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.462 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.462 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.463 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-06-06T04:34:42.463 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.463 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.463 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.463 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.463 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.463 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.463 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.463 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.464 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.464 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.464 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.464 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.464 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.464 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.464 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.464 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.465 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.465 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.465 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.465 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.465 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.465 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.465 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.466 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.466 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.466 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.466 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.466 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.466 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.466 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.466 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.467 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.467 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.467 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.467 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.467 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.467 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.467 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.468 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.468 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.468 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.468 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.468 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.468 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.468 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.468 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.468 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.469 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.469 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.469 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.469 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.469 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.469 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.469 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.469 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.470 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.470 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.470 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.470 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.470 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.470 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.471 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.471 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.471 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.471 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.471 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.471 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.471 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.472 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-06-06T04:34:42.472 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-06-06T04:34:42.472 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.472 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.472 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.472 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-06-06T04:34:42.472 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.472 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.472 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.473 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-06T04:34:42.473 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.473 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.473 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.473 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-06-06T04:34:42.473 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-06-06T04:34:42.473 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.474 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.474 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.474 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-06-06T04:34:42.474 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.474 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.474 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.474 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-06T04:34:42.474 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.475 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.475 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.475 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.475 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.475 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-06-06T04:34:42.475 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.476 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.476 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.476 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-06T04:34:42.476 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-06T04:34:42.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.476 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.476 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.476 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:42.477 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.477 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.477 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.477 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.477 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.477 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.477 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.478 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-06-06T04:34:42.478 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-06T04:34:42.478 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.478 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.478 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.478 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.478 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.478 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.479 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.479 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.479 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.479 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.479 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.479 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-06T04:34:42.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-06T04:34:42.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.480 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.480 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.480 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.480 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.480 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.480 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.480 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.481 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.481 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.481 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.481 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-06-06T04:34:42.481 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-06-06T04:34:42.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.481 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.482 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.482 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.482 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.482 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.482 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.482 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.482 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.482 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.483 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-06T04:34:42.483 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-06T04:34:42.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.483 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.483 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.483 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.483 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.484 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.484 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.484 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.484 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.484 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.484 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.484 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-06-06T04:34:42.485 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-06T04:34:42.485 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.485 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.485 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.485 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.485 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.485 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.485 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.486 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.486 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.486 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.486 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.486 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-06T04:34:42.486 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-06T04:34:42.486 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.487 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.487 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.487 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.487 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.487 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.487 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.487 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.487 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.487 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.488 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.488 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-06T04:34:42.488 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-06T04:34:42.488 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.488 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.488 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.488 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.489 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.489 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.489 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.489 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.489 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.489 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.489 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.490 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-06T04:34:42.490 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-06T04:34:42.490 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.490 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.490 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.490 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.490 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.490 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.491 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.491 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.491 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.491 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.491 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.491 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-06T04:34:42.491 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-06T04:34:42.492 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.492 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.492 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.492 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.492 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.492 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.492 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.492 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.493 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.493 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.493 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.493 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-06T04:34:42.493 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-06T04:34:42.493 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.493 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.493 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.494 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.494 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.494 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.494 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.494 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.494 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.494 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.494 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.495 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-06T04:34:42.495 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-06T04:34:42.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.495 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.495 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.495 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.495 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.496 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.496 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.496 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.496 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.496 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.496 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.496 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-06T04:34:42.497 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-06T04:34:42.497 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.497 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.497 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.497 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.497 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.497 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.497 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.497 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.498 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.498 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.498 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.498 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-06T04:34:42.498 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-06T04:34:42.498 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.498 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.499 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.499 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.499 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.499 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.499 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.499 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.499 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.499 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.500 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.500 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-06-06T04:34:42.500 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-06T04:34:42.500 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.500 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.500 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.500 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-06-06T04:34:42.501 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.501 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.501 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.501 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.501 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.501 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.501 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.502 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-06T04:34:42.502 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-06T04:34:42.502 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.502 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.502 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.502 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.502 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.502 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.503 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.503 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.503 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.503 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.503 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.503 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-06-06T04:34:42.503 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-06-06T04:34:42.503 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.504 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.504 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.504 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.504 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.504 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.504 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.504 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.505 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.505 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.505 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.505 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-06-06T04:34:42.505 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-06T04:34:42.505 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.505 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.506 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.506 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-06-06T04:34:42.506 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.506 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.506 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.506 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.506 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.506 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.507 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.507 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-06T04:34:42.507 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-06T04:34:42.507 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.507 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.507 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.507 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.508 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.508 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.508 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.508 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.508 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.508 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.508 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.509 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-06T04:34:42.509 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-06T04:34:42.509 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.509 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.509 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.509 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.509 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.509 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.509 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.510 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.510 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.510 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.510 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.510 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-06T04:34:42.510 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-06T04:34:42.510 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.511 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.511 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.511 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.511 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.511 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.511 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.511 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.511 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.512 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.512 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.512 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-06T04:34:42.512 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-06T04:34:42.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.512 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.512 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.513 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.513 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.513 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.513 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.513 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.513 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.513 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.513 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.514 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-06-06T04:34:42.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-06T04:34:42.514 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.514 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.514 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.514 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.514 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.515 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.515 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.515 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.515 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.515 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.515 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.515 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-06T04:34:42.516 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-06T04:34:42.516 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.516 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.516 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.516 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.516 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.516 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.516 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.516 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.517 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.517 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.517 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.517 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-06T04:34:42.517 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-06T04:34:42.517 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.517 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.518 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.518 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.518 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.518 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.518 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.518 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.518 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.518 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.519 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.519 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-06T04:34:42.519 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-06T04:34:42.519 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.519 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.519 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.519 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.520 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.520 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.520 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.520 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.520 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.520 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.520 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.521 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-06T04:34:42.521 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-06T04:34:42.521 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.521 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.521 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.521 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.521 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.521 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.522 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.522 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.522 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.522 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.522 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.522 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-06T04:34:42.522 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-06T04:34:42.522 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.523 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.523 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.523 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.523 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.523 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.523 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.523 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.524 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.524 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.524 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.524 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-06T04:34:42.524 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-06T04:34:42.524 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.524 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.525 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.525 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.525 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.525 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.525 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.525 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.525 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.525 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.525 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.526 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-06T04:34:42.526 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-06T04:34:42.526 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.526 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.526 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.526 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.527 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.527 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.527 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.527 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.527 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.527 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.527 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.528 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-06T04:34:42.528 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-06T04:34:42.528 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.528 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.528 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.528 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.528 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.528 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.528 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.529 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.529 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.529 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.529 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.529 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-06-06T04:34:42.529 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-06T04:34:42.529 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.530 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.530 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.530 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.530 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.530 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.530 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.530 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.530 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.531 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.531 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.531 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-06T04:34:42.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-06T04:34:42.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.531 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.532 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-06T04:34:42.532 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.532 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.532 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.532 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.532 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.532 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.532 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.533 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-06T04:34:42.533 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-06T04:34:42.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.533 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.533 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.533 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:42.533 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.534 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.534 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.534 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.534 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.534 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.534 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.534 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-06T04:34:42.535 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-06T04:34:42.535 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.535 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.535 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.535 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-06-06T04:34:42.535 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.535 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.535 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.536 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.536 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.536 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.536 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.536 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-06T04:34:42.536 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-06T04:34:42.536 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.537 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.537 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.537 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-06-06T04:34:42.537 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.537 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.537 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.537 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.537 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.537 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.538 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.538 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-06-06T04:34:42.538 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-06T04:34:42.538 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.538 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.538 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.538 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.539 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.539 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.539 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.539 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.539 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.539 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.539 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.540 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-06T04:34:42.540 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-06T04:34:42.540 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.540 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.540 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.540 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-06-06T04:34:42.540 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.540 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.541 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.541 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.541 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.541 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.541 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.541 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-06-06T04:34:42.541 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-06T04:34:42.542 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.542 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.542 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.542 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.542 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.546 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.546 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.546 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.546 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.546 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.546 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.547 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-06-06T04:34:42.547 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-06T04:34:42.547 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.547 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.547 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.547 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.547 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.547 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.548 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.548 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.548 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.548 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.548 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.548 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-06T04:34:42.548 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-06T04:34:42.549 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.549 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.549 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.549 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-06-06T04:34:42.549 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.549 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.549 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.549 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.550 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.550 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.550 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.550 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-06T04:34:42.550 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-06T04:34:42.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.550 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.551 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.551 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:42.551 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.551 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.551 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.551 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.551 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.551 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.552 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.552 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-06T04:34:42.552 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-06T04:34:42.552 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.552 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.552 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.552 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-06-06T04:34:42.553 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.553 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.553 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.553 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.553 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.553 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.553 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.554 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-06-06T04:34:42.554 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-06-06T04:34:42.554 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.554 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.554 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.554 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.554 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.554 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.555 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.555 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.555 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.555 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.555 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.555 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-06-06T04:34:42.555 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-06-06T04:34:42.555 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.556 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.556 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.556 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.556 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.556 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.556 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.556 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.556 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.557 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.557 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.557 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-06-06T04:34:42.557 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-06-06T04:34:42.557 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.557 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.557 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.558 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.558 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.558 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.558 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.558 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-06-06T04:34:42.558 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.558 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.559 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.559 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-06-06T04:34:42.559 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-06-06T04:34:42.559 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.559 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.559 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.559 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-06-06T04:34:42.560 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.560 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.560 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.560 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.560 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.560 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.560 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.560 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-06-06T04:34:42.561 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-06-06T04:34:42.561 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.561 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.561 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.561 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.561 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.561 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.561 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.562 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.562 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.562 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.562 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.562 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.562 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.562 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.563 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.563 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.563 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.563 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.563 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.563 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.563 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.563 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.564 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.564 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.564 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.564 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.564 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.564 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.564 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.564 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.565 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.565 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.565 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.565 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.565 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.565 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.565 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.566 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.566 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.566 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.566 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.566 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.566 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.566 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.567 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.567 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.567 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.567 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.567 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.567 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.567 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.567 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.568 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.568 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.568 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.568 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.568 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.568 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.568 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.568 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.568 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.569 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.569 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.569 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.569 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.569 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.569 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.569 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.570 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.570 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.570 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.570 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.570 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.570 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.570 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.571 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.571 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.571 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-06-06T04:34:42.571 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-06-06T04:34:42.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.571 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.572 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.572 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.572 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.572 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.572 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-06-06T04:34:42.572 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.572 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.573 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-06-06T04:34:42.573 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-06-06T04:34:42.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.573 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.573 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.573 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.574 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.574 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.574 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.574 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.574 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.574 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-06-06T04:34:42.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-06-06T04:34:42.575 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.575 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.575 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.575 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-06-06T04:34:42.575 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.575 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.575 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.575 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.576 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.576 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.576 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.576 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-06-06T04:34:42.576 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-06-06T04:34:42.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.576 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.577 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.577 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-06-06T04:34:42.577 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.577 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.577 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.577 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.577 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.577 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.577 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.578 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-06-06T04:34:42.578 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-06-06T04:34:42.578 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.578 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.578 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.578 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.578 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.579 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.579 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.579 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.579 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.579 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.579 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.579 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-06-06T04:34:42.580 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-06-06T04:34:42.580 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.580 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.580 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.580 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-06-06T04:34:42.580 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.580 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.580 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.581 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.581 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.581 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.581 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.581 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-06-06T04:34:42.581 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-06-06T04:34:42.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.582 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.582 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.582 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-06-06T04:34:42.582 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.582 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.582 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.582 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.582 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-06-06T04:34:42.583 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.583 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.583 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.583 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.583 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.583 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-06-06T04:34:42.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-06-06T04:34:42.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.584 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.584 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.584 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-06-06T04:34:42.584 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-06-06T04:34:42.584 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-06-06T04:34:42.584 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.584 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.585 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.585 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.585 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.585 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-06-06T04:34:42.585 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-06-06T04:34:42.585 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.585 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.586 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.586 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.586 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.586 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.586 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.586 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.586 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.586 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.587 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.587 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.587 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.587 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-06-06T04:34:42.587 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.587 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.587 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.588 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-06-06T04:34:42.588 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-06-06T04:34:42.588 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.588 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.588 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.588 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.588 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.588 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.589 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.589 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-06-06T04:34:42.589 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.589 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.589 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.589 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.589 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.590 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.590 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.590 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.590 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.590 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.590 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.590 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.591 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.591 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.591 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.591 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.591 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.591 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.591 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.591 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.592 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.592 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.592 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.592 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.592 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.592 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.592 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.593 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.593 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.593 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.593 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.593 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.593 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.593 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.593 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.594 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.594 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.594 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.594 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.594 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.594 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.594 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.595 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.595 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.595 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.595 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.595 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.595 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.595 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.595 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.596 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.596 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.596 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.596 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.596 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.596 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.596 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.597 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.597 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.597 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.597 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.597 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.597 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.597 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.597 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.597 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.598 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.598 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.598 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.598 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-06-06T04:34:42.598 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-06-06T04:34:42.598 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.599 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.599 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.599 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-06-06T04:34:42.599 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.599 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.599 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.599 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-06T04:34:42.599 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.600 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.600 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.600 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-06-06T04:34:42.600 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-06-06T04:34:42.600 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.600 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.600 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.600 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-06-06T04:34:42.601 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.601 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.601 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.601 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-06T04:34:42.601 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.601 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.601 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.602 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-06-06T04:34:42.602 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-06-06T04:34:42.602 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.602 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.602 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.602 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-06-06T04:34:42.602 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.602 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.603 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.603 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-06-06T04:34:42.603 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.603 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.603 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.603 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-06-06T04:34:42.603 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-06-06T04:34:42.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.604 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.604 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.604 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-06-06T04:34:42.604 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.604 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.604 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.604 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-06T04:34:42.605 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.605 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.605 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.605 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-06-06T04:34:42.605 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-06-06T04:34:42.605 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.605 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.606 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.606 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.606 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.606 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.606 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.606 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-06T04:34:42.606 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.606 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.607 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.607 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-06-06T04:34:42.607 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-06-06T04:34:42.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.607 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.607 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.607 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-06-06T04:34:42.608 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.608 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.608 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.608 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-06-06T04:34:42.608 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.608 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.608 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.608 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-06-06T04:34:42.609 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-06-06T04:34:42.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.609 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.609 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.609 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-06-06T04:34:42.609 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.609 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.609 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.610 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-06T04:34:42.610 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.610 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.610 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.610 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.610 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.611 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-06-06T04:34:42.611 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.611 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.611 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.611 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.611 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.611 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.611 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.612 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.612 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.612 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.612 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.612 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.612 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.612 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.612 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.613 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.613 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.613 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.613 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.613 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.613 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.613 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.613 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.614 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.614 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.614 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.614 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.614 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.614 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.614 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.615 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.615 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.615 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.615 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.615 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.615 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.615 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.616 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.616 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.616 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.616 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.616 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.616 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.616 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.616 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.616 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.617 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.617 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.617 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.617 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.617 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.617 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.617 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.617 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.618 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.618 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.618 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.618 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.618 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.618 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.618 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.619 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.619 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.619 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.619 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.619 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.619 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.619 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.619 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.620 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-06-06T04:34:42.620 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-06-06T04:34:42.620 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.620 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.620 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.620 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-06-06T04:34:42.621 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.621 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.621 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.621 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.621 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.621 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.621 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.622 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-06-06T04:34:42.622 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-06-06T04:34:42.622 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.622 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.622 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.622 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-06-06T04:34:42.622 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.622 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.623 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.623 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.623 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.623 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.623 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.623 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-06-06T04:34:42.623 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-06-06T04:34:42.624 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.624 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.624 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.624 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-06-06T04:34:42.624 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.624 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.624 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.624 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.625 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.625 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.625 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.625 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.625 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.625 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-06-06T04:34:42.626 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-06-06T04:34:42.626 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-06-06T04:34:42.626 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.626 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-06-06T04:34:42.626 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-06-06T04:34:42.626 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.626 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.626 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.627 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-06-06T04:34:42.627 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.627 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.627 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.627 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-06T04:34:42.627 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.627 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.627 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.628 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-06-06T04:34:42.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-06-06T04:34:42.628 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.628 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.628 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.628 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-06-06T04:34:42.628 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-06T04:34:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.629 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-06-06T04:34:42.630 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-06-06T04:34:42.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.630 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.630 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.630 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.630 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.630 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.630 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.631 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-06-06T04:34:42.631 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.631 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.631 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.631 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-06-06T04:34:42.631 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-06-06T04:34:42.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.631 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.632 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.632 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-06-06T04:34:42.632 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-06-06T04:34:42.632 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.632 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.632 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-06T04:34:42.632 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.632 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.633 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.633 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.633 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.633 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.633 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.633 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.633 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.634 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.634 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.634 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.634 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.634 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.634 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.634 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.634 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.635 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.635 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.635 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.635 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.635 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.635 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.635 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.636 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.636 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.636 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.636 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.636 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.636 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.636 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.636 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.637 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.637 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.637 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.637 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.637 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.637 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.637 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.638 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.638 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.638 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.638 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.638 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.638 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.638 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.638 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.639 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.639 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.639 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.639 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.639 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.639 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.639 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.639 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.639 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.640 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.640 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.640 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.640 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.640 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.640 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.641 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.641 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.641 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.641 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.641 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.641 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.641 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-06-06T04:34:42.642 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-06-06T04:34:42.642 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.642 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.642 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.642 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.642 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.642 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.642 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.643 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-06-06T04:34:42.643 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.643 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.643 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.643 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-06-06T04:34:42.643 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-06-06T04:34:42.643 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.644 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.644 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.644 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-06-06T04:34:42.644 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.644 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.644 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.644 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-06-06T04:34:42.644 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.645 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.645 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.645 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-06-06T04:34:42.645 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-06-06T04:34:42.645 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.645 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.645 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.646 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-06-06T04:34:42.646 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.646 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.646 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.646 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T04:34:42.646 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.646 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.646 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.647 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-06-06T04:34:42.647 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-06-06T04:34:42.647 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.647 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.647 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.647 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-06-06T04:34:42.647 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-06-06T04:34:42.648 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-06-06T04:34:42.648 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.648 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-06T04:34:42.648 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.648 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.648 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.648 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-06-06T04:34:42.649 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-06-06T04:34:42.649 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.649 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.649 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.649 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.649 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.649 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.649 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.650 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-06-06T04:34:42.650 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.650 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.650 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.650 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-06-06T04:34:42.650 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-06-06T04:34:42.650 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.651 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.651 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.651 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-06-06T04:34:42.651 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.651 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.651 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.651 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T04:34:42.651 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.652 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.652 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.652 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.652 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.652 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-06-06T04:34:42.652 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.653 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.653 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.653 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.653 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.653 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.653 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.653 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.653 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.653 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.654 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.654 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.654 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.654 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.654 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.654 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.654 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.655 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.655 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.655 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.655 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.655 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.655 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.655 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.655 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.656 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.656 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.656 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.656 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.656 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.656 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.656 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.656 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.657 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.657 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.657 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.657 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.657 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.657 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.657 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.658 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.658 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.658 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.658 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.658 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.658 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.658 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.658 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.659 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.659 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.659 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.659 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.659 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.659 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.659 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.659 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.660 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.660 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.660 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.660 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.660 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.660 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.660 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.661 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.661 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.661 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.661 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.661 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.661 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.661 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.661 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.662 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-06-06T04:34:42.662 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.662 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.662 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.662 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.662 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.663 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.663 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.663 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.663 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.663 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.663 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.663 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.663 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.664 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.664 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.664 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.664 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.664 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.664 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.664 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.664 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.664 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.665 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.665 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.665 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.665 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.665 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.665 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.666 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.666 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.666 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.666 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.666 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.666 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.666 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.667 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.667 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.667 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.667 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.667 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.667 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.667 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.668 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.668 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.668 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.668 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.668 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.668 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.668 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.668 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.669 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.669 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.669 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.669 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.669 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.669 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.669 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.670 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.670 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.670 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.670 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.670 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.670 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.670 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.670 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.671 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.671 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.671 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.671 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-06-06T04:34:42.671 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.671 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.672 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.672 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-06-06T04:34:42.672 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-06-06T04:34:42.672 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.672 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.672 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.672 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-06-06T04:34:42.672 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.673 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.673 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.673 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-06T04:34:42.673 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.673 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.673 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.673 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-06-06T04:34:42.674 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-06-06T04:34:42.674 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.674 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.674 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.674 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:42.674 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-06-06T04:34:42.674 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.674 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.675 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-06T04:34:42.675 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.675 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.675 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.675 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.675 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.676 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.676 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.676 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.676 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.676 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.676 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.676 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.676 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.677 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.677 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.677 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.677 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.677 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.677 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.677 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.677 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.678 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.678 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.678 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.678 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.678 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.678 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.678 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.679 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.679 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.679 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.679 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.679 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.679 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.679 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.680 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.680 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.680 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.680 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.680 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.680 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.680 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.681 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.681 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.681 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.681 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.681 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.681 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.681 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.681 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.682 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.682 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.682 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.682 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.682 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.682 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.682 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.683 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.683 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.683 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.683 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.683 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.683 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.683 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.683 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.684 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-06-06T04:34:42.684 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.684 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.685 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.685 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-06-06T04:34:42.685 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-06-06T04:34:42.685 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.685 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.685 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.685 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-06-06T04:34:42.685 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.686 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.686 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.686 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-06-06T04:34:42.686 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-06T04:34:42.686 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.686 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.686 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.687 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.687 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.687 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.687 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.687 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.687 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.687 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.687 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.688 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.688 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.688 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.688 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.688 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.688 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.688 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.688 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.689 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.689 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.689 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.689 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.689 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.689 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.689 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.689 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.690 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.690 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.690 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.690 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.690 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.690 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.690 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.690 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.691 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.691 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.691 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.691 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.691 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.691 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.691 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.692 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.692 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.692 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.692 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.692 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.692 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.692 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.692 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.693 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.693 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.693 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.693 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.693 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.693 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.693 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.694 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.694 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.694 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.694 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.694 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.694 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.694 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.694 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.694 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.695 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.695 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.695 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-06-06T04:34:42.695 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-06-06T04:34:42.695 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.695 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.695 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.696 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.696 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.696 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.696 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.696 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-06-06T04:34:42.696 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-06T04:34:42.696 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.696 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.697 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-06-06T04:34:42.697 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-06-06T04:34:42.697 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.697 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.697 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.697 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-06-06T04:34:42.697 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.698 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.698 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.698 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-06-06T04:34:42.698 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.698 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.698 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.698 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-06-06T04:34:42.699 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-06-06T04:34:42.699 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.699 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.699 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.699 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-06-06T04:34:42.699 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.699 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.699 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.700 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-06-06T04:34:42.700 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.700 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.700 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.700 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-06-06T04:34:42.700 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-06-06T04:34:42.700 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.700 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.701 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.701 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.701 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.701 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.701 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.701 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-06-06T04:34:42.701 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.701 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.702 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.702 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-06-06T04:34:42.702 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-06-06T04:34:42.702 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.702 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.702 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.702 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-06-06T04:34:42.703 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.703 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.703 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.703 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-06T04:34:42.703 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-06T04:34:42.703 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.703 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.703 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.704 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.704 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.704 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-06-06T04:34:42.704 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.704 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.704 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.704 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.705 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.705 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.705 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.705 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.705 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.705 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.705 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.705 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.706 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.706 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.706 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.706 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.706 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.706 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.706 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.706 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.707 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.707 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.707 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.707 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.707 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.707 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.707 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.708 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.708 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.708 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.708 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.708 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.708 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.708 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.708 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.709 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.709 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.709 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.709 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.709 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.709 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.709 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.710 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.710 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.710 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.710 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.710 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.710 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.710 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.710 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.711 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.711 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.711 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.711 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.711 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.711 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.711 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.712 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.712 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.712 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.712 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.712 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.712 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.712 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.712 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.712 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.713 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.713 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.713 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.713 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.713 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.713 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-06-06T04:34:42.714 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.714 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.714 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.714 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.714 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.714 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.714 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.715 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.715 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.715 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.715 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.715 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.715 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.715 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.715 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.715 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.716 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.716 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.716 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.716 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.716 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.716 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.716 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.717 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.717 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.717 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.717 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.717 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.717 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.717 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.717 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.718 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.718 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.718 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.718 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.718 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.718 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.718 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.719 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.719 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.719 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.719 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.719 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.719 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.719 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.719 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.720 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.720 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.720 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.720 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.720 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.720 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.720 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.720 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.721 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.721 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.721 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.721 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.721 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.721 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.721 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.722 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.722 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.722 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.722 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.722 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.722 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.722 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.722 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.723 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.723 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.723 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-06-06T04:34:42.723 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.723 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.723 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.723 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.724 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.724 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.724 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.724 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.724 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.724 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.724 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.724 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.725 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.725 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.725 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.725 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.725 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.725 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.725 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.725 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.726 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.726 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.726 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.726 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.726 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.726 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.726 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.727 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.727 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.727 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.727 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.727 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.727 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.727 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.727 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.728 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.728 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.728 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.728 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.728 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.728 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.728 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.729 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.729 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.729 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.729 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.729 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.729 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.729 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.729 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.730 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.730 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.730 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.730 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.730 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.730 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.730 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.731 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.731 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.731 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.731 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.731 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.731 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.731 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.731 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.732 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.732 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.732 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.732 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.732 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.732 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.733 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-06-06T04:34:42.733 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.733 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.733 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.733 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-06-06T04:34:42.733 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-06-06T04:34:42.733 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.733 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.733 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.734 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.734 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.734 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.734 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.734 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-06T04:34:42.734 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.734 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.734 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.735 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.735 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.735 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.735 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.735 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.735 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.735 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.736 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.736 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.736 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.736 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.736 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.736 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.736 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.736 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.737 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.737 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.737 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.737 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.737 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.737 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.737 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.738 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.738 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.738 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.738 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.738 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.738 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.738 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.738 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.739 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.739 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.739 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.739 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.739 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.739 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.739 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.740 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.740 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.740 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.740 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.740 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.740 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.740 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.740 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.741 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.741 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.741 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.741 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.741 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.741 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.741 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.741 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.742 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.742 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.742 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.742 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.742 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.742 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.742 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.743 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.743 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.743 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.743 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.743 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.743 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.743 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-06-06T04:34:42.744 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-06-06T04:34:42.744 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.744 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.744 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.744 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.744 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.744 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.744 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.745 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-06-06T04:34:42.745 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-06-06T04:34:42.745 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-06-06T04:34:42.745 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.745 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-06-06T04:34:42.745 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.745 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.745 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.746 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.746 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.746 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-06-06T04:34:42.746 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.746 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.746 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.746 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.747 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.747 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.747 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.747 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.747 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.747 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.747 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.747 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.748 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.748 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.748 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.748 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.748 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.748 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.748 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.748 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.749 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.749 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.749 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.749 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.749 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.749 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.749 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.750 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.750 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.750 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.750 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.750 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.750 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.750 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.750 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.751 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.751 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.754 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.754 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.754 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.754 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.754 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.754 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.755 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.755 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.755 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.755 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.755 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.755 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.755 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.755 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.756 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.756 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.756 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.756 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.756 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.756 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.756 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.757 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.757 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.757 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.757 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.757 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.757 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.757 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.758 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.758 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.758 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.758 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.758 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.758 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-06-06T04:34:42.759 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.759 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.759 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.759 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.759 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.759 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.759 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.759 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.760 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.760 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.760 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.760 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.760 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.760 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.760 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.760 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.761 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.761 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.761 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.761 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.761 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.761 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.761 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.762 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.762 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.762 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.762 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.762 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.762 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.762 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.763 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.763 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.763 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.763 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.763 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.763 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.763 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.764 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.764 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.764 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.764 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.764 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.764 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.764 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.764 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.765 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.765 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.765 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.765 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.765 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.765 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.765 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.765 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.766 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.766 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.766 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.766 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.766 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.766 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.766 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.767 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.767 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.767 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.767 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.767 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.767 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.767 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.767 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.768 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.768 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.768 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.768 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-06-06T04:34:42.768 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.768 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.769 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.769 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.769 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.769 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.769 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.769 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.769 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.769 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.769 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.770 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.770 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.770 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.770 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.770 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.770 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.770 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.771 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.771 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.771 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.771 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.771 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.771 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.771 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.772 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.772 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.772 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.772 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.772 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.772 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.772 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.772 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.773 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.773 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.773 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.773 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.773 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.773 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.773 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.774 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.774 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.774 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.774 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.774 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.774 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.774 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.774 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.775 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.775 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.775 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.775 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.775 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.775 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.775 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.775 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.776 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.776 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.776 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.776 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.776 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.776 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.776 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.777 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.777 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.777 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.777 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.777 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.777 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.777 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-06-06T04:34:42.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-06-06T04:34:42.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.778 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.778 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.778 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-06-06T04:34:42.778 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.778 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.778 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.779 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.779 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.779 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.779 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.779 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-06-06T04:34:42.779 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-06-06T04:34:42.779 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:42.780 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.780 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.780 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-06-06T04:34:42.780 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-06-06T04:34:42.780 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.780 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.780 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-06-06T04:34:42.780 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-06T04:34:42.781 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.781 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.781 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.781 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.781 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-06-06T04:34:42.781 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.782 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.782 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.782 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-06-06T04:34:42.782 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-06-06T04:34:42.782 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.782 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.782 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.782 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.783 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.783 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.783 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.783 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-06T04:34:42.783 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.783 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.784 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-06-06T04:34:42.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-06-06T04:34:42.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.784 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.784 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.784 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.784 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.784 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.785 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.785 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.785 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.785 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.785 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.785 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.785 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.785 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.786 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.786 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.786 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.786 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.786 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.786 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.786 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.787 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.787 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.787 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.787 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.787 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.787 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.787 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.787 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.787 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.788 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.788 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.788 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.788 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.788 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.788 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.788 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.789 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.789 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.789 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.789 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.789 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.789 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.789 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.790 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.790 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.790 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.790 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.790 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.790 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.790 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.790 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.791 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.791 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.791 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.791 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.791 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.791 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.791 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.791 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.792 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.792 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.792 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.792 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.792 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.792 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.793 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.793 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.793 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.793 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.793 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.793 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.793 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.793 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.794 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.794 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-06-06T04:34:42.794 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-06-06T04:34:42.794 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.794 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.794 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.794 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-06-06T04:34:42.795 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.795 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.795 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.795 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-06-06T04:34:42.795 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.795 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.795 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.796 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-06-06T04:34:42.796 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-06-06T04:34:42.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.796 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.796 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.796 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-06-06T04:34:42.796 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.796 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.797 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.797 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-06-06T04:34:42.797 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.797 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.797 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.798 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-06-06T04:34:42.798 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.798 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.798 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.798 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-06-06T04:34:42.798 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-06-06T04:34:42.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.799 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.799 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.799 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.799 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.799 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.799 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.799 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.799 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.800 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.800 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.800 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-06-06T04:34:42.800 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-06-06T04:34:42.800 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.800 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.800 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.801 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.801 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.801 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.801 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.801 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-06T04:34:42.801 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-06T04:34:42.801 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.801 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.802 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.802 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.802 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.802 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.802 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.802 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.803 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.803 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.803 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.803 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.803 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.803 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.803 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.803 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.804 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.804 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.804 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.804 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.804 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.804 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.804 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.805 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.805 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.805 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.805 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.805 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.805 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.805 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.805 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.806 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.806 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.806 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.806 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.806 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.806 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.806 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.807 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.807 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.807 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.807 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.807 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.807 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.807 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.807 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.808 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.808 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.808 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.808 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.808 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.808 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.808 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.809 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.809 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.809 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.809 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.809 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.809 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.809 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.810 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.810 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.810 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.810 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.810 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.810 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.810 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-06-06T04:34:42.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-06-06T04:34:42.811 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.811 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.811 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.811 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.811 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.811 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.811 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.812 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.812 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.812 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.812 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-06T04:34:42.812 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-06T04:34:42.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.813 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.813 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-06-06T04:34:42.813 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.813 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.813 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.813 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.813 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.814 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.814 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-06-06T04:34:42.814 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-06-06T04:34:42.814 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:42.814 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.815 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-06-06T04:34:42.815 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.815 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.815 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.815 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.815 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.815 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.816 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.816 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-06-06T04:34:42.816 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-06-06T04:34:42.816 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.816 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.816 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.816 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-06-06T04:34:42.817 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.817 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.817 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.817 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-06T04:34:42.817 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.817 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.818 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-06-06T04:34:42.818 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-06-06T04:34:42.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.818 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.818 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.818 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-06-06T04:34:42.818 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.818 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.819 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.819 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-06T04:34:42.819 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.819 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.819 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-06-06T04:34:42.819 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-06-06T04:34:42.820 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.820 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.820 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.820 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-06-06T04:34:42.820 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.820 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.820 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.820 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.821 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.821 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.821 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.821 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-06-06T04:34:42.821 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-06-06T04:34:42.821 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.821 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.822 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-06-06T04:34:42.822 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.822 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.822 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.822 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-06-06T04:34:42.822 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-06-06T04:34:42.822 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.823 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.823 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.823 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.823 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.823 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-06-06T04:34:42.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-06-06T04:34:42.823 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.824 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.824 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.824 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-06-06T04:34:42.824 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-06-06T04:34:42.824 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-06-06T04:34:42.824 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.824 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.824 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.825 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.825 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.825 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-06-06T04:34:42.825 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.826 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.826 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.826 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.826 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.826 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.826 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.826 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.826 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.827 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.827 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.827 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.827 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.827 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.827 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.827 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.827 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.828 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.828 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.828 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.828 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.828 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.828 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.828 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.829 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.829 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.829 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.829 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.829 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.829 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.829 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.830 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.830 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.830 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.830 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.830 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.830 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.831 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.831 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.831 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.831 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.831 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.831 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.831 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.832 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.832 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.832 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.832 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.832 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.832 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.832 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.832 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.833 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.833 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.833 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.833 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.833 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.833 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.834 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.834 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.834 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.834 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.834 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.834 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.834 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.835 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-06-06T04:34:42.835 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-06-06T04:34:42.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.835 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.835 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.835 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-06-06T04:34:42.835 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.835 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.836 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.836 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.836 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.836 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.836 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.836 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-06-06T04:34:42.836 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-06-06T04:34:42.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.837 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.837 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.837 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.837 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.837 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.837 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.837 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.838 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.838 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.838 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.838 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-06-06T04:34:42.838 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-06-06T04:34:42.838 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.838 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.839 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.839 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.839 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.839 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.839 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.839 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.839 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.839 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.840 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.840 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.840 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-06-06T04:34:42.840 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.840 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.841 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.841 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-06-06T04:34:42.841 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-06-06T04:34:42.841 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.841 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.841 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.841 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.841 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.842 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.842 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.842 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.842 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.842 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.842 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.842 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-06-06T04:34:42.843 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-06-06T04:34:42.843 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.843 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.843 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.843 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.843 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.843 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.844 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.844 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.844 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.844 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.844 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.844 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.844 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.845 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.845 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.845 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.845 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.845 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.845 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.845 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.845 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.845 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.846 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.846 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.846 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.846 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.846 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.846 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.846 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.846 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.847 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.847 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.847 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.847 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.847 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.847 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.848 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.848 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.848 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.848 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.848 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.848 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.848 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.848 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.849 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.849 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.849 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.849 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.849 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.849 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.849 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.850 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.850 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.850 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.850 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.850 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.850 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.850 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.851 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.851 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.851 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.851 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.851 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.851 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.851 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.852 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.852 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.852 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.852 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.852 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.852 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.852 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.853 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.853 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.853 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.853 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-06-06T04:34:42.853 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-06-06T04:34:42.853 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.853 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.854 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.854 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.854 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.854 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.854 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.854 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.854 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.855 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.855 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-06-06T04:34:42.855 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-06-06T04:34:42.855 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.855 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.855 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.855 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.855 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.856 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.856 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.856 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.856 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.856 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.856 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.856 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.857 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.857 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-06-06T04:34:42.857 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.857 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.857 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.857 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.858 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.858 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.858 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.858 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.858 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.858 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.858 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.859 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.859 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.859 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.859 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.859 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.859 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.859 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.859 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.860 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.860 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.860 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.860 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.860 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.860 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.860 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.861 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.861 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.861 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.861 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.861 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.861 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.861 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.861 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.862 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.862 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.862 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.862 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.862 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.862 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.863 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.863 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.863 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.863 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.863 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.863 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.863 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.863 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.864 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.864 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.864 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.864 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.864 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.864 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.864 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.865 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.865 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.865 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.865 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.865 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.865 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.865 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.865 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.866 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.866 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.866 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.866 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.866 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.866 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.867 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-06-06T04:34:42.867 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.867 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.867 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.867 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.867 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.867 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.867 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.868 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.868 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.868 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.868 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.868 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.868 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.868 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.868 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.869 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.869 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.869 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.869 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.869 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.869 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.869 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.869 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.870 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.870 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.870 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.870 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.870 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.870 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.871 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.871 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.871 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.871 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.871 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.871 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.871 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.872 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.872 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.872 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.872 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.872 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.872 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.872 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.873 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.873 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.873 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.873 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.873 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.873 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.873 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.873 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.874 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.874 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.874 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.874 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.874 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.874 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.874 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.875 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.875 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.875 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.875 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.875 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.875 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.875 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.875 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.876 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.876 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.876 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-06-06T04:34:42.876 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-06-06T04:34:42.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.876 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.876 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.877 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-06-06T04:34:42.877 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.877 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.877 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.877 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-06T04:34:42.877 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.877 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.877 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.878 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-06-06T04:34:42.878 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.878 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.878 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.879 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.879 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.879 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.879 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.879 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.879 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.879 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.880 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.880 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.880 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.880 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.880 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.880 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.880 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.880 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.880 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.881 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.881 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.881 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.881 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.881 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.881 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.881 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.882 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.882 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.882 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.882 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.882 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.882 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.882 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.883 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.883 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.883 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.883 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.883 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.883 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.883 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.884 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.884 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.884 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.884 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.884 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.884 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.884 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.884 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.885 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.885 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.885 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.885 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.885 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.885 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.885 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.886 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.886 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.886 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.886 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.886 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.886 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.886 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.886 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.887 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.887 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.887 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.887 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.887 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.887 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-06-06T04:34:42.887 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-06-06T04:34:42.888 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.888 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.888 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.888 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.888 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.888 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.888 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.888 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.889 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.889 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.889 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.889 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-06-06T04:34:42.889 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-06-06T04:34:42.889 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.889 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.890 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.890 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-06-06T04:34:42.890 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.890 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.890 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.890 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.890 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.890 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.891 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.891 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-06-06T04:34:42.891 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-06-06T04:34:42.891 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.891 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.891 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.891 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.892 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.892 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.892 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.892 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.892 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.892 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.892 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.893 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-06-06T04:34:42.893 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-06-06T04:34:42.893 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.893 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.893 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.893 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.893 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.893 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.893 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.894 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.894 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.894 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.894 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.894 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-06-06T04:34:42.894 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-06-06T04:34:42.895 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-06-06T04:34:42.895 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.895 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.895 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.895 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.895 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.895 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.896 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.896 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.896 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.896 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-06-06T04:34:42.896 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-06-06T04:34:42.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.896 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.897 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.897 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.897 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.897 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.897 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.897 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.897 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.897 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.898 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.898 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-06-06T04:34:42.898 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-06-06T04:34:42.898 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.898 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.898 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.898 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.899 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.899 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.899 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.899 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.899 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.899 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.899 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.899 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-06-06T04:34:42.900 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-06-06T04:34:42.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.900 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.900 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.900 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.900 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.900 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.901 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.901 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.901 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.901 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.901 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-06-06T04:34:42.901 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-06-06T04:34:42.901 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.902 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.902 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.902 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.902 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-06-06T04:34:42.902 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-06-06T04:34:42.902 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.902 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.902 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.903 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.903 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.903 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-06-06T04:34:42.903 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-06-06T04:34:42.903 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.903 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.903 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.904 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.904 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.904 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.904 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.904 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.904 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.904 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.904 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.905 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-06-06T04:34:42.905 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-06-06T04:34:42.905 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.905 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.905 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.905 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.905 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.905 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.906 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.906 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.906 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.906 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.906 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.906 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-06-06T04:34:42.907 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-06-06T04:34:42.907 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.907 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.907 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.907 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.907 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.907 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.907 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.907 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.908 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.908 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.908 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.908 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.908 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.908 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-06-06T04:34:42.909 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.909 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.909 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.909 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-06-06T04:34:42.909 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-06-06T04:34:42.909 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.909 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.909 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.910 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.910 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.910 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.910 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.910 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-06-06T04:34:42.910 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.910 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.910 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.911 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-06-06T04:34:42.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-06-06T04:34:42.911 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.911 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.911 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.911 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.911 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.912 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.912 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.912 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-06-06T04:34:42.912 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.912 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.912 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.912 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.913 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.913 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.913 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.913 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.913 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.913 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.913 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.914 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.914 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.914 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.914 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.914 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.914 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.914 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.914 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.915 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.915 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.915 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.915 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.915 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.915 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.915 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.915 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.916 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.916 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.916 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.916 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.916 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.916 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.916 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.916 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.917 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.917 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.917 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.917 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.917 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.917 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.918 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.918 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.918 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.918 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.918 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.918 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.918 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.918 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.919 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.919 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.919 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.919 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.919 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.919 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.919 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.920 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.920 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.920 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.920 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.920 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.920 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.920 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.920 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.921 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.921 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.921 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.921 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.921 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.921 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-06-06T04:34:42.922 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.922 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.922 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.922 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.922 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.922 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.923 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.923 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.923 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.923 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.923 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.923 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.923 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.923 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.924 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.924 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.924 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.924 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.924 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.924 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.924 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.925 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.925 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.925 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.925 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.925 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.925 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.925 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.925 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.926 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.926 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.926 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.926 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.926 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.926 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.927 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.927 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.927 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.927 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.927 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.927 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.927 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.927 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.928 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.928 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.928 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.928 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.928 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.928 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.928 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.928 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.929 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.930 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-06-06T04:34:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.931 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.932 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.933 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.937 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.939 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-06-06T04:34:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-06-06T04:34:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-06-06T04:34:42.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-06T04:34:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.942 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-06-06T04:34:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-06-06T04:34:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.943 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-06-06T04:34:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.945 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.946 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.946 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.946 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.946 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.946 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.946 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.946 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.946 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.947 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.947 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.947 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.947 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.947 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.947 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.948 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.949 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.950 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.950 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.950 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.950 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.950 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.950 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.950 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.950 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.951 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.951 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.951 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.951 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.951 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.951 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.951 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.952 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-06-06T04:34:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-06-06T04:34:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-06-06T04:34:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.954 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-06T04:34:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.955 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-06-06T04:34:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-06-06T04:34:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.959 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-06T04:34:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.960 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-06-06T04:34:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-06-06T04:34:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.961 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-06T04:34:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-06-06T04:34:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-06-06T04:34:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.963 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-06T04:34:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-06-06T04:34:42.964 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-06-06T04:34:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-06T04:34:42.965 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-06-06T04:34:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-06-06T04:34:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.966 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.967 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-06-06T04:34:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-06-06T04:34:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.968 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.969 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-06-06T04:34:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-06-06T04:34:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-06T04:34:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.970 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-06-06T04:34:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-06-06T04:34:42.971 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.972 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-06-06T04:34:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-06-06T04:34:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.973 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-06-06T04:34:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-06-06T04:34:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-06-06T04:34:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-06-06T04:34:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-06-06T04:34:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.975 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-06-06T04:34:42.976 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-06-06T04:34:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.977 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-06-06T04:34:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-06-06T04:34:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.978 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.979 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.980 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.983 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.986 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.987 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-06-06T04:34:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-06-06T04:34:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.989 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-06-06T04:34:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-06-06T04:34:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.990 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.991 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-06-06T04:34:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-06-06T04:34:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-06T04:34:42.992 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.993 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-06-06T04:34:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:42.994 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:42.995 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.996 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.997 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:42.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:43.000 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:43.001 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.002 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-06-06T04:34:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:43.003 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:43.004 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:43.005 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:43.006 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:43.007 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.008 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:43.009 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.011 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-06-06T04:34:43.012 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-06-06T04:34:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-06-06T04:34:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-06T04:34:43.013 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-06-06T04:34:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-06-06T04:34:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.014 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-06-06T04:34:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.015 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-06-06T04:34:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-06-06T04:34:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-06-06T04:34:43.016 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-06T04:34:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-06-06T04:34:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-06-06T04:34:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-06-06T04:34:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.018 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-06T04:34:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-06-06T04:34:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-06-06T04:34:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-06-06T04:34:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-06-06T04:34:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-06-06T04:34:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-06-06T04:34:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-06-06T04:34:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.020 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-06-06T04:34:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.021 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-06-06T04:34:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-06-06T04:34:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:43.022 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.023 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-06-06T04:34:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-06-06T04:34:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-06-06T04:34:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-06-06T04:34:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.024 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:43.025 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:43.025 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.025 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.025 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.025 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-06-06T04:34:43.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-06-06T04:34:43.025 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:43.025 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.026 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:43.026 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:43.026 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.026 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.026 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:43.026 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:43.026 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:43.026 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:43.027 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:43.027 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:43.027 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:43.027 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:43.027 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:43.027 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.027 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.027 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.028 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-06-06T04:34:43.028 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-06-06T04:34:43.028 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:43.028 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.028 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:43.028 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:43.028 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.029 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.029 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:43.029 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:43.029 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.029 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.029 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.029 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-06-06T04:34:43.030 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-06-06T04:34:43.030 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:43.030 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.030 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:43.030 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-06-06T04:34:43.030 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.030 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.030 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-06-06T04:34:43.031 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-06-06T04:34:43.031 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-06-06T04:34:43.031 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-06-06T04:34:43.031 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-06-06T04:34:43.031 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-06-06T04:34:43.031 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-06-06T04:34:43.031 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:43.031 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:43.032 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.032 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.032 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.032 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-06-06T04:34:43.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-06-06T04:34:43.032 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-06-06T04:34:43.032 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.032 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-06-06T04:34:43.033 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-06-06T04:34:43.033 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.033 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.033 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:43.033 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:43.033 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.033 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.033 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.034 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-06-06T04:34:43.034 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-06-06T04:34:43.034 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:43.034 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.034 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:43.034 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-06-06T04:34:43.034 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.035 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.035 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:43.035 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:43.035 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.035 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.035 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.035 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-06-06T04:34:43.036 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-06-06T04:34:43.036 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-06-06T04:34:43.036 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.036 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:43.036 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-06-06T04:34:43.036 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.036 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.036 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:43.037 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:43.037 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.037 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.037 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.037 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-06-06T04:34:43.037 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-06-06T04:34:43.037 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-06-06T04:34:43.037 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-06-06T04:34:43.038 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-06-06T04:34:43.038 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-06T04:34:43.038 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-06-06T04:34:43.038 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-06-06T04:34:43.038 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-06-06T04:34:43.038 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-06-06T04:34:43.038 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-06-06T04:34:43.038 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-06-06T04:34:43.039 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-06-06T04:34:43.039 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:43.039 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:43.039 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:43.039 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:43.039 INFO:teuthology.orchestra.run.smithi045.stdout: "services": { 2024-06-06T04:34:43.039 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard": "https://172.21.15.45:8443/" 2024-06-06T04:34:43.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.040 INFO:teuthology.orchestra.run.smithi045.stdout: "always_on_modules": { 2024-06-06T04:34:43.040 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus": [ 2024-06-06T04:34:43.040 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-06-06T04:34:43.040 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-06-06T04:34:43.040 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-06-06T04:34:43.040 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-06-06T04:34:43.040 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-06-06T04:34:43.040 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-06-06T04:34:43.041 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-06-06T04:34:43.041 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-06-06T04:34:43.041 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-06-06T04:34:43.041 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-06-06T04:34:43.041 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:43.041 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific": [ 2024-06-06T04:34:43.041 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-06-06T04:34:43.041 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-06-06T04:34:43.041 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-06-06T04:34:43.042 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-06-06T04:34:43.042 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-06-06T04:34:43.042 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-06-06T04:34:43.042 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-06-06T04:34:43.042 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-06-06T04:34:43.042 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-06-06T04:34:43.042 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-06-06T04:34:43.042 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:43.043 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy": [ 2024-06-06T04:34:43.043 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-06-06T04:34:43.043 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-06-06T04:34:43.043 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-06-06T04:34:43.043 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-06-06T04:34:43.043 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-06-06T04:34:43.043 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-06-06T04:34:43.043 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-06-06T04:34:43.044 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-06-06T04:34:43.044 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-06-06T04:34:43.044 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-06-06T04:34:43.044 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:43.044 INFO:teuthology.orchestra.run.smithi045.stdout: "reef": [ 2024-06-06T04:34:43.044 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-06-06T04:34:43.044 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-06-06T04:34:43.044 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-06-06T04:34:43.045 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-06-06T04:34:43.045 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-06-06T04:34:43.045 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-06-06T04:34:43.045 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-06-06T04:34:43.045 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-06-06T04:34:43.045 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-06-06T04:34:43.045 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-06-06T04:34:43.045 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-06-06T04:34:43.045 INFO:teuthology.orchestra.run.smithi045.stdout: "squid": [ 2024-06-06T04:34:43.046 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-06-06T04:34:43.046 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-06-06T04:34:43.046 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-06-06T04:34:43.046 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-06-06T04:34:43.046 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-06-06T04:34:43.046 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-06-06T04:34:43.046 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-06-06T04:34:43.046 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-06-06T04:34:43.047 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-06-06T04:34:43.047 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-06-06T04:34:43.047 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-06-06T04:34:43.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.047 INFO:teuthology.orchestra.run.smithi045.stdout: "last_failure_osd_epoch": 3, 2024-06-06T04:34:43.047 INFO:teuthology.orchestra.run.smithi045.stdout: "active_clients": [ 2024-06-06T04:34:43.047 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:43.047 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-06-06T04:34:43.048 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-06-06T04:34:43.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:43.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-06-06T04:34:43.048 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-06-06T04:34:43.048 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2850286095 2024-06-06T04:34:43.048 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:43.048 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-06-06T04:34:43.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:43.049 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "libcephsqlite", 2024-06-06T04:34:43.049 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-06-06T04:34:43.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:43.049 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-06-06T04:34:43.049 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-06-06T04:34:43.049 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1119866721 2024-06-06T04:34:43.049 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:43.049 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-06-06T04:34:43.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:43.050 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-06-06T04:34:43.050 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-06-06T04:34:43.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:43.050 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-06-06T04:34:43.050 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-06-06T04:34:43.050 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3473588129 2024-06-06T04:34:43.051 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:43.051 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-06-06T04:34:43.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:43.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:43.051 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-06-06T04:34:43.051 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-06-06T04:34:43.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-06-06T04:34:43.051 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-06-06T04:34:43.052 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-06-06T04:34:43.052 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1520152515 2024-06-06T04:34:43.052 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:43.052 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-06-06T04:34:43.052 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:43.052 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-06-06T04:34:43.052 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:34:43.054 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:41 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/38394639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T04:34:43.054 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:41 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/4166876004' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-06T04:34:43.054 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:42 smithi154 ceph-mon[36428]: pgmap v528: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-06-06T04:34:43.054 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:41 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/38394639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T04:34:43.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:41 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/4166876004' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-06T04:34:43.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:42 smithi086 ceph-mon[36053]: pgmap v528: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-06-06T04:34:43.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.783+0000 7f4debdd5640 1 -- 172.21.15.45:0/4166876004 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4dcc0a2b90 msgr2=0x7f4dcc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:43.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.783+0000 7f4debdd5640 1 --2- 172.21.15.45:0/4166876004 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4dcc0a2b90 0x7f4dcc0a5050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4de4199190 tx=0x7f4dd800b390 comp rx=0 tx=0).stop 2024-06-06T04:34:43.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.783+0000 7f4debdd5640 1 -- 172.21.15.45:0/4166876004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de41008e0 msgr2=0x7f4de4197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:34:43.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.783+0000 7f4debdd5640 1 --2- 172.21.15.45:0/4166876004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de41008e0 0x7f4de4197c90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400e9c0 tx=0x7f4dd400ee80 comp rx=0 tx=0).stop 2024-06-06T04:34:43.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.783+0000 7f4debdd5640 1 -- 172.21.15.45:0/4166876004 shutdown_connections 2024-06-06T04:34:43.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.783+0000 7f4debdd5640 1 --2- 172.21.15.45:0/4166876004 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f4dcc0a2b90 0x7f4dcc0a5050 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:43.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.783+0000 7f4debdd5640 1 --2- 172.21.15.45:0/4166876004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4de4102490 0x7f4de419f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:43.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.783+0000 7f4debdd5640 1 --2- 172.21.15.45:0/4166876004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4de4101ad0 0x7f4de41981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:43.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.783+0000 7f4debdd5640 1 --2- 172.21.15.45:0/4166876004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de41008e0 0x7f4de4197c90 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:34:43.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.783+0000 7f4debdd5640 1 -- 172.21.15.45:0/4166876004 >> 172.21.15.45:0/4166876004 conn(0x7f4de40fbe50 msgr2=0x7f4de40fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:34:43.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.783+0000 7f4debdd5640 1 -- 172.21.15.45:0/4166876004 shutdown_connections 2024-06-06T04:34:43.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:34:40.783+0000 7f4debdd5640 1 -- 172.21.15.45:0/4166876004 wait complete. 2024-06-06T04:34:43.064 DEBUG:tasks.check_counter:Getting stats from a 2024-06-06T04:34:43.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-06-06T04:34:43.065 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:41 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/38394639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T04:34:43.065 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:41 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/4166876004' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-06T04:34:43.065 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:42 smithi045 ceph-mon[32299]: pgmap v528: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-06-06T04:34:43.325 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:34:44.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:44 smithi086 ceph-mon[36053]: pgmap v529: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-06T04:34:44.609 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:44 smithi045 ceph-mon[32299]: pgmap v529: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-06T04:34:44.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:44 smithi154 ceph-mon[36428]: pgmap v529: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-06T04:34:45.228 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:34:45.229 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T04:34:45.229 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-06-06T04:34:45.229 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-06-06T04:34:45.229 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-06-06T04:34:45.229 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-06-06T04:34:45.229 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-06-06T04:34:45.230 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-06-06T04:34:45.230 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.001582879, 2024-06-06T04:34:45.230 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-06-06T04:34:45.230 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000291155, 2024-06-06T04:34:45.230 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T04:34:45.230 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:45.230 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:45.230 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:45.231 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:45.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:45.231 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:45.231 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:45.231 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:45.231 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:45.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:45.231 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T04:34:45.232 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T04:34:45.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:45.232 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T04:34:45.232 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 1, 2024-06-06T04:34:45.232 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 95, 2024-06-06T04:34:45.232 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 160, 2024-06-06T04:34:45.232 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 94496, 2024-06-06T04:34:45.232 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 3, 2024-06-06T04:34:45.232 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2024-06-06T04:34:45.233 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.007859271, 2024-06-06T04:34:45.233 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.005063446, 2024-06-06T04:34:45.233 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001767405, 2024-06-06T04:34:45.233 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T04:34:45.233 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:45.233 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 93, 2024-06-06T04:34:45.233 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.004331239, 2024-06-06T04:34:45.233 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000046572 2024-06-06T04:34:45.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:45.234 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:45.234 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:45.234 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:45.234 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:45.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:45.234 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-06T04:34:45.234 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 94496 2024-06-06T04:34:45.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:45.235 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T04:34:45.235 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 133, 2024-06-06T04:34:45.235 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 126, 2024-06-06T04:34:45.235 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 730656, 2024-06-06T04:34:45.235 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 76496, 2024-06-06T04:34:45.235 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-06-06T04:34:45.235 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-06-06T04:34:45.235 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.021654076, 2024-06-06T04:34:45.236 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.008099872, 2024-06-06T04:34:45.236 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.014201578, 2024-06-06T04:34:45.236 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.002205066, 2024-06-06T04:34:45.236 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:45.236 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 126, 2024-06-06T04:34:45.236 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.007574629, 2024-06-06T04:34:45.236 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000060116 2024-06-06T04:34:45.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:45.236 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:45.237 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:45.237 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:45.237 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:45.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:45.237 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 730656, 2024-06-06T04:34:45.237 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 76496 2024-06-06T04:34:45.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:45.237 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-06-06T04:34:45.238 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-06-06T04:34:45.238 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-06-06T04:34:45.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:45.238 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-06-06T04:34:45.238 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-06-06T04:34:45.238 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-06-06T04:34:45.238 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T04:34:45.238 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-06-06T04:34:45.239 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T04:34:45.239 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-06-06T04:34:45.239 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T04:34:45.239 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T04:34:45.239 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T04:34:45.239 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T04:34:45.239 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T04:34:45.239 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-06-06T04:34:45.239 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T04:34:45.240 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T04:34:45.240 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-06-06T04:34:45.240 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-06-06T04:34:45.240 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-06-06T04:34:45.240 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-06-06T04:34:45.240 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T04:34:45.240 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T04:34:45.240 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T04:34:45.241 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T04:34:45.241 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T04:34:45.241 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-06-06T04:34:45.241 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-06-06T04:34:45.241 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-06-06T04:34:45.241 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T04:34:45.241 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T04:34:45.241 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-06-06T04:34:45.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-06-06T04:34:45.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-06-06T04:34:45.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-06-06T04:34:45.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T04:34:45.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-06-06T04:34:45.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T04:34:45.242 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-06-06T04:34:45.242 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148863, 2024-06-06T04:34:45.243 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 28, 2024-06-06T04:34:45.243 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-06-06T04:34:45.243 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-06-06T04:34:45.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-06-06T04:34:45.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-06-06T04:34:45.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-06-06T04:34:45.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-06-06T04:34:45.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-06-06T04:34:45.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-06-06T04:34:45.244 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-06-06T04:34:45.244 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-06-06T04:34:45.244 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T04:34:45.244 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-06-06T04:34:45.244 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-06-06T04:34:45.244 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-06-06T04:34:45.244 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-06-06T04:34:45.244 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-06-06T04:34:45.245 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-06-06T04:34:45.245 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-06-06T04:34:45.245 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-06-06T04:34:45.245 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-06-06T04:34:45.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:45.245 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T04:34:45.245 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-06-06T04:34:45.245 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-06-06T04:34:45.246 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-06-06T04:34:45.246 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 130, 2024-06-06T04:34:45.246 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 709485, 2024-06-06T04:34:45.246 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-06-06T04:34:45.246 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 130, 2024-06-06T04:34:45.246 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-06-06T04:34:45.246 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-06-06T04:34:45.246 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 130, 2024-06-06T04:34:45.247 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 709485, 2024-06-06T04:34:45.247 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-06-06T04:34:45.247 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:45.247 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:45.247 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:45.247 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:45.247 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:45.247 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:34:45.604 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:45 smithi045 ceph-mon[32299]: pgmap v530: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-06T04:34:45.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:45 smithi086 ceph-mon[36053]: pgmap v530: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-06T04:34:45.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:45 smithi154 ceph-mon[36428]: pgmap v530: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-06T04:34:45.980 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-06-06T04:34:45.980 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-06-06T04:34:45.980 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-06-06T04:34:45.980 DEBUG:tasks.check_counter:Getting stats from d 2024-06-06T04:34:45.981 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-06-06T04:34:46.234 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:34:47.824 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:47 smithi045 ceph-mon[32299]: pgmap v531: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-06-06T04:34:47.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:47 smithi086 ceph-mon[36053]: pgmap v531: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-06-06T04:34:47.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:47 smithi154 ceph-mon[36428]: pgmap v531: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-06-06T04:34:48.096 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:34:48.096 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T04:34:48.097 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-06-06T04:34:48.097 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-06-06T04:34:48.097 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-06-06T04:34:48.097 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-06-06T04:34:48.097 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-06-06T04:34:48.097 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-06-06T04:34:48.097 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.001598999, 2024-06-06T04:34:48.097 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-06-06T04:34:48.098 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000411681, 2024-06-06T04:34:48.098 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T04:34:48.098 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:48.098 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:48.098 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:48.098 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:48.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:48.098 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:48.099 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:48.099 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:48.099 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:48.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:48.099 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T04:34:48.099 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T04:34:48.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:48.099 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T04:34:48.100 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 1, 2024-06-06T04:34:48.100 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 100, 2024-06-06T04:34:48.100 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 160, 2024-06-06T04:34:48.100 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 95376, 2024-06-06T04:34:48.100 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 3, 2024-06-06T04:34:48.100 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2024-06-06T04:34:48.100 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.008092947, 2024-06-06T04:34:48.100 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.005795248, 2024-06-06T04:34:48.100 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001228717, 2024-06-06T04:34:48.101 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T04:34:48.101 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:48.101 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 98, 2024-06-06T04:34:48.101 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.004344274, 2024-06-06T04:34:48.101 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000044329 2024-06-06T04:34:48.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:48.101 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:48.101 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:48.102 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:48.102 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:48.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:48.102 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-06T04:34:48.102 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 95376 2024-06-06T04:34:48.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:48.102 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T04:34:48.102 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 140, 2024-06-06T04:34:48.103 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 133, 2024-06-06T04:34:48.103 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 732624, 2024-06-06T04:34:48.103 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 80496, 2024-06-06T04:34:48.103 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-06-06T04:34:48.103 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-06-06T04:34:48.103 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.023544974, 2024-06-06T04:34:48.103 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.007665989, 2024-06-06T04:34:48.103 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.017788469, 2024-06-06T04:34:48.103 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.002442255, 2024-06-06T04:34:48.104 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:48.104 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 133, 2024-06-06T04:34:48.104 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.006943992, 2024-06-06T04:34:48.104 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000052210 2024-06-06T04:34:48.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:48.104 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:48.104 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:48.104 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:48.104 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:48.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:48.105 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 732624, 2024-06-06T04:34:48.105 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 80496 2024-06-06T04:34:48.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:48.105 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-06-06T04:34:48.105 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-06-06T04:34:48.105 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-06-06T04:34:48.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:48.106 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-06-06T04:34:48.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-06-06T04:34:48.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-06-06T04:34:48.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T04:34:48.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-06-06T04:34:48.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T04:34:48.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-06-06T04:34:48.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T04:34:48.107 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T04:34:48.107 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T04:34:48.107 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T04:34:48.107 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T04:34:48.107 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-06-06T04:34:48.107 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T04:34:48.107 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T04:34:48.107 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-06-06T04:34:48.107 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-06-06T04:34:48.108 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-06-06T04:34:48.108 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-06-06T04:34:48.108 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T04:34:48.108 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T04:34:48.108 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T04:34:48.108 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T04:34:48.108 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T04:34:48.108 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-06-06T04:34:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-06-06T04:34:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-06-06T04:34:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T04:34:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T04:34:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-06-06T04:34:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-06-06T04:34:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-06-06T04:34:48.109 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-06-06T04:34:48.110 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T04:34:48.110 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-06-06T04:34:48.110 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T04:34:48.110 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-06-06T04:34:48.110 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148823, 2024-06-06T04:34:48.110 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 27, 2024-06-06T04:34:48.110 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-06-06T04:34:48.110 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-06-06T04:34:48.111 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-06-06T04:34:48.111 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-06-06T04:34:48.111 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-06-06T04:34:48.111 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-06-06T04:34:48.111 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-06-06T04:34:48.111 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-06-06T04:34:48.111 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-06-06T04:34:48.111 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-06-06T04:34:48.112 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T04:34:48.112 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-06-06T04:34:48.112 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-06-06T04:34:48.112 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-06-06T04:34:48.112 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-06-06T04:34:48.112 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-06-06T04:34:48.112 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-06-06T04:34:48.112 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-06-06T04:34:48.113 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-06-06T04:34:48.113 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-06-06T04:34:48.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:48.113 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T04:34:48.113 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-06-06T04:34:48.113 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-06-06T04:34:48.113 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-06-06T04:34:48.113 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 137, 2024-06-06T04:34:48.114 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 710459, 2024-06-06T04:34:48.114 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-06-06T04:34:48.114 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 137, 2024-06-06T04:34:48.114 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-06-06T04:34:48.114 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-06-06T04:34:48.114 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 137, 2024-06-06T04:34:48.114 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 710459, 2024-06-06T04:34:48.114 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-06-06T04:34:48.115 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:48.115 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:48.115 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:48.115 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:48.115 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:48.115 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:34:48.891 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-06-06T04:34:48.891 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-06-06T04:34:48.891 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-06-06T04:34:48.891 DEBUG:tasks.check_counter:Getting stats from g 2024-06-06T04:34:48.891 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-06-06T04:34:49.145 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:34:49.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:49 smithi086 ceph-mon[36053]: pgmap v532: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-06-06T04:34:49.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:49 smithi154 ceph-mon[36428]: pgmap v532: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-06-06T04:34:49.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:49 smithi045 ceph-mon[32299]: pgmap v532: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-06-06T04:34:51.001 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:34:51.001 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T04:34:51.001 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-06-06T04:34:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-06-06T04:34:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-06-06T04:34:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-06-06T04:34:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-06-06T04:34:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-06-06T04:34:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.001239547, 2024-06-06T04:34:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-06-06T04:34:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000179093, 2024-06-06T04:34:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T04:34:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T04:34:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T04:34:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T04:34:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 133, 2024-06-06T04:34:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 128, 2024-06-06T04:34:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 728816, 2024-06-06T04:34:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 78128, 2024-06-06T04:34:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-06-06T04:34:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-06-06T04:34:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.023184486, 2024-06-06T04:34:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.008392445, 2024-06-06T04:34:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.015341866, 2024-06-06T04:34:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.002448400, 2024-06-06T04:34:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 127, 2024-06-06T04:34:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.005609579, 2024-06-06T04:34:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000044169 2024-06-06T04:34:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:51.007 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:51.007 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:51.007 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:51.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:51.007 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 728816, 2024-06-06T04:34:51.007 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 78128 2024-06-06T04:34:51.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:51.007 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T04:34:51.008 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 5, 2024-06-06T04:34:51.008 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 99, 2024-06-06T04:34:51.008 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 3152, 2024-06-06T04:34:51.008 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 95296, 2024-06-06T04:34:51.008 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 3, 2024-06-06T04:34:51.008 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2024-06-06T04:34:51.008 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.008844497, 2024-06-06T04:34:51.008 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.005902156, 2024-06-06T04:34:51.009 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001984965, 2024-06-06T04:34:51.009 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T04:34:51.009 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:51.009 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 98, 2024-06-06T04:34:51.009 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.005023407, 2024-06-06T04:34:51.009 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000051259 2024-06-06T04:34:51.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:51.009 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:51.010 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:51.010 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:51.010 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:51.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:51.010 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-06-06T04:34:51.010 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 95296 2024-06-06T04:34:51.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:51.010 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-06-06T04:34:51.011 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-06-06T04:34:51.011 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-06-06T04:34:51.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:51.011 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-06-06T04:34:51.011 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-06-06T04:34:51.011 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-06-06T04:34:51.011 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T04:34:51.011 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-06-06T04:34:51.012 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T04:34:51.012 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-06-06T04:34:51.012 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T04:34:51.012 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T04:34:51.012 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T04:34:51.012 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T04:34:51.012 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T04:34:51.012 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-06-06T04:34:51.013 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T04:34:51.013 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T04:34:51.013 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-06-06T04:34:51.013 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-06-06T04:34:51.013 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-06-06T04:34:51.013 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-06-06T04:34:51.013 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T04:34:51.013 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T04:34:51.014 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T04:34:51.014 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T04:34:51.014 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T04:34:51.014 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-06-06T04:34:51.014 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-06-06T04:34:51.014 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-06-06T04:34:51.014 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T04:34:51.014 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T04:34:51.015 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-06-06T04:34:51.015 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-06-06T04:34:51.015 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-06-06T04:34:51.015 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-06-06T04:34:51.015 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T04:34:51.015 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-06-06T04:34:51.015 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T04:34:51.015 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-06-06T04:34:51.015 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148818, 2024-06-06T04:34:51.016 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 28, 2024-06-06T04:34:51.016 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-06-06T04:34:51.016 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-06-06T04:34:51.016 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-06-06T04:34:51.016 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-06-06T04:34:51.016 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-06-06T04:34:51.016 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-06-06T04:34:51.016 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-06-06T04:34:51.017 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-06-06T04:34:51.017 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-06-06T04:34:51.017 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-06-06T04:34:51.017 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T04:34:51.017 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-06-06T04:34:51.017 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-06-06T04:34:51.017 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-06-06T04:34:51.018 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-06-06T04:34:51.018 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-06-06T04:34:51.018 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-06-06T04:34:51.018 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-06-06T04:34:51.018 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-06-06T04:34:51.018 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-06-06T04:34:51.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:51.018 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T04:34:51.019 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-06-06T04:34:51.019 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-06-06T04:34:51.019 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-06-06T04:34:51.019 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 134, 2024-06-06T04:34:51.019 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 710069, 2024-06-06T04:34:51.019 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-06-06T04:34:51.019 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 134, 2024-06-06T04:34:51.019 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-06-06T04:34:51.019 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-06-06T04:34:51.020 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 134, 2024-06-06T04:34:51.020 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 710069, 2024-06-06T04:34:51.020 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-06-06T04:34:51.020 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:51.020 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:51.020 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:51.020 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:51.020 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:51.021 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:34:51.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:51 smithi045 ceph-mon[32299]: pgmap v533: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:34:51.804 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-06-06T04:34:51.804 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-06-06T04:34:51.804 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-06-06T04:34:51.804 DEBUG:tasks.check_counter:Getting stats from j 2024-06-06T04:34:51.805 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-06-06T04:34:51.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:51 smithi086 ceph-mon[36053]: pgmap v533: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:34:51.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:51 smithi154 ceph-mon[36428]: pgmap v533: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:34:52.056 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:34:53.821 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:53 smithi045 ceph-mon[32299]: pgmap v534: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:34:53.847 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-06-06T04:34:53.848 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T04:34:53.848 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 4, 2024-06-06T04:34:53.848 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 4, 2024-06-06T04:34:53.848 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 2992, 2024-06-06T04:34:53.848 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 448, 2024-06-06T04:34:53.848 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-06-06T04:34:53.848 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-06-06T04:34:53.849 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.001646413, 2024-06-06T04:34:53.849 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000040422, 2024-06-06T04:34:53.849 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000660309, 2024-06-06T04:34:53.849 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T04:34:53.849 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:53.849 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3, 2024-06-06T04:34:53.849 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000884916, 2024-06-06T04:34:53.849 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000294972 2024-06-06T04:34:53.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:53.850 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:53.850 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:53.850 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:53.850 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:53.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:53.850 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-06-06T04:34:53.850 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2024-06-06T04:34:53.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:53.851 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T04:34:53.851 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 1, 2024-06-06T04:34:53.851 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 102, 2024-06-06T04:34:53.851 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 160, 2024-06-06T04:34:53.851 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 95728, 2024-06-06T04:34:53.851 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 3, 2024-06-06T04:34:53.851 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2024-06-06T04:34:53.852 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.008504256, 2024-06-06T04:34:53.852 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.005933378, 2024-06-06T04:34:53.852 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001442583, 2024-06-06T04:34:53.852 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T04:34:53.852 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:53.852 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 100, 2024-06-06T04:34:53.852 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.005115871, 2024-06-06T04:34:53.852 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000051158 2024-06-06T04:34:53.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:53.853 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:53.853 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:53.853 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:53.853 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:53.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:53.853 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-06T04:34:53.853 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 95728 2024-06-06T04:34:53.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:53.854 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T04:34:53.854 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 139, 2024-06-06T04:34:53.854 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 134, 2024-06-06T04:34:53.854 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 730448, 2024-06-06T04:34:53.854 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 81968, 2024-06-06T04:34:53.854 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-06-06T04:34:53.854 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-06-06T04:34:53.854 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.023957216, 2024-06-06T04:34:53.855 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.008077112, 2024-06-06T04:34:53.855 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.016876815, 2024-06-06T04:34:53.855 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.002586123, 2024-06-06T04:34:53.855 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:53.855 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 133, 2024-06-06T04:34:53.855 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.005780206, 2024-06-06T04:34:53.855 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000043460 2024-06-06T04:34:53.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:53.855 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:53.856 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:53.856 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:53.856 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:53.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:53.856 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 730448, 2024-06-06T04:34:53.856 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 81968 2024-06-06T04:34:53.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:53.856 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-06-06T04:34:53.857 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-06-06T04:34:53.857 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-06-06T04:34:53.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:53.857 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-06-06T04:34:53.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-06-06T04:34:53.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-06-06T04:34:53.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T04:34:53.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-06-06T04:34:53.857 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T04:34:53.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-06-06T04:34:53.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T04:34:53.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T04:34:53.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T04:34:53.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T04:34:53.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T04:34:53.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-06-06T04:34:53.858 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T04:34:53.859 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T04:34:53.859 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-06-06T04:34:53.859 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-06-06T04:34:53.859 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-06-06T04:34:53.859 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-06-06T04:34:53.859 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T04:34:53.859 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T04:34:53.859 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T04:34:53.859 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T04:34:53.860 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T04:34:53.860 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-06-06T04:34:53.860 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-06-06T04:34:53.860 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-06-06T04:34:53.860 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T04:34:53.860 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T04:34:53.860 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-06-06T04:34:53.860 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-06-06T04:34:53.861 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-06-06T04:34:53.861 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-06-06T04:34:53.861 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T04:34:53.861 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-06-06T04:34:53.861 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T04:34:53.861 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-06-06T04:34:53.861 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148823, 2024-06-06T04:34:53.861 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 27, 2024-06-06T04:34:53.861 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-06-06T04:34:53.862 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-06-06T04:34:53.862 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-06-06T04:34:53.862 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-06-06T04:34:53.862 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-06-06T04:34:53.862 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-06-06T04:34:53.862 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-06-06T04:34:53.862 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-06-06T04:34:53.862 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-06-06T04:34:53.863 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-06-06T04:34:53.863 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T04:34:53.863 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-06-06T04:34:53.863 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-06-06T04:34:53.863 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-06-06T04:34:53.863 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-06-06T04:34:53.863 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-06-06T04:34:53.863 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-06-06T04:34:53.863 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-06-06T04:34:53.864 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-06-06T04:34:53.864 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-06-06T04:34:53.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-06-06T04:34:53.864 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T04:34:53.864 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-06-06T04:34:53.864 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-06-06T04:34:53.864 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-06-06T04:34:53.864 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 140, 2024-06-06T04:34:53.865 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 710849, 2024-06-06T04:34:53.865 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-06-06T04:34:53.865 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 140, 2024-06-06T04:34:53.865 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-06-06T04:34:53.865 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-06-06T04:34:53.865 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 140, 2024-06-06T04:34:53.865 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 710849, 2024-06-06T04:34:53.865 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-06-06T04:34:53.866 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-06-06T04:34:53.866 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-06-06T04:34:53.866 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-06-06T04:34:53.866 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:53.866 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-06-06T04:34:53.866 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-06-06T04:34:53.867 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:53 smithi086 ceph-mon[36053]: pgmap v534: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:34:53.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:53 smithi154 ceph-mon[36428]: pgmap v534: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:34:54.681 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-06-06T04:34:54.681 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-06-06T04:34:54.681 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-06-06T04:34:54.681 DEBUG:tasks.check_counter:Getting stats from b 2024-06-06T04:34:54.682 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-06-06T04:34:54.931 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T04:34:55.609 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:55 smithi086 ceph-mon[36053]: pgmap v535: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-06T04:34:55.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:55 smithi154 ceph-mon[36428]: pgmap v535: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-06T04:34:55.972 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:55 smithi045 ceph-mon[32299]: pgmap v535: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-06T04:34:56.746 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-06-06T04:34:56.747 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T04:34:56.747 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 7768, 2024-06-06T04:34:56.747 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 7904, 2024-06-06T04:34:56.747 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3364120, 2024-06-06T04:34:56.748 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 17815220, 2024-06-06T04:34:56.748 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 39, 2024-06-06T04:34:56.748 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 6, 2024-06-06T04:34:56.748 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.520040801, 2024-06-06T04:34:56.748 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.178255689, 2024-06-06T04:34:56.749 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.136832561, 2024-06-06T04:34:56.749 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.172758149, 2024-06-06T04:34:56.749 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:56.749 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 7904, 2024-06-06T04:34:56.750 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 3.159748339, 2024-06-06T04:34:56.750 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000399765 2024-06-06T04:34:56.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.750 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:56.750 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1431, 2024-06-06T04:34:56.751 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000330199, 2024-06-06T04:34:56.751 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000230 2024-06-06T04:34:56.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.751 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 836784, 2024-06-06T04:34:56.751 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 95328 2024-06-06T04:34:56.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T04:34:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 29993, 2024-06-06T04:34:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 48758, 2024-06-06T04:34:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 10722217, 2024-06-06T04:34:56.752 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 28431574, 2024-06-06T04:34:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 33, 2024-06-06T04:34:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 6, 2024-06-06T04:34:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 1.803368195, 2024-06-06T04:34:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.708141301, 2024-06-06T04:34:56.753 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 1.000381293, 2024-06-06T04:34:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.211400749, 2024-06-06T04:34:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 48756, 2024-06-06T04:34:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 7.329798206, 2024-06-06T04:34:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000150336 2024-06-06T04:34:56.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 58555, 2024-06-06T04:34:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.008867428, 2024-06-06T04:34:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000151 2024-06-06T04:34:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.755 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 57392, 2024-06-06T04:34:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 281008 2024-06-06T04:34:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T04:34:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 10281, 2024-06-06T04:34:56.756 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 10457, 2024-06-06T04:34:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3566685, 2024-06-06T04:34:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 26281391, 2024-06-06T04:34:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 12, 2024-06-06T04:34:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-06-06T04:34:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.694318165, 2024-06-06T04:34:56.757 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.277139354, 2024-06-06T04:34:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.140460087, 2024-06-06T04:34:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.233608959, 2024-06-06T04:34:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 10455, 2024-06-06T04:34:56.758 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 7.851793069, 2024-06-06T04:34:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000751008 2024-06-06T04:34:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:56.759 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1064, 2024-06-06T04:34:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000205271, 2024-06-06T04:34:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000192 2024-06-06T04:34:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T04:34:56.760 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T04:34:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-06-06T04:34:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-06-06T04:34:56.761 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-06-06T04:34:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-06-06T04:34:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-06-06T04:34:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-06-06T04:34:56.762 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3661, 2024-06-06T04:34:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 12.219087496, 2024-06-06T04:34:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.003337636 2024-06-06T04:34:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:56.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-06-06T04:34:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-06-06T04:34:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-06-06T04:34:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5593, 2024-06-06T04:34:56.764 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.944110475, 2024-06-06T04:34:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000168802 2024-06-06T04:34:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-06-06T04:34:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 12625, 2024-06-06T04:34:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 12599, 2024-06-06T04:34:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-06-06T04:34:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 12599, 2024-06-06T04:34:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 26.154835315, 2024-06-06T04:34:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002075945 2024-06-06T04:34:56.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-06-06T04:34:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 26, 2024-06-06T04:34:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 1, 2024-06-06T04:34:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-06-06T04:34:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 741, 2024-06-06T04:34:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 22, 2024-06-06T04:34:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-06-06T04:34:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 4713, 2024-06-06T04:34:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 749, 2024-06-06T04:34:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 3952, 2024-06-06T04:34:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 12, 2024-06-06T04:34:56.768 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 3643, 2024-06-06T04:34:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 28, 2024-06-06T04:34:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 3507, 2024-06-06T04:34:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 3507, 2024-06-06T04:34:56.769 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 5, 2024-06-06T04:34:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 15169, 2024-06-06T04:34:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 12835, 2024-06-06T04:34:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 23, 2024-06-06T04:34:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-06-06T04:34:56.770 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-06-06T04:34:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-06-06T04:34:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 3, 2024-06-06T04:34:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-06-06T04:34:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-06-06T04:34:56.771 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 3, 2024-06-06T04:34:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 26, 2024-06-06T04:34:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 1, 2024-06-06T04:34:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-06-06T04:34:56.772 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-06-06T04:34:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 3477, 2024-06-06T04:34:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-06-06T04:34:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 3454, 2024-06-06T04:34:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 26758916, 2024-06-06T04:34:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-06-06T04:34:56.773 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 3477, 2024-06-06T04:34:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-06-06T04:34:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T04:34:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-06-06T04:34:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 93, 2024-06-06T04:34:56.774 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 4, 2024-06-06T04:34:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 89, 2024-06-06T04:34:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 3384, 2024-06-06T04:34:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-06-06T04:34:56.775 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 1057, 2024-06-06T04:34:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 9987, 2024-06-06T04:34:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 8, 2024-06-06T04:34:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T04:34:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T04:34:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 8981, 2024-06-06T04:34:56.776 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 6825, 2024-06-06T04:34:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 36, 2024-06-06T04:34:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 13540 2024-06-06T04:34:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-06-06T04:34:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-06-06T04:34:56.777 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-06-06T04:34:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-06-06T04:34:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-06-06T04:34:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 7, 2024-06-06T04:34:56.778 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-06-06T04:34:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-06-06T04:34:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-06-06T04:34:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 3262, 2024-06-06T04:34:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 3259, 2024-06-06T04:34:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 3, 2024-06-06T04:34:56.779 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-06-06T04:34:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-06-06T04:34:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-06-06T04:34:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-06-06T04:34:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-06-06T04:34:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-06-06T04:34:56.780 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-06-06T04:34:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-06-06T04:34:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 4, 2024-06-06T04:34:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 3, 2024-06-06T04:34:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-06-06T04:34:56.781 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 22, 2024-06-06T04:34:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-06-06T04:34:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-06-06T04:34:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.782 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-06-06T04:34:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-06-06T04:34:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 21270, 2024-06-06T04:34:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 20042, 2024-06-06T04:34:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 20042, 2024-06-06T04:34:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 1228, 2024-06-06T04:34:56.783 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-06-06T04:34:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 2233, 2024-06-06T04:34:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 37, 2024-06-06T04:34:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 35, 2024-06-06T04:34:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 35, 2024-06-06T04:34:56.784 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 2, 2024-06-06T04:34:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 1, 2024-06-06T04:34:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-06-06T04:34:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 3, 2024-06-06T04:34:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 43797375, 2024-06-06T04:34:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 46024998, 2024-06-06T04:34:56.785 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 0, 2024-06-06T04:34:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-06-06T04:34:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 439, 2024-06-06T04:34:56.786 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 12.272733252, 2024-06-06T04:34:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.027956112 2024-06-06T04:34:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 0 2024-06-06T04:34:56.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-06-06T04:34:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 3528, 2024-06-06T04:34:56.788 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 6816, 2024-06-06T04:34:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 3288, 2024-06-06T04:34:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 237, 2024-06-06T04:34:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 273, 2024-06-06T04:34:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 36, 2024-06-06T04:34:56.789 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 4713, 2024-06-06T04:34:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 8149, 2024-06-06T04:34:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 3436, 2024-06-06T04:34:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 3507, 2024-06-06T04:34:56.790 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 6805, 2024-06-06T04:34:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 3298, 2024-06-06T04:34:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 107752, 2024-06-06T04:34:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 231804 2024-06-06T04:34:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.791 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-06-06T04:34:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 13054, 2024-06-06T04:34:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 13, 2024-06-06T04:34:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 12625, 2024-06-06T04:34:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 24, 2024-06-06T04:34:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 28, 2024-06-06T04:34:56.792 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-06-06T04:34:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 6706, 2024-06-06T04:34:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 16.478027897, 2024-06-06T04:34:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002457206 2024-06-06T04:34:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-06-06T04:34:56.793 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1055, 2024-06-06T04:34:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.302580247, 2024-06-06T04:34:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000286805 2024-06-06T04:34:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-06-06T04:34:56.794 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-06-06T04:34:56.795 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3, 2024-06-06T04:34:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.006086746, 2024-06-06T04:34:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002028915 2024-06-06T04:34:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-06-06T04:34:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1172, 2024-06-06T04:34:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.784064059, 2024-06-06T04:34:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000668996 2024-06-06T04:34:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.797 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-06-06T04:34:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.798 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-06-06T04:34:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-06-06T04:34:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.800 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-06-06T04:34:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.801 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-06-06T04:34:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.802 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-06-06T04:34:56.803 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-06-06T04:34:56.804 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 85, 2024-06-06T04:34:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.095101768, 2024-06-06T04:34:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001118844 2024-06-06T04:34:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-06-06T04:34:56.805 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-06-06T04:34:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-06-06T04:34:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 135, 2024-06-06T04:34:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.171419259, 2024-06-06T04:34:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001269772 2024-06-06T04:34:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-06-06T04:34:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 104, 2024-06-06T04:34:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.535108430, 2024-06-06T04:34:56.809 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.005145273 2024-06-06T04:34:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-06-06T04:34:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 82, 2024-06-06T04:34:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.186883735, 2024-06-06T04:34:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002279069 2024-06-06T04:34:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-06-06T04:34:56.810 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-06-06T04:34:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.811 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-06-06T04:34:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 22, 2024-06-06T04:34:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1.170513252, 2024-06-06T04:34:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.053205147 2024-06-06T04:34:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.812 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-06-06T04:34:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-06-06T04:34:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.813 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-06-06T04:34:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 8, 2024-06-06T04:34:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.050401563, 2024-06-06T04:34:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.006300195 2024-06-06T04:34:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-06-06T04:34:56.814 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-06-06T04:34:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 8, 2024-06-06T04:34:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.003615737, 2024-06-06T04:34:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000451967 2024-06-06T04:34:56.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-06-06T04:34:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-06-06T04:34:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 9, 2024-06-06T04:34:56.816 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.025043622, 2024-06-06T04:34:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002782624 2024-06-06T04:34:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-06-06T04:34:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 7, 2024-06-06T04:34:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.005917722, 2024-06-06T04:34:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000845388 2024-06-06T04:34:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.817 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-06-06T04:34:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3203, 2024-06-06T04:34:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 6.340071278, 2024-06-06T04:34:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001979416 2024-06-06T04:34:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-06-06T04:34:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-06-06T04:34:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-06-06T04:34:56.818 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-06-06T04:34:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-06-06T04:34:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 4, 2024-06-06T04:34:56.819 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 3, 2024-06-06T04:34:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 1, 2024-06-06T04:34:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-06-06T04:34:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 8276, 2024-06-06T04:34:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 8276, 2024-06-06T04:34:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 217, 2024-06-06T04:34:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-06-06T04:34:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.820 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-06-06T04:34:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-06-06T04:34:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-06-06T04:34:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T04:34:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-06-06T04:34:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T04:34:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-06-06T04:34:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T04:34:56.821 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T04:34:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T04:34:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T04:34:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T04:34:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-06-06T04:34:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T04:34:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T04:34:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-06-06T04:34:56.822 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-06-06T04:34:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-06-06T04:34:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-06-06T04:34:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T04:34:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T04:34:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T04:34:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T04:34:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T04:34:56.823 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-06-06T04:34:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-06-06T04:34:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-06-06T04:34:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T04:34:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T04:34:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-06-06T04:34:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-06-06T04:34:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-06-06T04:34:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-06-06T04:34:56.824 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T04:34:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-06-06T04:34:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T04:34:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-06-06T04:34:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 307067, 2024-06-06T04:34:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 118, 2024-06-06T04:34:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-06-06T04:34:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-06-06T04:34:56.825 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-06-06T04:34:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-06-06T04:34:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-06-06T04:34:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-06-06T04:34:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-06-06T04:34:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-06-06T04:34:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 8992, 2024-06-06T04:34:56.826 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 173, 2024-06-06T04:34:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T04:34:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-06-06T04:34:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-06-06T04:34:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-06-06T04:34:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 12681585, 2024-06-06T04:34:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 89372, 2024-06-06T04:34:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-06-06T04:34:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-06-06T04:34:56.827 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-06-06T04:34:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-06-06T04:34:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-06-06T04:34:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-06-06T04:34:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-06-06T04:34:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 24830, 2024-06-06T04:34:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 54046439, 2024-06-06T04:34:56.828 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 37, 2024-06-06T04:34:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 24830, 2024-06-06T04:34:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-06-06T04:34:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 24830, 2024-06-06T04:34:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 4896.961579010, 2024-06-06T04:34:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.197219556 2024-06-06T04:34:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-06-06T04:34:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-06-06T04:34:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 24830, 2024-06-06T04:34:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 45645 2024-06-06T04:34:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 24830, 2024-06-06T04:34:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 3567, 2024-06-06T04:34:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 21263, 2024-06-06T04:34:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-06-06T04:34:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-06-06T04:34:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 483, 2024-06-06T04:34:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 12747, 2024-06-06T04:34:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 85, 2024-06-06T04:34:56.831 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 520, 2024-06-06T04:34:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 88, 2024-06-06T04:34:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-06-06T04:34:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-06-06T04:34:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-06-06T04:34:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 6, 2024-06-06T04:34:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 7049, 2024-06-06T04:34:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-06-06T04:34:56.832 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-06-06T04:34:56.833 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-06-06T04:34:56.834 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 3482, 2024-06-06T04:34:56.835 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 19269, 2024-06-06T04:34:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-06-06T04:34:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-06-06T04:34:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-06-06T04:34:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-06-06T04:34:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-06-06T04:34:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-06-06T04:34:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T04:34:56.836 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-06-06T04:34:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-06-06T04:34:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 8, 2024-06-06T04:34:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-06-06T04:34:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-06-06T04:34:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-06-06T04:34:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-06-06T04:34:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-06-06T04:34:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-06-06T04:34:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-06-06T04:34:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-06-06T04:34:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-06-06T04:34:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-06-06T04:34:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-06-06T04:34:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-06-06T04:34:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-06-06T04:34:56.838 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-06-06T04:34:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-06-06T04:34:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-06-06T04:34:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 100, 2024-06-06T04:34:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-06-06T04:34:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 11, 2024-06-06T04:34:56.839 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 12, 2024-06-06T04:34:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 12, 2024-06-06T04:34:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-06-06T04:34:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-06-06T04:34:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 1626, 2024-06-06T04:34:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-06-06T04:34:56.840 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 280 2024-06-06T04:34:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-06-06T04:34:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 1, 2024-06-06T04:34:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-06-06T04:34:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-06-06T04:34:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-06-06T04:34:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.841 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-06-06T04:34:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-06-06T04:34:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-06-06T04:34:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-06-06T04:34:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 17, 2024-06-06T04:34:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-06-06T04:34:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 10, 2024-06-06T04:34:56.842 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 6503, 2024-06-06T04:34:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 3259, 2024-06-06T04:34:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-06-06T04:34:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T04:34:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-06-06T04:34:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-06-06T04:34:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-06-06T04:34:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 48038, 2024-06-06T04:34:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 13306427, 2024-06-06T04:34:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-06-06T04:34:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 48038, 2024-06-06T04:34:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-06-06T04:34:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-06-06T04:34:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 48038, 2024-06-06T04:34:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 13306427, 2024-06-06T04:34:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-06-06T04:34:56.844 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-06-06T04:34:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-06-06T04:34:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-06-06T04:34:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-06-06T04:34:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-06-06T04:34:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-06-06T04:34:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-06-06T04:34:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-06-06T04:34:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 24830, 2024-06-06T04:34:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 54401534, 2024-06-06T04:34:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 24830, 2024-06-06T04:34:56.846 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 54401534, 2024-06-06T04:34:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-06-06T04:34:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-06-06T04:34:56.847 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-06-06T04:34:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-06-06T04:34:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-06-06T04:34:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-06-06T04:34:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-06-06T04:34:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-06-06T04:34:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-06-06T04:34:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 24830, 2024-06-06T04:34:56.848 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 24830, 2024-06-06T04:34:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 24830, 2024-06-06T04:34:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 24830, 2024-06-06T04:34:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-06-06T04:34:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:56.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-06-06T04:34:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-06-06T04:34:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-06-06T04:34:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-06-06T04:34:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 3259, 2024-06-06T04:34:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 355111, 2024-06-06T04:34:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-06-06T04:34:56.850 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 3259, 2024-06-06T04:34:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-06-06T04:34:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-06-06T04:34:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 81, 2024-06-06T04:34:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 355111, 2024-06-06T04:34:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-06-06T04:34:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.851 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x562026d12500": { 2024-06-06T04:34:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-06-06T04:34:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-06-06T04:34:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-06-06T04:34:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 21270, 2024-06-06T04:34:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 41830694, 2024-06-06T04:34:56.852 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-06-06T04:34:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 21270, 2024-06-06T04:34:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-06-06T04:34:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-06-06T04:34:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 439, 2024-06-06T04:34:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 41830694, 2024-06-06T04:34:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-06-06T04:34:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:56.853 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:56.854 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:56.854 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-06-06T04:34:57.548 INFO:tasks.check_counter:Daemon mds.b mds.exported=3 2024-06-06T04:34:57.548 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-06-06T04:34:57.548 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=22 2024-06-06T04:34:57.549 DEBUG:tasks.check_counter:Getting stats from e 2024-06-06T04:34:57.549 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-06-06T04:34:57.799 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T04:34:57.833 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:57 smithi086 ceph-mon[36053]: pgmap v536: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:34:57.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:57 smithi154 ceph-mon[36428]: pgmap v536: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:34:57.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:57 smithi045 ceph-mon[32299]: pgmap v536: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:34:59.589 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:34:59 smithi086 ceph-mon[36053]: pgmap v537: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:34:59.618 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-06-06T04:34:59.618 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T04:34:59.618 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 401, 2024-06-06T04:34:59.619 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 403, 2024-06-06T04:34:59.619 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 124429, 2024-06-06T04:34:59.619 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 127581, 2024-06-06T04:34:59.619 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-06-06T04:34:59.619 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-06-06T04:34:59.619 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.036194834, 2024-06-06T04:34:59.619 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.013883426, 2024-06-06T04:34:59.619 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.008987206, 2024-06-06T04:34:59.620 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.009465802, 2024-06-06T04:34:59.620 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:59.620 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 401, 2024-06-06T04:34:59.620 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.015159794, 2024-06-06T04:34:59.620 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000037804 2024-06-06T04:34:59.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.620 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:59.620 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.621 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.621 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.621 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T04:34:59.621 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T04:34:59.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.621 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T04:34:59.621 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 157, 2024-06-06T04:34:59.621 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 144, 2024-06-06T04:34:59.622 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 775454, 2024-06-06T04:34:59.622 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 83602, 2024-06-06T04:34:59.622 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-06-06T04:34:59.622 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-06-06T04:34:59.622 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.025938407, 2024-06-06T04:34:59.622 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.008713207, 2024-06-06T04:34:59.622 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.016689089, 2024-06-06T04:34:59.622 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.003343203, 2024-06-06T04:34:59.623 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:59.623 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 144, 2024-06-06T04:34:59.623 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.009421241, 2024-06-06T04:34:59.623 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000065425 2024-06-06T04:34:59.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.623 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:59.623 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.623 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.624 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.624 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 774352, 2024-06-06T04:34:59.624 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 82960 2024-06-06T04:34:59.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.624 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T04:34:59.624 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 686, 2024-06-06T04:34:59.624 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 787, 2024-06-06T04:34:59.624 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 203770, 2024-06-06T04:34:59.625 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 448886, 2024-06-06T04:34:59.625 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 5, 2024-06-06T04:34:59.625 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-06-06T04:34:59.625 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.072660129, 2024-06-06T04:34:59.625 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.025066051, 2024-06-06T04:34:59.625 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.016906089, 2024-06-06T04:34:59.625 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.023378143, 2024-06-06T04:34:59.625 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:34:59.626 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 785, 2024-06-06T04:34:59.626 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.032939403, 2024-06-06T04:34:59.626 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000041961 2024-06-06T04:34:59.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.626 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:34:59.626 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.626 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.626 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.627 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-06T04:34:59.627 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 231056 2024-06-06T04:34:59.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.627 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-06-06T04:34:59.627 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-06-06T04:34:59.627 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-06-06T04:34:59.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.627 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-06-06T04:34:59.628 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-06-06T04:34:59.628 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-06-06T04:34:59.628 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1845, 2024-06-06T04:34:59.628 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.050092928, 2024-06-06T04:34:59.628 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000027150 2024-06-06T04:34:59.628 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:59.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.628 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-06-06T04:34:59.629 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-06-06T04:34:59.629 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-06-06T04:34:59.629 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.629 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.629 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.629 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:59.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.629 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-06-06T04:34:59.629 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-06-06T04:34:59.630 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-06-06T04:34:59.630 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-06-06T04:34:59.630 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.630 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.630 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.630 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-06-06T04:34:59.630 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-06-06T04:34:59.631 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-06-06T04:34:59.631 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-06-06T04:34:59.631 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-06-06T04:34:59.631 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-06-06T04:34:59.631 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-06-06T04:34:59.631 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 17, 2024-06-06T04:34:59.631 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 0, 2024-06-06T04:34:59.631 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 17, 2024-06-06T04:34:59.632 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-06-06T04:34:59.632 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 17, 2024-06-06T04:34:59.632 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-06-06T04:34:59.632 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-06-06T04:34:59.632 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-06-06T04:34:59.632 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 3, 2024-06-06T04:34:59.632 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-06-06T04:34:59.632 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-06-06T04:34:59.632 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-06-06T04:34:59.633 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-06-06T04:34:59.633 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-06-06T04:34:59.633 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-06-06T04:34:59.633 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-06-06T04:34:59.633 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-06-06T04:34:59.633 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-06-06T04:34:59.633 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-06-06T04:34:59.633 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-06-06T04:34:59.634 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-06-06T04:34:59.634 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-06-06T04:34:59.634 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-06-06T04:34:59.634 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-06-06T04:34:59.634 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-06-06T04:34:59.634 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-06-06T04:34:59.634 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-06-06T04:34:59.634 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-06-06T04:34:59.635 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T04:34:59.635 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-06-06T04:34:59.635 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T04:34:59.635 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-06-06T04:34:59.635 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-06-06T04:34:59.635 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T04:34:59.635 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T04:34:59.635 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-06-06T04:34:59.635 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-06-06T04:34:59.636 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T04:34:59.636 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-06-06T04:34:59.636 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T04:34:59.636 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T04:34:59.636 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T04:34:59.636 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-06-06T04:34:59.636 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-06-06T04:34:59.636 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-06-06T04:34:59.636 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-06-06T04:34:59.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.637 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-06-06T04:34:59.637 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-06-06T04:34:59.637 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-06-06T04:34:59.637 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-06-06T04:34:59.637 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-06-06T04:34:59.637 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-06-06T04:34:59.637 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-06-06T04:34:59.638 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-06-06T04:34:59.638 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-06-06T04:34:59.638 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-06-06T04:34:59.638 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-06-06T04:34:59.638 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-06-06T04:34:59.638 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-06-06T04:34:59.638 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-06-06T04:34:59.638 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-06-06T04:34:59.639 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-06-06T04:34:59.639 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-06-06T04:34:59.639 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-06-06T04:34:59.639 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-06-06T04:34:59.639 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-06-06T04:34:59.639 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T04:34:59.639 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-06-06T04:34:59.639 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-06-06T04:34:59.640 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-06-06T04:34:59.640 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-06-06T04:34:59.640 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-06-06T04:34:59.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.640 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-06-06T04:34:59.640 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-06-06T04:34:59.640 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-06-06T04:34:59.640 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-06-06T04:34:59.640 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-06-06T04:34:59.641 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 11, 2024-06-06T04:34:59.641 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-06-06T04:34:59.641 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-06-06T04:34:59.641 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-06-06T04:34:59.641 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-06-06T04:34:59.641 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-06-06T04:34:59.641 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 2, 2024-06-06T04:34:59.641 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 2, 2024-06-06T04:34:59.642 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-06-06T04:34:59.642 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-06-06T04:34:59.642 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-06-06T04:34:59.642 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 4213748, 2024-06-06T04:34:59.642 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 4213174, 2024-06-06T04:34:59.642 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-06-06T04:34:59.642 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.642 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.642 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.643 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 11 2024-06-06T04:34:59.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.643 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-06-06T04:34:59.643 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 18, 2024-06-06T04:34:59.643 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 18, 2024-06-06T04:34:59.643 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 0, 2024-06-06T04:34:59.643 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 15, 2024-06-06T04:34:59.644 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 15, 2024-06-06T04:34:59.644 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 0, 2024-06-06T04:34:59.644 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 17, 2024-06-06T04:34:59.644 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 17, 2024-06-06T04:34:59.644 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 0, 2024-06-06T04:34:59.644 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-06-06T04:34:59.644 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-06-06T04:34:59.644 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-06-06T04:34:59.645 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 50064, 2024-06-06T04:34:59.645 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 215420 2024-06-06T04:34:59.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.645 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-06-06T04:34:59.645 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-06-06T04:34:59.645 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-06-06T04:34:59.645 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-06-06T04:34:59.645 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-06-06T04:34:59.646 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-06-06T04:34:59.646 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-06-06T04:34:59.646 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.646 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.646 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.646 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-06-06T04:34:59.646 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.647 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.647 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.647 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-06-06T04:34:59.647 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.647 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.647 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.647 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-06-06T04:34:59.648 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.648 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.648 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.648 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-06-06T04:34:59.648 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.648 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.648 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.649 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-06-06T04:34:59.649 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.649 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.649 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.649 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-06-06T04:34:59.649 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.649 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.650 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.650 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-06-06T04:34:59.650 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.650 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.650 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.650 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-06-06T04:34:59.651 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.651 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.651 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.651 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-06-06T04:34:59.651 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.651 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.651 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.652 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-06-06T04:34:59.652 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.652 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.652 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.652 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-06-06T04:34:59.652 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.653 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.653 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.653 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-06-06T04:34:59.654 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.654 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.654 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.654 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-06-06T04:34:59.654 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.654 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.654 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.655 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-06-06T04:34:59.655 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.655 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.655 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.655 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-06-06T04:34:59.655 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.656 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.656 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.656 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-06-06T04:34:59.656 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.656 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.656 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.656 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-06-06T04:34:59.657 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.657 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.657 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.657 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-06-06T04:34:59.657 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.657 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.657 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.658 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-06-06T04:34:59.658 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.658 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.658 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.658 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-06-06T04:34:59.658 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.659 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.659 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.659 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-06-06T04:34:59.659 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.659 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.659 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.659 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-06-06T04:34:59.660 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.660 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.660 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.660 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-06-06T04:34:59.660 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.660 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.660 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.661 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-06-06T04:34:59.661 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.661 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.661 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.661 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-06-06T04:34:59.661 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.661 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.662 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.662 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-06-06T04:34:59.662 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.662 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.662 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.662 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-06-06T04:34:59.663 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.663 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.663 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.663 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-06-06T04:34:59.663 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.663 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.663 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.664 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-06-06T04:34:59.664 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-06-06T04:34:59.664 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-06-06T04:34:59.664 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.664 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.664 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.664 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:59.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.664 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-06-06T04:34:59.665 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-06-06T04:34:59.665 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 2, 2024-06-06T04:34:59.665 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 1, 2024-06-06T04:34:59.665 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 1, 2024-06-06T04:34:59.665 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-06-06T04:34:59.665 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-06-06T04:34:59.665 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-06-06T04:34:59.665 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 3046, 2024-06-06T04:34:59.666 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-06-06T04:34:59.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.666 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-06-06T04:34:59.666 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-06-06T04:34:59.666 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-06-06T04:34:59.666 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T04:34:59.666 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-06-06T04:34:59.666 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T04:34:59.667 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-06-06T04:34:59.667 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T04:34:59.667 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T04:34:59.667 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T04:34:59.667 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T04:34:59.667 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T04:34:59.667 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-06-06T04:34:59.667 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T04:34:59.667 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T04:34:59.668 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-06-06T04:34:59.668 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-06-06T04:34:59.668 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-06-06T04:34:59.668 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-06-06T04:34:59.668 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T04:34:59.668 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T04:34:59.668 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T04:34:59.668 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T04:34:59.669 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T04:34:59.669 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-06-06T04:34:59.669 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-06-06T04:34:59.669 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-06-06T04:34:59.669 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T04:34:59.669 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T04:34:59.669 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-06-06T04:34:59.669 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-06-06T04:34:59.669 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-06-06T04:34:59.670 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-06-06T04:34:59.670 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T04:34:59.670 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-06-06T04:34:59.670 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T04:34:59.670 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-06-06T04:34:59.670 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 198121, 2024-06-06T04:34:59.670 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 55, 2024-06-06T04:34:59.671 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-06-06T04:34:59.671 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-06-06T04:34:59.671 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-06-06T04:34:59.671 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-06-06T04:34:59.671 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-06-06T04:34:59.671 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-06-06T04:34:59.671 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-06-06T04:34:59.671 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-06-06T04:34:59.671 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 8992, 2024-06-06T04:34:59.672 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 173, 2024-06-06T04:34:59.672 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T04:34:59.672 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-06-06T04:34:59.672 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-06-06T04:34:59.672 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-06-06T04:34:59.672 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 71109, 2024-06-06T04:34:59.672 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 230, 2024-06-06T04:34:59.672 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-06-06T04:34:59.673 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-06-06T04:34:59.673 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-06-06T04:34:59.673 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-06-06T04:34:59.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.673 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-06-06T04:34:59.673 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-06-06T04:34:59.673 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-06-06T04:34:59.673 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 1088, 2024-06-06T04:34:59.673 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-06-06T04:34:59.674 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-06-06T04:34:59.674 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 1088, 2024-06-06T04:34:59.674 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-06-06T04:34:59.674 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1088, 2024-06-06T04:34:59.674 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.651986239, 2024-06-06T04:34:59.674 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000599252 2024-06-06T04:34:59.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.674 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-06-06T04:34:59.675 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-06-06T04:34:59.675 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1088, 2024-06-06T04:34:59.675 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1089 2024-06-06T04:34:59.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.675 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 1088, 2024-06-06T04:34:59.675 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 1088, 2024-06-06T04:34:59.675 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-06-06T04:34:59.675 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-06-06T04:34:59.676 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-06-06T04:34:59.676 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 685, 2024-06-06T04:34:59.676 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-06-06T04:34:59.676 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 401, 2024-06-06T04:34:59.676 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-06-06T04:34:59.676 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-06-06T04:34:59.676 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-06-06T04:34:59.676 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-06-06T04:34:59.677 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-06-06T04:34:59.677 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-06-06T04:34:59.677 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-06-06T04:34:59.677 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-06-06T04:34:59.677 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-06-06T04:34:59.677 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-06-06T04:34:59.677 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-06-06T04:34:59.677 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-06-06T04:34:59.677 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-06-06T04:34:59.678 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-06-06T04:34:59.678 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-06-06T04:34:59.678 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-06-06T04:34:59.678 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-06-06T04:34:59.678 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-06-06T04:34:59.678 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T04:34:59.678 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-06-06T04:34:59.678 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-06-06T04:34:59.679 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-06-06T04:34:59.679 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-06-06T04:34:59.679 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-06-06T04:34:59.679 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-06-06T04:34:59.679 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-06-06T04:34:59.679 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-06-06T04:34:59.679 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-06-06T04:34:59.679 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-06-06T04:34:59.679 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-06-06T04:34:59.680 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-06-06T04:34:59.680 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-06-06T04:34:59.680 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-06-06T04:34:59.680 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-06-06T04:34:59.680 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-06-06T04:34:59.680 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-06-06T04:34:59.680 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-06-06T04:34:59.680 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-06-06T04:34:59.681 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 100, 2024-06-06T04:34:59.681 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-06-06T04:34:59.681 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 2, 2024-06-06T04:34:59.681 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 6, 2024-06-06T04:34:59.681 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 6, 2024-06-06T04:34:59.681 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-06-06T04:34:59.681 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-06-06T04:34:59.681 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-06-06T04:34:59.681 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-06-06T04:34:59.682 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-06-06T04:34:59.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.682 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-06-06T04:34:59.682 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-06-06T04:34:59.682 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-06-06T04:34:59.682 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-06-06T04:34:59.682 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-06-06T04:34:59.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.683 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-06-06T04:34:59.683 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-06-06T04:34:59.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.683 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-06-06T04:34:59.683 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-06-06T04:34:59.683 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T04:34:59.683 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-06-06T04:34:59.683 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-06-06T04:34:59.683 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-06-06T04:34:59.684 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-06-06T04:34:59.684 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-06-06T04:34:59.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.684 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T04:34:59.684 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-06-06T04:34:59.684 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-06-06T04:34:59.684 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-06-06T04:34:59.684 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 1240, 2024-06-06T04:34:59.685 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 981358, 2024-06-06T04:34:59.685 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-06-06T04:34:59.685 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 1240, 2024-06-06T04:34:59.685 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-06-06T04:34:59.685 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-06-06T04:34:59.685 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 1240, 2024-06-06T04:34:59.685 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 981358, 2024-06-06T04:34:59.685 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-06-06T04:34:59.685 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.686 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.686 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.686 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:59.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.686 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-06-06T04:34:59.686 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-06-06T04:34:59.686 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-06-06T04:34:59.687 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-06-06T04:34:59.687 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-06-06T04:34:59.687 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-06-06T04:34:59.687 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-06-06T04:34:59.687 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-06-06T04:34:59.687 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 1088, 2024-06-06T04:34:59.687 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 19449, 2024-06-06T04:34:59.687 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 1088, 2024-06-06T04:34:59.687 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 19449, 2024-06-06T04:34:59.688 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-06-06T04:34:59.688 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.688 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.688 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.688 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:59.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.688 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-06-06T04:34:59.688 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-06-06T04:34:59.689 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-06-06T04:34:59.689 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-06-06T04:34:59.689 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-06-06T04:34:59.689 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-06-06T04:34:59.689 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-06-06T04:34:59.689 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-06-06T04:34:59.689 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 1088, 2024-06-06T04:34:59.689 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 1088, 2024-06-06T04:34:59.689 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 1088, 2024-06-06T04:34:59.690 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1088, 2024-06-06T04:34:59.690 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-06-06T04:34:59.690 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.690 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.690 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.690 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:59.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.690 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-06-06T04:34:59.691 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-06-06T04:34:59.691 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-06-06T04:34:59.691 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-06-06T04:34:59.691 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-06-06T04:34:59.691 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-06-06T04:34:59.691 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-06-06T04:34:59.691 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-06-06T04:34:59.691 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-06-06T04:34:59.691 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-06-06T04:34:59.692 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-06-06T04:34:59.692 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-06-06T04:34:59.692 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-06-06T04:34:59.692 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.692 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.692 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.692 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:59.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:34:59.693 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x562ebe0ac6e0": { 2024-06-06T04:34:59.693 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-06-06T04:34:59.693 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-06-06T04:34:59.693 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-06-06T04:34:59.693 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-06-06T04:34:59.693 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-06-06T04:34:59.693 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-06-06T04:34:59.693 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-06-06T04:34:59.694 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-06-06T04:34:59.694 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-06-06T04:34:59.694 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-06-06T04:34:59.694 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-06-06T04:34:59.694 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-06-06T04:34:59.694 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:34:59.694 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:34:59.694 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:34:59.695 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:59.695 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:34:59.695 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-06-06T04:34:59.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:34:59 smithi154 ceph-mon[36428]: pgmap v537: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:34:59.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:34:59 smithi045 ceph-mon[32299]: pgmap v537: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:35:00.472 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-06-06T04:35:00.472 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-06-06T04:35:00.472 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-06-06T04:35:00.472 DEBUG:tasks.check_counter:Getting stats from h 2024-06-06T04:35:00.473 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-06-06T04:35:00.710 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T04:35:00.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:00 smithi086 ceph-mon[36053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:35:00.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:00 smithi154 ceph-mon[36428]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:35:00.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:00 smithi045 ceph-mon[32299]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:35:01.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:01 smithi086 ceph-mon[36053]: pgmap v538: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:35:01.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:01 smithi154 ceph-mon[36428]: pgmap v538: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:35:01.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:01 smithi045 ceph-mon[32299]: pgmap v538: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:35:02.429 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-06-06T04:35:02.429 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T04:35:02.429 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 856, 2024-06-06T04:35:02.429 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 858, 2024-06-06T04:35:02.430 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 21112591, 2024-06-06T04:35:02.430 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 272218, 2024-06-06T04:35:02.430 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 7, 2024-06-06T04:35:02.430 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-06-06T04:35:02.430 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.085674437, 2024-06-06T04:35:02.430 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.027696322, 2024-06-06T04:35:02.430 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.029062602, 2024-06-06T04:35:02.430 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.021976368, 2024-06-06T04:35:02.431 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:02.431 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 856, 2024-06-06T04:35:02.431 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.027812590, 2024-06-06T04:35:02.431 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000032491 2024-06-06T04:35:02.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.431 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:02.431 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.432 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.432 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.432 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T04:35:02.432 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T04:35:02.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.432 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T04:35:02.432 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 132, 2024-06-06T04:35:02.433 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 232, 2024-06-06T04:35:02.433 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 12616530, 2024-06-06T04:35:02.433 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 274220, 2024-06-06T04:35:02.433 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-06-06T04:35:02.433 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-06-06T04:35:02.433 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.024193809, 2024-06-06T04:35:02.433 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.007459294, 2024-06-06T04:35:02.433 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.011248046, 2024-06-06T04:35:02.434 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.003605113, 2024-06-06T04:35:02.434 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:02.434 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 230, 2024-06-06T04:35:02.434 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.012065243, 2024-06-06T04:35:02.434 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000052457 2024-06-06T04:35:02.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.434 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:02.435 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.435 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.435 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.435 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-06T04:35:02.435 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 232496 2024-06-06T04:35:02.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.435 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T04:35:02.436 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 465, 2024-06-06T04:35:02.436 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 452, 2024-06-06T04:35:02.436 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 9239627, 2024-06-06T04:35:02.436 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 181061, 2024-06-06T04:35:02.436 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-06-06T04:35:02.436 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-06-06T04:35:02.436 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.056269481, 2024-06-06T04:35:02.436 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.016072649, 2024-06-06T04:35:02.436 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.028816561, 2024-06-06T04:35:02.437 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.012851941, 2024-06-06T04:35:02.437 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:02.437 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 452, 2024-06-06T04:35:02.437 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.017945802, 2024-06-06T04:35:02.437 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000039703 2024-06-06T04:35:02.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.437 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:02.438 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.438 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.438 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.438 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 774240, 2024-06-06T04:35:02.438 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 82480 2024-06-06T04:35:02.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.438 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-06-06T04:35:02.439 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-06-06T04:35:02.439 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-06-06T04:35:02.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.439 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-06-06T04:35:02.439 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-06-06T04:35:02.439 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-06-06T04:35:02.439 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2145, 2024-06-06T04:35:02.439 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.130737095, 2024-06-06T04:35:02.440 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000060949 2024-06-06T04:35:02.440 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:35:02.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.440 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-06-06T04:35:02.440 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-06-06T04:35:02.440 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-06-06T04:35:02.440 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.440 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.441 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.441 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:35:02.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.441 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-06-06T04:35:02.441 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-06-06T04:35:02.441 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-06-06T04:35:02.441 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-06-06T04:35:02.441 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.442 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.442 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.442 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-06-06T04:35:02.442 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-06-06T04:35:02.442 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-06-06T04:35:02.442 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-06-06T04:35:02.442 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-06-06T04:35:02.443 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-06-06T04:35:02.443 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-06-06T04:35:02.443 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 7953, 2024-06-06T04:35:02.443 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 5275, 2024-06-06T04:35:02.443 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 2678, 2024-06-06T04:35:02.443 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-06-06T04:35:02.443 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 418, 2024-06-06T04:35:02.443 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-06-06T04:35:02.444 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-06-06T04:35:02.444 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-06-06T04:35:02.444 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 4, 2024-06-06T04:35:02.444 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-06-06T04:35:02.444 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-06-06T04:35:02.444 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-06-06T04:35:02.444 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-06-06T04:35:02.444 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-06-06T04:35:02.445 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-06-06T04:35:02.445 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-06-06T04:35:02.445 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-06-06T04:35:02.445 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-06-06T04:35:02.445 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-06-06T04:35:02.445 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-06-06T04:35:02.445 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-06-06T04:35:02.445 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-06-06T04:35:02.446 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-06-06T04:35:02.446 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-06-06T04:35:02.446 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-06-06T04:35:02.446 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 3454, 2024-06-06T04:35:02.446 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 26758916, 2024-06-06T04:35:02.446 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-06-06T04:35:02.446 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T04:35:02.446 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-06-06T04:35:02.447 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T04:35:02.447 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-06-06T04:35:02.447 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-06-06T04:35:02.447 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T04:35:02.447 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T04:35:02.447 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-06-06T04:35:02.447 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-06-06T04:35:02.447 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T04:35:02.447 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-06-06T04:35:02.448 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T04:35:02.448 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T04:35:02.448 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T04:35:02.448 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-06-06T04:35:02.448 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-06-06T04:35:02.448 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-06-06T04:35:02.448 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-06-06T04:35:02.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.449 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-06-06T04:35:02.449 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-06-06T04:35:02.449 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-06-06T04:35:02.449 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-06-06T04:35:02.449 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-06-06T04:35:02.449 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-06-06T04:35:02.450 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-06-06T04:35:02.450 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-06-06T04:35:02.450 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-06-06T04:35:02.450 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-06-06T04:35:02.450 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-06-06T04:35:02.450 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-06-06T04:35:02.450 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-06-06T04:35:02.450 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-06-06T04:35:02.451 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-06-06T04:35:02.451 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-06-06T04:35:02.451 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-06-06T04:35:02.451 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-06-06T04:35:02.451 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-06-06T04:35:02.451 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-06-06T04:35:02.451 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T04:35:02.451 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-06-06T04:35:02.452 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-06-06T04:35:02.452 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-06-06T04:35:02.452 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-06-06T04:35:02.452 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-06-06T04:35:02.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.452 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-06-06T04:35:02.452 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-06-06T04:35:02.452 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-06-06T04:35:02.453 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-06-06T04:35:02.453 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-06-06T04:35:02.453 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 21270, 2024-06-06T04:35:02.453 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-06-06T04:35:02.453 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-06-06T04:35:02.453 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-06-06T04:35:02.453 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-06-06T04:35:02.453 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-06-06T04:35:02.454 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 2, 2024-06-06T04:35:02.454 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 12, 2024-06-06T04:35:02.454 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-06-06T04:35:02.454 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-06-06T04:35:02.454 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 43797375, 2024-06-06T04:35:02.454 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 46024998, 2024-06-06T04:35:02.454 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 46019846, 2024-06-06T04:35:02.454 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-06-06T04:35:02.455 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.455 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.455 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.455 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 21270 2024-06-06T04:35:02.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.455 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-06-06T04:35:02.455 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 3528, 2024-06-06T04:35:02.455 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 6811, 2024-06-06T04:35:02.456 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 3283, 2024-06-06T04:35:02.456 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 237, 2024-06-06T04:35:02.456 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 273, 2024-06-06T04:35:02.456 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 36, 2024-06-06T04:35:02.456 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 7953, 2024-06-06T04:35:02.456 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 8021, 2024-06-06T04:35:02.456 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 68, 2024-06-06T04:35:02.456 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-06-06T04:35:02.457 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-06-06T04:35:02.457 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-06-06T04:35:02.457 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 69140, 2024-06-06T04:35:02.457 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 215420 2024-06-06T04:35:02.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.457 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-06-06T04:35:02.457 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-06-06T04:35:02.457 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-06-06T04:35:02.458 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-06-06T04:35:02.458 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-06-06T04:35:02.458 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-06-06T04:35:02.458 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-06-06T04:35:02.458 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.458 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.458 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.459 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-06-06T04:35:02.459 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.459 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.459 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.459 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-06-06T04:35:02.459 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.459 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.460 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.460 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-06-06T04:35:02.460 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.460 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.460 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.460 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-06-06T04:35:02.461 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.461 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.461 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.461 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-06-06T04:35:02.461 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.461 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.461 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.462 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-06-06T04:35:02.462 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.462 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.462 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.462 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-06-06T04:35:02.462 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.463 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.463 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.463 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-06-06T04:35:02.463 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.463 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.463 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.464 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-06-06T04:35:02.464 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.464 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.464 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.464 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-06-06T04:35:02.464 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.464 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.465 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.465 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-06-06T04:35:02.466 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.466 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.466 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.466 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-06-06T04:35:02.466 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.466 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.467 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.467 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-06-06T04:35:02.467 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.467 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.467 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.467 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-06-06T04:35:02.468 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.468 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.468 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.468 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-06-06T04:35:02.468 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.468 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.469 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.469 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-06-06T04:35:02.469 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.469 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.469 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.469 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-06-06T04:35:02.470 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.470 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.470 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.470 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-06-06T04:35:02.470 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.470 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.470 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.471 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-06-06T04:35:02.471 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.471 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.471 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.471 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-06-06T04:35:02.471 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.472 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.472 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.472 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-06-06T04:35:02.472 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.472 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.472 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.473 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-06-06T04:35:02.473 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.473 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.473 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.473 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-06-06T04:35:02.473 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.473 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.474 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.474 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-06-06T04:35:02.474 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.474 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.474 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.474 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-06-06T04:35:02.475 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.475 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.475 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.475 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-06-06T04:35:02.475 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.475 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.475 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.476 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-06-06T04:35:02.476 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.476 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.476 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.476 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-06-06T04:35:02.476 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.477 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.477 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.477 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-06-06T04:35:02.477 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-06-06T04:35:02.477 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-06-06T04:35:02.477 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.477 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.478 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.478 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:35:02.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.478 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-06-06T04:35:02.478 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-06-06T04:35:02.478 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 4, 2024-06-06T04:35:02.478 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 3, 2024-06-06T04:35:02.479 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 2, 2024-06-06T04:35:02.479 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-06-06T04:35:02.479 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-06-06T04:35:02.479 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-06-06T04:35:02.479 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 5878, 2024-06-06T04:35:02.479 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-06-06T04:35:02.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.479 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-06-06T04:35:02.480 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-06-06T04:35:02.480 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-06-06T04:35:02.480 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T04:35:02.480 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-06-06T04:35:02.480 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T04:35:02.480 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-06-06T04:35:02.480 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T04:35:02.480 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T04:35:02.480 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T04:35:02.481 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T04:35:02.481 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T04:35:02.481 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-06-06T04:35:02.481 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T04:35:02.481 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T04:35:02.481 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-06-06T04:35:02.481 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-06-06T04:35:02.481 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-06-06T04:35:02.482 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-06-06T04:35:02.482 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T04:35:02.482 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T04:35:02.482 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T04:35:02.482 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T04:35:02.482 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T04:35:02.482 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-06-06T04:35:02.483 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-06-06T04:35:02.483 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-06-06T04:35:02.483 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T04:35:02.483 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T04:35:02.483 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-06-06T04:35:02.483 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-06-06T04:35:02.483 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-06-06T04:35:02.483 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-06-06T04:35:02.484 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T04:35:02.484 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-06-06T04:35:02.484 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T04:35:02.484 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-06-06T04:35:02.484 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 213479, 2024-06-06T04:35:02.484 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 60, 2024-06-06T04:35:02.484 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-06-06T04:35:02.484 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-06-06T04:35:02.485 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-06-06T04:35:02.485 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-06-06T04:35:02.485 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-06-06T04:35:02.485 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-06-06T04:35:02.485 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-06-06T04:35:02.485 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-06-06T04:35:02.485 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 8992, 2024-06-06T04:35:02.485 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 173, 2024-06-06T04:35:02.486 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T04:35:02.486 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-06-06T04:35:02.486 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-06-06T04:35:02.486 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-06-06T04:35:02.486 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 13331208, 2024-06-06T04:35:02.486 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 54528, 2024-06-06T04:35:02.486 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-06-06T04:35:02.486 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-06-06T04:35:02.487 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-06-06T04:35:02.487 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-06-06T04:35:02.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.487 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-06-06T04:35:02.487 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-06-06T04:35:02.487 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-06-06T04:35:02.487 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 1297, 2024-06-06T04:35:02.487 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 26, 2024-06-06T04:35:02.488 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 4, 2024-06-06T04:35:02.488 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 1297, 2024-06-06T04:35:02.488 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-06-06T04:35:02.488 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1297, 2024-06-06T04:35:02.488 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.978979329, 2024-06-06T04:35:02.488 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000754802 2024-06-06T04:35:02.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.488 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-06-06T04:35:02.489 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-06-06T04:35:02.489 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1297, 2024-06-06T04:35:02.489 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1298 2024-06-06T04:35:02.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.489 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 1297, 2024-06-06T04:35:02.489 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 1297, 2024-06-06T04:35:02.489 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-06-06T04:35:02.489 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-06-06T04:35:02.490 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-06-06T04:35:02.490 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 813, 2024-06-06T04:35:02.490 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-06-06T04:35:02.490 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 481, 2024-06-06T04:35:02.490 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-06-06T04:35:02.490 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-06-06T04:35:02.490 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-06-06T04:35:02.490 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-06-06T04:35:02.491 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-06-06T04:35:02.491 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-06-06T04:35:02.491 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-06-06T04:35:02.491 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-06-06T04:35:02.491 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-06-06T04:35:02.491 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-06-06T04:35:02.491 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 2, 2024-06-06T04:35:02.491 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-06-06T04:35:02.492 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-06-06T04:35:02.492 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-06-06T04:35:02.492 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-06-06T04:35:02.492 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-06-06T04:35:02.492 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-06-06T04:35:02.492 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-06-06T04:35:02.492 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T04:35:02.492 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-06-06T04:35:02.493 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-06-06T04:35:02.493 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-06-06T04:35:02.493 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-06-06T04:35:02.493 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-06-06T04:35:02.493 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-06-06T04:35:02.493 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-06-06T04:35:02.493 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-06-06T04:35:02.493 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-06-06T04:35:02.493 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-06-06T04:35:02.494 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-06-06T04:35:02.494 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-06-06T04:35:02.494 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-06-06T04:35:02.494 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-06-06T04:35:02.494 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-06-06T04:35:02.494 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-06-06T04:35:02.494 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-06-06T04:35:02.494 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-06-06T04:35:02.495 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-06-06T04:35:02.495 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 100, 2024-06-06T04:35:02.495 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-06-06T04:35:02.495 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 2, 2024-06-06T04:35:02.495 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 8, 2024-06-06T04:35:02.495 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 8, 2024-06-06T04:35:02.495 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-06-06T04:35:02.495 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-06-06T04:35:02.496 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-06-06T04:35:02.496 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-06-06T04:35:02.496 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-06-06T04:35:02.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.496 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-06-06T04:35:02.496 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-06-06T04:35:02.496 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-06-06T04:35:02.496 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-06-06T04:35:02.497 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-06-06T04:35:02.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.497 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-06-06T04:35:02.497 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-06-06T04:35:02.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.497 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-06-06T04:35:02.497 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-06-06T04:35:02.497 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T04:35:02.498 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-06-06T04:35:02.498 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-06-06T04:35:02.498 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-06-06T04:35:02.498 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-06-06T04:35:02.498 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-06-06T04:35:02.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.498 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T04:35:02.498 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-06-06T04:35:02.499 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-06-06T04:35:02.499 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-06-06T04:35:02.499 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 1449, 2024-06-06T04:35:02.499 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 42827649, 2024-06-06T04:35:02.499 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:02.499 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 1449, 2024-06-06T04:35:02.499 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-06-06T04:35:02.499 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-06-06T04:35:02.500 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 1449, 2024-06-06T04:35:02.500 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 42827649, 2024-06-06T04:35:02.500 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-06-06T04:35:02.500 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.500 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.500 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.500 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:35:02.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.501 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-06-06T04:35:02.501 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-06-06T04:35:02.501 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-06-06T04:35:02.501 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-06-06T04:35:02.501 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-06-06T04:35:02.501 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-06-06T04:35:02.501 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:02.501 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-06-06T04:35:02.502 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 1297, 2024-06-06T04:35:02.502 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 41831616, 2024-06-06T04:35:02.502 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 1297, 2024-06-06T04:35:02.502 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 41831616, 2024-06-06T04:35:02.502 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-06-06T04:35:02.502 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.502 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.503 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.503 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:35:02.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.503 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-06-06T04:35:02.503 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-06-06T04:35:02.503 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-06-06T04:35:02.503 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-06-06T04:35:02.503 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-06-06T04:35:02.504 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-06-06T04:35:02.504 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:02.504 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-06-06T04:35:02.504 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 1297, 2024-06-06T04:35:02.504 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 1297, 2024-06-06T04:35:02.504 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 1297, 2024-06-06T04:35:02.504 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1297, 2024-06-06T04:35:02.504 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-06-06T04:35:02.504 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.505 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.505 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.505 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:35:02.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.505 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-06-06T04:35:02.505 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-06-06T04:35:02.505 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-06-06T04:35:02.505 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-06-06T04:35:02.506 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-06-06T04:35:02.506 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-06-06T04:35:02.506 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:02.506 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-06-06T04:35:02.506 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-06-06T04:35:02.506 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-06-06T04:35:02.506 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-06-06T04:35:02.506 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-06-06T04:35:02.507 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-06-06T04:35:02.507 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.507 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.507 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.507 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:35:02.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:02.507 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x55f843e566e0": { 2024-06-06T04:35:02.507 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-06-06T04:35:02.507 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-06-06T04:35:02.508 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-06-06T04:35:02.508 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-06-06T04:35:02.508 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-06-06T04:35:02.508 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:02.508 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-06-06T04:35:02.508 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-06-06T04:35:02.508 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-06-06T04:35:02.508 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-06-06T04:35:02.509 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-06-06T04:35:02.509 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-06-06T04:35:02.509 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:02.509 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:02.509 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:02.509 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:35:02.509 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:35:02.509 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-06-06T04:35:03.217 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-06-06T04:35:03.218 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-06-06T04:35:03.218 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-06-06T04:35:03.218 DEBUG:tasks.check_counter:Getting stats from k 2024-06-06T04:35:03.218 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-06-06T04:35:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T04:35:04.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:03 smithi045 ceph-mon[32299]: pgmap v539: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:35:04.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:03 smithi086 ceph-mon[36053]: pgmap v539: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:35:04.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:03 smithi154 ceph-mon[36428]: pgmap v539: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:35:05.376 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-06-06T04:35:05.377 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T04:35:05.377 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 4, 2024-06-06T04:35:05.377 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 4, 2024-06-06T04:35:05.377 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 2992, 2024-06-06T04:35:05.377 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 448, 2024-06-06T04:35:05.377 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-06-06T04:35:05.378 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 0, 2024-06-06T04:35:05.378 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.001346031, 2024-06-06T04:35:05.378 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000043706, 2024-06-06T04:35:05.378 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.000566721, 2024-06-06T04:35:05.378 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T04:35:05.378 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:05.378 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3, 2024-06-06T04:35:05.378 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000756290, 2024-06-06T04:35:05.379 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000252096 2024-06-06T04:35:05.379 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:05.379 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:05.379 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:05.379 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:05.379 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:05.379 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:05.379 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-06-06T04:35:05.380 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 448 2024-06-06T04:35:05.380 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:05.380 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T04:35:05.380 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1, 2024-06-06T04:35:05.380 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 105, 2024-06-06T04:35:05.380 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 160, 2024-06-06T04:35:05.380 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 96256, 2024-06-06T04:35:05.380 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 3, 2024-06-06T04:35:05.380 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-06-06T04:35:05.381 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.006753252, 2024-06-06T04:35:05.381 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.004597069, 2024-06-06T04:35:05.381 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.000847447, 2024-06-06T04:35:05.381 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T04:35:05.381 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:05.381 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 103, 2024-06-06T04:35:05.381 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.006959313, 2024-06-06T04:35:05.381 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000067566 2024-06-06T04:35:05.381 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:05.382 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:05.382 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:05.382 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:05.382 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:05.382 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:05.382 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-06T04:35:05.382 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 96256 2024-06-06T04:35:05.382 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:05.383 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T04:35:05.383 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 144, 2024-06-06T04:35:05.383 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 139, 2024-06-06T04:35:05.383 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 731904, 2024-06-06T04:35:05.383 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 84832, 2024-06-06T04:35:05.383 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 2, 2024-06-06T04:35:05.383 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-06-06T04:35:05.383 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.026026724, 2024-06-06T04:35:05.383 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.008362203, 2024-06-06T04:35:05.384 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.016806482, 2024-06-06T04:35:05.384 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.003703949, 2024-06-06T04:35:05.384 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:05.384 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 138, 2024-06-06T04:35:05.384 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.006635446, 2024-06-06T04:35:05.384 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000048082 2024-06-06T04:35:05.384 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:05.384 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:05.384 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:05.385 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:05.385 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:05.385 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:05.385 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 731904, 2024-06-06T04:35:05.385 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 84832 2024-06-06T04:35:05.385 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:05.385 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-06-06T04:35:05.385 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 0, 2024-06-06T04:35:05.386 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-06-06T04:35:05.386 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:05.386 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-06-06T04:35:05.386 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-06-06T04:35:05.386 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-06-06T04:35:05.386 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T04:35:05.386 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-06-06T04:35:05.386 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T04:35:05.387 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-06-06T04:35:05.387 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T04:35:05.387 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T04:35:05.387 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T04:35:05.387 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T04:35:05.387 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T04:35:05.387 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-06-06T04:35:05.387 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T04:35:05.387 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T04:35:05.388 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-06-06T04:35:05.388 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-06-06T04:35:05.388 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-06-06T04:35:05.388 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-06-06T04:35:05.388 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T04:35:05.388 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T04:35:05.388 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T04:35:05.388 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T04:35:05.389 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T04:35:05.389 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-06-06T04:35:05.389 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-06-06T04:35:05.389 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-06-06T04:35:05.389 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T04:35:05.389 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T04:35:05.389 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-06-06T04:35:05.389 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-06-06T04:35:05.390 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-06-06T04:35:05.390 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-06-06T04:35:05.390 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T04:35:05.390 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-06-06T04:35:05.390 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T04:35:05.390 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-06-06T04:35:05.390 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 148967, 2024-06-06T04:35:05.390 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 29, 2024-06-06T04:35:05.391 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-06-06T04:35:05.391 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-06-06T04:35:05.391 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-06-06T04:35:05.391 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-06-06T04:35:05.391 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-06-06T04:35:05.391 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-06-06T04:35:05.391 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-06-06T04:35:05.391 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-06-06T04:35:05.392 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 0, 2024-06-06T04:35:05.392 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 0, 2024-06-06T04:35:05.392 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T04:35:05.392 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-06-06T04:35:05.392 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-06-06T04:35:05.392 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-06-06T04:35:05.392 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 928, 2024-06-06T04:35:05.392 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2, 2024-06-06T04:35:05.392 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-06-06T04:35:05.393 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-06-06T04:35:05.393 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-06-06T04:35:05.393 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-06-06T04:35:05.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-06-06T04:35:05.393 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T04:35:05.393 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-06-06T04:35:05.393 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-06-06T04:35:05.394 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-06-06T04:35:05.394 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 145, 2024-06-06T04:35:05.394 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 711602, 2024-06-06T04:35:05.394 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:05.394 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 145, 2024-06-06T04:35:05.394 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-06-06T04:35:05.394 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-06-06T04:35:05.394 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 145, 2024-06-06T04:35:05.395 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 711602, 2024-06-06T04:35:05.395 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-06-06T04:35:05.395 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-06-06T04:35:05.395 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-06-06T04:35:05.395 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-06-06T04:35:05.395 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:35:05.395 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-06-06T04:35:05.395 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-06-06T04:35:06.106 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-06-06T04:35:06.106 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-06-06T04:35:06.106 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-06-06T04:35:06.107 DEBUG:tasks.check_counter:Getting stats from c 2024-06-06T04:35:06.107 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-06-06T04:35:06.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:05 smithi045 ceph-mon[32299]: pgmap v540: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-06T04:35:06.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:05 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:35:06.354 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T04:35:06.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:05 smithi086 ceph-mon[36053]: pgmap v540: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-06T04:35:06.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:05 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:35:06.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:05 smithi154 ceph-mon[36428]: pgmap v540: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-06T04:35:06.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:05 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T04:35:07.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:06 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:35:07.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:06 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:35:07.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:06 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:35:07.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:06 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:35:07.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:06 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:35:07.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:06 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:35:07.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:06 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T04:35:07.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:06 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T04:35:07.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:06 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' 2024-06-06T04:35:08.047 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:07 smithi154 ceph-mon[36428]: pgmap v541: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:35:08.182 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-06-06T04:35:08.182 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T04:35:08.182 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 666, 2024-06-06T04:35:08.182 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 491, 2024-06-06T04:35:08.182 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 195741, 2024-06-06T04:35:08.183 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 200535, 2024-06-06T04:35:08.183 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 10, 2024-06-06T04:35:08.183 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 3, 2024-06-06T04:35:08.183 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.057230742, 2024-06-06T04:35:08.183 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.019999974, 2024-06-06T04:35:08.183 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.039220329, 2024-06-06T04:35:08.183 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.003405578, 2024-06-06T04:35:08.183 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:08.183 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 489, 2024-06-06T04:35:08.184 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.027047082, 2024-06-06T04:35:08.184 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000055311 2024-06-06T04:35:08.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.184 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:08.184 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1084, 2024-06-06T04:35:08.184 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000261469, 2024-06-06T04:35:08.184 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000241 2024-06-06T04:35:08.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.185 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T04:35:08.185 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T04:35:08.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.185 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T04:35:08.185 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 239, 2024-06-06T04:35:08.185 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 225, 2024-06-06T04:35:08.185 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 800359, 2024-06-06T04:35:08.185 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 165453, 2024-06-06T04:35:08.186 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 11, 2024-06-06T04:35:08.186 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 5, 2024-06-06T04:35:08.186 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.033733196, 2024-06-06T04:35:08.186 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.010524789, 2024-06-06T04:35:08.186 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.023724942, 2024-06-06T04:35:08.186 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.004008127, 2024-06-06T04:35:08.186 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:08.187 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 225, 2024-06-06T04:35:08.187 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.019584586, 2024-06-06T04:35:08.187 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000087042 2024-06-06T04:35:08.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.187 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:08.188 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 49, 2024-06-06T04:35:08.188 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000006849, 2024-06-06T04:35:08.188 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000139 2024-06-06T04:35:08.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.188 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 789424, 2024-06-06T04:35:08.188 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 91312 2024-06-06T04:35:08.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.188 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T04:35:08.188 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 14, 2024-06-06T04:35:08.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 115, 2024-06-06T04:35:08.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 2925, 2024-06-06T04:35:08.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 310616, 2024-06-06T04:35:08.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 13, 2024-06-06T04:35:08.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 5, 2024-06-06T04:35:08.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.010538252, 2024-06-06T04:35:08.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.004571097, 2024-06-06T04:35:08.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.004229159, 2024-06-06T04:35:08.190 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000367649, 2024-06-06T04:35:08.190 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:08.190 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 113, 2024-06-06T04:35:08.190 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.014060059, 2024-06-06T04:35:08.190 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000124425 2024-06-06T04:35:08.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.190 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:08.191 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 6, 2024-06-06T04:35:08.191 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000516, 2024-06-06T04:35:08.191 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000086 2024-06-06T04:35:08.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.191 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-06T04:35:08.191 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 234352 2024-06-06T04:35:08.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.191 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-06-06T04:35:08.191 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-06-06T04:35:08.192 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-06-06T04:35:08.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.192 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-06-06T04:35:08.192 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-06-06T04:35:08.192 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-06-06T04:35:08.192 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 70, 2024-06-06T04:35:08.192 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.003406582, 2024-06-06T04:35:08.192 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000048665 2024-06-06T04:35:08.193 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:08.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.193 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-06-06T04:35:08.193 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-06-06T04:35:08.193 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-06-06T04:35:08.193 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 2, 2024-06-06T04:35:08.193 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000009699, 2024-06-06T04:35:08.194 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000004849 2024-06-06T04:35:08.194 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:08.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.194 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-06-06T04:35:08.194 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 10, 2024-06-06T04:35:08.194 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 9, 2024-06-06T04:35:08.194 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-06-06T04:35:08.194 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 9, 2024-06-06T04:35:08.195 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.003622261, 2024-06-06T04:35:08.195 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000402473 2024-06-06T04:35:08.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.195 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-06-06T04:35:08.195 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 1, 2024-06-06T04:35:08.195 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 0, 2024-06-06T04:35:08.195 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 0, 2024-06-06T04:35:08.195 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 11, 2024-06-06T04:35:08.196 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 0, 2024-06-06T04:35:08.196 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 0, 2024-06-06T04:35:08.196 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 27, 2024-06-06T04:35:08.196 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 0, 2024-06-06T04:35:08.196 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 27, 2024-06-06T04:35:08.196 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 0, 2024-06-06T04:35:08.196 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 27, 2024-06-06T04:35:08.196 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 0, 2024-06-06T04:35:08.196 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 7, 2024-06-06T04:35:08.197 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 7, 2024-06-06T04:35:08.197 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 3, 2024-06-06T04:35:08.197 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 19, 2024-06-06T04:35:08.197 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 10, 2024-06-06T04:35:08.197 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 0, 2024-06-06T04:35:08.197 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 2, 2024-06-06T04:35:08.197 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 0, 2024-06-06T04:35:08.198 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-06-06T04:35:08.198 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 0, 2024-06-06T04:35:08.198 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 0, 2024-06-06T04:35:08.198 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-06-06T04:35:08.198 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 0, 2024-06-06T04:35:08.198 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 0, 2024-06-06T04:35:08.198 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 1, 2024-06-06T04:35:08.199 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 8, 2024-06-06T04:35:08.199 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-06-06T04:35:08.199 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 6, 2024-06-06T04:35:08.199 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-06-06T04:35:08.199 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 3381, 2024-06-06T04:35:08.199 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 24837330, 2024-06-06T04:35:08.199 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 0, 2024-06-06T04:35:08.200 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 6, 2024-06-06T04:35:08.200 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-06-06T04:35:08.200 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T04:35:08.200 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-06-06T04:35:08.200 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 4, 2024-06-06T04:35:08.200 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T04:35:08.200 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 4, 2024-06-06T04:35:08.200 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 2, 2024-06-06T04:35:08.200 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 0, 2024-06-06T04:35:08.201 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 1, 2024-06-06T04:35:08.201 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 0, 2024-06-06T04:35:08.201 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T04:35:08.201 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T04:35:08.201 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T04:35:08.201 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 5, 2024-06-06T04:35:08.201 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 0, 2024-06-06T04:35:08.201 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 0, 2024-06-06T04:35:08.202 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 1 2024-06-06T04:35:08.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.202 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-06-06T04:35:08.202 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 0, 2024-06-06T04:35:08.202 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 0, 2024-06-06T04:35:08.202 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 0, 2024-06-06T04:35:08.202 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 3, 2024-06-06T04:35:08.202 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 0, 2024-06-06T04:35:08.202 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 0, 2024-06-06T04:35:08.203 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-06-06T04:35:08.203 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-06-06T04:35:08.203 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 0, 2024-06-06T04:35:08.203 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 0, 2024-06-06T04:35:08.203 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-06-06T04:35:08.203 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-06-06T04:35:08.203 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-06-06T04:35:08.203 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-06-06T04:35:08.204 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-06-06T04:35:08.204 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-06-06T04:35:08.204 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-06-06T04:35:08.204 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 0, 2024-06-06T04:35:08.204 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 0, 2024-06-06T04:35:08.204 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T04:35:08.204 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 0, 2024-06-06T04:35:08.204 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-06-06T04:35:08.205 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 0, 2024-06-06T04:35:08.205 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-06-06T04:35:08.205 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0 2024-06-06T04:35:08.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.205 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-06-06T04:35:08.205 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 0, 2024-06-06T04:35:08.205 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 11, 2024-06-06T04:35:08.206 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 0, 2024-06-06T04:35:08.206 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 0, 2024-06-06T04:35:08.206 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 11, 2024-06-06T04:35:08.206 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-06-06T04:35:08.206 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 0, 2024-06-06T04:35:08.206 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 2, 2024-06-06T04:35:08.206 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 0, 2024-06-06T04:35:08.206 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 0, 2024-06-06T04:35:08.207 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 2, 2024-06-06T04:35:08.207 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 2, 2024-06-06T04:35:08.207 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-06-06T04:35:08.207 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 0, 2024-06-06T04:35:08.207 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 4194304, 2024-06-06T04:35:08.207 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 4213748, 2024-06-06T04:35:08.207 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 0, 2024-06-06T04:35:08.207 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-06-06T04:35:08.208 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 9, 2024-06-06T04:35:08.208 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.019690812, 2024-06-06T04:35:08.208 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.002187868 2024-06-06T04:35:08.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.208 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 0 2024-06-06T04:35:08.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.208 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-06-06T04:35:08.208 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 28, 2024-06-06T04:35:08.208 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 28, 2024-06-06T04:35:08.209 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 0, 2024-06-06T04:35:08.209 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 25, 2024-06-06T04:35:08.209 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 26, 2024-06-06T04:35:08.209 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 1, 2024-06-06T04:35:08.209 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 27, 2024-06-06T04:35:08.209 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 28, 2024-06-06T04:35:08.209 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 1, 2024-06-06T04:35:08.209 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 7, 2024-06-06T04:35:08.210 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 8, 2024-06-06T04:35:08.210 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 1, 2024-06-06T04:35:08.210 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 52608, 2024-06-06T04:35:08.210 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 223612 2024-06-06T04:35:08.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.210 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-06-06T04:35:08.210 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 11, 2024-06-06T04:35:08.210 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 0, 2024-06-06T04:35:08.210 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 10, 2024-06-06T04:35:08.211 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 9, 2024-06-06T04:35:08.211 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 0, 2024-06-06T04:35:08.211 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-06-06T04:35:08.211 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.211 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.211 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.211 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-06-06T04:35:08.212 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1, 2024-06-06T04:35:08.212 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000209150, 2024-06-06T04:35:08.212 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000209150 2024-06-06T04:35:08.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.212 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-06-06T04:35:08.212 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.212 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.212 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.213 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-06-06T04:35:08.213 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.213 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.213 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.213 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.213 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-06-06T04:35:08.213 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 6, 2024-06-06T04:35:08.213 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.002445923, 2024-06-06T04:35:08.214 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000407653 2024-06-06T04:35:08.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.214 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-06-06T04:35:08.214 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.214 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.214 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.214 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-06-06T04:35:08.215 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.215 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.215 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.215 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-06-06T04:35:08.215 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.215 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.215 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.216 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-06-06T04:35:08.216 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.216 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.216 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.216 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-06-06T04:35:08.216 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.216 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.217 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.217 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-06-06T04:35:08.217 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.217 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.217 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.218 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-06-06T04:35:08.218 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.218 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.218 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.219 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-06-06T04:35:08.219 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.219 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.219 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.219 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-06-06T04:35:08.219 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.220 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.220 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.220 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.220 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-06-06T04:35:08.220 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1, 2024-06-06T04:35:08.220 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000476821, 2024-06-06T04:35:08.220 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000476821 2024-06-06T04:35:08.220 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.220 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-06-06T04:35:08.221 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1, 2024-06-06T04:35:08.221 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000490367, 2024-06-06T04:35:08.221 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000490367 2024-06-06T04:35:08.221 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.221 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-06-06T04:35:08.221 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.221 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.221 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.222 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.222 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-06-06T04:35:08.222 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.222 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.222 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.222 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.222 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-06-06T04:35:08.222 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.223 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.223 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.223 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.223 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-06-06T04:35:08.223 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.223 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.223 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.223 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.223 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-06-06T04:35:08.224 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.224 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.224 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.224 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.224 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-06-06T04:35:08.224 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.224 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.224 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.225 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.225 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-06-06T04:35:08.225 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.225 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.225 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.225 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.225 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-06-06T04:35:08.225 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.226 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.226 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.226 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.226 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-06-06T04:35:08.226 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.226 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.226 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.227 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.227 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-06-06T04:35:08.227 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.227 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.227 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.227 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.227 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-06-06T04:35:08.227 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.228 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.228 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.228 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.228 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-06-06T04:35:08.228 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.228 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.228 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.228 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.229 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-06-06T04:35:08.229 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.229 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.229 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.229 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.229 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-06-06T04:35:08.229 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-06-06T04:35:08.229 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-06-06T04:35:08.230 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.230 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.230 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.230 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:08.230 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.230 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-06-06T04:35:08.230 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 1, 2024-06-06T04:35:08.230 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 2, 2024-06-06T04:35:08.231 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 1, 2024-06-06T04:35:08.231 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 1, 2024-06-06T04:35:08.231 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-06-06T04:35:08.231 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 8, 2024-06-06T04:35:08.231 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 8, 2024-06-06T04:35:08.231 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 226, 2024-06-06T04:35:08.231 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-06-06T04:35:08.232 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.232 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-06-06T04:35:08.232 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-06-06T04:35:08.232 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-06-06T04:35:08.232 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T04:35:08.232 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-06-06T04:35:08.232 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T04:35:08.233 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-06-06T04:35:08.233 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T04:35:08.233 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T04:35:08.233 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T04:35:08.233 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T04:35:08.233 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T04:35:08.233 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-06-06T04:35:08.233 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T04:35:08.233 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T04:35:08.234 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-06-06T04:35:08.234 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-06-06T04:35:08.234 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-06-06T04:35:08.234 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-06-06T04:35:08.234 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T04:35:08.234 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T04:35:08.234 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T04:35:08.235 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T04:35:08.235 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T04:35:08.235 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-06-06T04:35:08.235 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-06-06T04:35:08.235 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-06-06T04:35:08.235 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T04:35:08.235 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T04:35:08.235 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-06-06T04:35:08.236 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-06-06T04:35:08.236 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-06-06T04:35:08.236 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-06-06T04:35:08.236 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T04:35:08.236 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-06-06T04:35:08.236 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T04:35:08.236 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-06-06T04:35:08.236 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 264303, 2024-06-06T04:35:08.236 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 92, 2024-06-06T04:35:08.237 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-06-06T04:35:08.237 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-06-06T04:35:08.237 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-06-06T04:35:08.237 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-06-06T04:35:08.237 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-06-06T04:35:08.237 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-06-06T04:35:08.237 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-06-06T04:35:08.237 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-06-06T04:35:08.238 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 8992, 2024-06-06T04:35:08.238 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 173, 2024-06-06T04:35:08.238 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T04:35:08.238 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-06-06T04:35:08.238 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-06-06T04:35:08.238 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-06-06T04:35:08.238 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 117045, 2024-06-06T04:35:08.238 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 540, 2024-06-06T04:35:08.239 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-06-06T04:35:08.239 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-06-06T04:35:08.239 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-06-06T04:35:08.239 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-06-06T04:35:08.239 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.239 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-06-06T04:35:08.239 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-06-06T04:35:08.239 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-06-06T04:35:08.239 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 56, 2024-06-06T04:35:08.240 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 31159, 2024-06-06T04:35:08.240 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 33, 2024-06-06T04:35:08.240 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 56, 2024-06-06T04:35:08.240 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-06-06T04:35:08.240 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 56, 2024-06-06T04:35:08.240 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.225993696, 2024-06-06T04:35:08.240 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.004035601 2024-06-06T04:35:08.240 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.240 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-06-06T04:35:08.241 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-06-06T04:35:08.241 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 56, 2024-06-06T04:35:08.241 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 81 2024-06-06T04:35:08.241 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.241 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 56, 2024-06-06T04:35:08.241 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 6, 2024-06-06T04:35:08.241 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 50, 2024-06-06T04:35:08.241 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-06-06T04:35:08.242 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-06-06T04:35:08.242 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 0, 2024-06-06T04:35:08.242 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 10, 2024-06-06T04:35:08.242 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 0, 2024-06-06T04:35:08.242 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 9, 2024-06-06T04:35:08.242 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 9, 2024-06-06T04:35:08.242 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-06-06T04:35:08.242 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-06-06T04:35:08.243 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 0, 2024-06-06T04:35:08.243 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 0, 2024-06-06T04:35:08.243 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 6, 2024-06-06T04:35:08.243 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-06-06T04:35:08.243 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-06-06T04:35:08.243 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-06-06T04:35:08.243 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 6, 2024-06-06T04:35:08.243 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 20, 2024-06-06T04:35:08.243 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-06-06T04:35:08.244 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-06-06T04:35:08.244 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-06-06T04:35:08.244 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-06-06T04:35:08.244 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-06-06T04:35:08.244 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-06-06T04:35:08.244 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T04:35:08.244 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-06-06T04:35:08.244 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-06-06T04:35:08.244 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-06-06T04:35:08.245 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-06-06T04:35:08.245 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-06-06T04:35:08.245 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-06-06T04:35:08.245 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-06-06T04:35:08.245 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-06-06T04:35:08.245 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-06-06T04:35:08.245 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-06-06T04:35:08.245 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-06-06T04:35:08.245 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-06-06T04:35:08.246 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-06-06T04:35:08.246 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-06-06T04:35:08.246 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-06-06T04:35:08.246 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-06-06T04:35:08.246 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-06-06T04:35:08.246 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-06-06T04:35:08.246 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-06-06T04:35:08.246 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 100, 2024-06-06T04:35:08.247 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-06-06T04:35:08.247 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 2, 2024-06-06T04:35:08.247 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 10, 2024-06-06T04:35:08.247 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 10, 2024-06-06T04:35:08.247 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 0, 2024-06-06T04:35:08.247 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-06-06T04:35:08.247 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 19, 2024-06-06T04:35:08.247 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 0, 2024-06-06T04:35:08.247 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 2 2024-06-06T04:35:08.248 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.248 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-06-06T04:35:08.248 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 1, 2024-06-06T04:35:08.248 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-06-06T04:35:08.248 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-06-06T04:35:08.248 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-06-06T04:35:08.248 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.248 INFO:teuthology.orchestra.run.smithi154.stdout: "osd-slow-ops": { 2024-06-06T04:35:08.249 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 0 2024-06-06T04:35:08.249 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.249 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-06-06T04:35:08.249 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-06-06T04:35:08.249 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T04:35:08.249 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-06-06T04:35:08.249 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 0, 2024-06-06T04:35:08.249 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 0, 2024-06-06T04:35:08.249 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 0, 2024-06-06T04:35:08.250 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-06-06T04:35:08.250 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.250 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T04:35:08.250 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-06-06T04:35:08.250 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-06-06T04:35:08.250 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-06-06T04:35:08.250 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 915, 2024-06-06T04:35:08.251 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 904010, 2024-06-06T04:35:08.251 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:08.251 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 915, 2024-06-06T04:35:08.251 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-06-06T04:35:08.251 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-06-06T04:35:08.251 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 915, 2024-06-06T04:35:08.251 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 904010, 2024-06-06T04:35:08.251 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-06-06T04:35:08.252 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.252 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.252 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.252 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:08.252 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.252 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-06-06T04:35:08.252 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-06-06T04:35:08.252 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-06-06T04:35:08.253 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-06-06T04:35:08.253 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-06-06T04:35:08.253 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-06-06T04:35:08.253 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:08.253 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-06-06T04:35:08.253 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 56, 2024-06-06T04:35:08.253 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 31159, 2024-06-06T04:35:08.253 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 56, 2024-06-06T04:35:08.253 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 31159, 2024-06-06T04:35:08.254 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-06-06T04:35:08.254 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.254 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.254 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.254 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:08.254 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.254 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-06-06T04:35:08.254 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-06-06T04:35:08.255 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-06-06T04:35:08.255 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-06-06T04:35:08.255 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-06-06T04:35:08.255 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-06-06T04:35:08.255 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:08.255 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-06-06T04:35:08.255 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 56, 2024-06-06T04:35:08.255 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 56, 2024-06-06T04:35:08.256 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 56, 2024-06-06T04:35:08.256 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 56, 2024-06-06T04:35:08.256 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-06-06T04:35:08.256 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.256 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.256 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.256 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:08.256 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.256 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-06-06T04:35:08.257 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-06-06T04:35:08.257 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-06-06T04:35:08.257 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-06-06T04:35:08.257 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-06-06T04:35:08.257 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-06-06T04:35:08.257 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:08.257 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-06-06T04:35:08.257 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-06-06T04:35:08.258 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-06-06T04:35:08.258 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 0, 2024-06-06T04:35:08.258 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 0, 2024-06-06T04:35:08.258 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-06-06T04:35:08.258 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.258 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.258 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.258 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:08.259 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:08.259 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x557f661b2500": { 2024-06-06T04:35:08.259 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-06-06T04:35:08.259 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-06-06T04:35:08.259 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-06-06T04:35:08.259 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 11, 2024-06-06T04:35:08.259 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 19444, 2024-06-06T04:35:08.259 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:08.259 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 11, 2024-06-06T04:35:08.260 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-06-06T04:35:08.260 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-06-06T04:35:08.260 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 9, 2024-06-06T04:35:08.260 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 19444, 2024-06-06T04:35:08.260 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-06-06T04:35:08.260 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:08.260 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:08.260 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:08.260 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:08.261 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:08.261 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-06-06T04:35:08.276 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:07 smithi086 ceph-mon[36053]: pgmap v541: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:35:08.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:07 smithi045 ceph-mon[32299]: pgmap v541: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:35:08.916 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-06-06T04:35:08.916 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-06-06T04:35:08.916 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-06-06T04:35:08.916 DEBUG:tasks.check_counter:Getting stats from f 2024-06-06T04:35:08.916 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-06-06T04:35:09.171 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T04:35:10.314 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:09 smithi154 ceph-mon[36428]: pgmap v542: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:35:10.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:09 smithi086 ceph-mon[36053]: pgmap v542: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:35:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:09 smithi045 ceph-mon[32299]: pgmap v542: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:35:10.914 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-06-06T04:35:10.915 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T04:35:10.915 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 1054, 2024-06-06T04:35:10.915 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 1056, 2024-06-06T04:35:10.915 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 409035, 2024-06-06T04:35:10.915 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 335179, 2024-06-06T04:35:10.915 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 6, 2024-06-06T04:35:10.916 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 2, 2024-06-06T04:35:10.916 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.091577027, 2024-06-06T04:35:10.916 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.030141921, 2024-06-06T04:35:10.916 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.023378931, 2024-06-06T04:35:10.916 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.030385234, 2024-06-06T04:35:10.916 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:10.916 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1054, 2024-06-06T04:35:10.916 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.038622690, 2024-06-06T04:35:10.916 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000036643 2024-06-06T04:35:10.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.917 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:10.917 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.917 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.917 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.917 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T04:35:10.917 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T04:35:10.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.918 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T04:35:10.918 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 3, 2024-06-06T04:35:10.918 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 106, 2024-06-06T04:35:10.918 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 716, 2024-06-06T04:35:10.918 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 236954, 2024-06-06T04:35:10.918 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 4, 2024-06-06T04:35:10.918 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 3, 2024-06-06T04:35:10.918 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.006671808, 2024-06-06T04:35:10.918 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.004327988, 2024-06-06T04:35:10.919 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.001173967, 2024-06-06T04:35:10.919 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000086536, 2024-06-06T04:35:10.919 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:10.919 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 104, 2024-06-06T04:35:10.919 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.007010331, 2024-06-06T04:35:10.919 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000067407 2024-06-06T04:35:10.919 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.919 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:10.919 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.920 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.920 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.920 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-06T04:35:10.920 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 236256 2024-06-06T04:35:10.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.920 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T04:35:10.920 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 160, 2024-06-06T04:35:10.920 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 147, 2024-06-06T04:35:10.921 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 775804, 2024-06-06T04:35:10.921 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 85502, 2024-06-06T04:35:10.921 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 3, 2024-06-06T04:35:10.921 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 2, 2024-06-06T04:35:10.921 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.026857575, 2024-06-06T04:35:10.921 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.008644754, 2024-06-06T04:35:10.921 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.018573839, 2024-06-06T04:35:10.921 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.003412426, 2024-06-06T04:35:10.921 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:10.922 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 147, 2024-06-06T04:35:10.922 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.009434108, 2024-06-06T04:35:10.922 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000064177 2024-06-06T04:35:10.922 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.922 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:10.922 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.922 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.922 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.922 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.923 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 775536, 2024-06-06T04:35:10.923 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 85184 2024-06-06T04:35:10.923 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.923 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-06-06T04:35:10.923 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-06-06T04:35:10.923 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-06-06T04:35:10.923 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.923 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-06-06T04:35:10.923 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-06-06T04:35:10.924 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-06-06T04:35:10.924 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1871, 2024-06-06T04:35:10.924 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.051607686, 2024-06-06T04:35:10.924 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000027582 2024-06-06T04:35:10.924 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:10.924 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.924 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-06-06T04:35:10.924 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-06-06T04:35:10.924 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-06-06T04:35:10.924 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.925 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.925 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.925 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:10.925 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.925 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-06-06T04:35:10.925 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 0, 2024-06-06T04:35:10.925 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 0, 2024-06-06T04:35:10.925 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-06-06T04:35:10.925 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.926 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.926 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.926 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.926 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-06-06T04:35:10.926 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 0, 2024-06-06T04:35:10.926 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 0, 2024-06-06T04:35:10.926 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 0, 2024-06-06T04:35:10.926 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 0, 2024-06-06T04:35:10.926 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 0, 2024-06-06T04:35:10.927 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 0, 2024-06-06T04:35:10.927 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 29, 2024-06-06T04:35:10.927 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 0, 2024-06-06T04:35:10.927 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 29, 2024-06-06T04:35:10.927 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 0, 2024-06-06T04:35:10.927 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 29, 2024-06-06T04:35:10.927 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 0, 2024-06-06T04:35:10.927 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 0, 2024-06-06T04:35:10.927 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 0, 2024-06-06T04:35:10.927 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 3, 2024-06-06T04:35:10.928 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 0, 2024-06-06T04:35:10.928 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 0, 2024-06-06T04:35:10.928 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 0, 2024-06-06T04:35:10.928 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 0, 2024-06-06T04:35:10.928 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 0, 2024-06-06T04:35:10.928 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-06-06T04:35:10.928 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 0, 2024-06-06T04:35:10.928 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 0, 2024-06-06T04:35:10.928 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-06-06T04:35:10.929 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 0, 2024-06-06T04:35:10.929 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 0, 2024-06-06T04:35:10.929 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 0, 2024-06-06T04:35:10.929 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 0, 2024-06-06T04:35:10.929 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-06-06T04:35:10.929 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 0, 2024-06-06T04:35:10.929 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-06-06T04:35:10.929 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 0, 2024-06-06T04:35:10.929 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 0, 2024-06-06T04:35:10.930 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 0, 2024-06-06T04:35:10.930 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T04:35:10.930 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-06-06T04:35:10.930 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T04:35:10.930 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-06-06T04:35:10.930 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 0, 2024-06-06T04:35:10.930 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T04:35:10.930 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T04:35:10.930 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 0, 2024-06-06T04:35:10.931 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 0, 2024-06-06T04:35:10.931 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T04:35:10.931 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 0, 2024-06-06T04:35:10.931 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T04:35:10.931 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T04:35:10.931 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T04:35:10.931 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 0, 2024-06-06T04:35:10.931 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 0, 2024-06-06T04:35:10.931 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 0, 2024-06-06T04:35:10.932 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 0 2024-06-06T04:35:10.932 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.932 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-06-06T04:35:10.932 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 0, 2024-06-06T04:35:10.932 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 0, 2024-06-06T04:35:10.932 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 0, 2024-06-06T04:35:10.932 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 0, 2024-06-06T04:35:10.932 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 0, 2024-06-06T04:35:10.932 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 0, 2024-06-06T04:35:10.933 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-06-06T04:35:10.933 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-06-06T04:35:10.933 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 0, 2024-06-06T04:35:10.933 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 0, 2024-06-06T04:35:10.933 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-06-06T04:35:10.933 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-06-06T04:35:10.933 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-06-06T04:35:10.933 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-06-06T04:35:10.934 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-06-06T04:35:10.934 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-06-06T04:35:10.934 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-06-06T04:35:10.934 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 0, 2024-06-06T04:35:10.934 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 0, 2024-06-06T04:35:10.934 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T04:35:10.934 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 0, 2024-06-06T04:35:10.934 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-06-06T04:35:10.934 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 0, 2024-06-06T04:35:10.935 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-06-06T04:35:10.935 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0 2024-06-06T04:35:10.935 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.935 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-06-06T04:35:10.935 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 0, 2024-06-06T04:35:10.935 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 0, 2024-06-06T04:35:10.935 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 0, 2024-06-06T04:35:10.935 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 0, 2024-06-06T04:35:10.935 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 77, 2024-06-06T04:35:10.936 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-06-06T04:35:10.936 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 0, 2024-06-06T04:35:10.936 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 0, 2024-06-06T04:35:10.936 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 0, 2024-06-06T04:35:10.936 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 0, 2024-06-06T04:35:10.936 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 2, 2024-06-06T04:35:10.936 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 2, 2024-06-06T04:35:10.936 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-06-06T04:35:10.936 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 0, 2024-06-06T04:35:10.937 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 4194304, 2024-06-06T04:35:10.937 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 4300926, 2024-06-06T04:35:10.937 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 4300352, 2024-06-06T04:35:10.937 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-06-06T04:35:10.937 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.937 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.937 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.937 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.938 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 77 2024-06-06T04:35:10.938 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.938 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-06-06T04:35:10.938 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 21, 2024-06-06T04:35:10.938 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 36, 2024-06-06T04:35:10.938 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 15, 2024-06-06T04:35:10.938 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 12, 2024-06-06T04:35:10.938 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 16, 2024-06-06T04:35:10.938 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 4, 2024-06-06T04:35:10.939 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 29, 2024-06-06T04:35:10.939 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 44, 2024-06-06T04:35:10.939 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 15, 2024-06-06T04:35:10.939 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 0, 2024-06-06T04:35:10.939 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 0, 2024-06-06T04:35:10.939 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 0, 2024-06-06T04:35:10.939 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 50396, 2024-06-06T04:35:10.939 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 215420 2024-06-06T04:35:10.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.940 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-06-06T04:35:10.940 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 0, 2024-06-06T04:35:10.940 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 0, 2024-06-06T04:35:10.940 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 0, 2024-06-06T04:35:10.940 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 0, 2024-06-06T04:35:10.940 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 0, 2024-06-06T04:35:10.940 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-06-06T04:35:10.940 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.940 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.941 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.941 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-06-06T04:35:10.941 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.941 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.941 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.942 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-06-06T04:35:10.942 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.942 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.942 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.942 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-06-06T04:35:10.942 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.942 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.942 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.943 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-06-06T04:35:10.943 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.943 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.943 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.943 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-06-06T04:35:10.943 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.944 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.944 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.944 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.944 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-06-06T04:35:10.944 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.944 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.944 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.944 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.944 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-06-06T04:35:10.945 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.945 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.945 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.945 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.945 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-06-06T04:35:10.945 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.945 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.945 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.945 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.945 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-06-06T04:35:10.946 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.946 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.946 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.946 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.946 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-06-06T04:35:10.946 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.946 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.946 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.947 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.947 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-06-06T04:35:10.947 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.947 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.947 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.947 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.948 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-06-06T04:35:10.948 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.948 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.948 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.948 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.948 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-06-06T04:35:10.948 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.948 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.948 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.949 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.949 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-06-06T04:35:10.949 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.949 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.949 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.949 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.949 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-06-06T04:35:10.949 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.950 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.950 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.950 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.950 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-06-06T04:35:10.950 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.950 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.950 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.951 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.951 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-06-06T04:35:10.951 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.951 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.951 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.951 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.951 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-06-06T04:35:10.951 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.951 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.952 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.952 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.952 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-06-06T04:35:10.952 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.952 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.952 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.952 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.952 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-06-06T04:35:10.953 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.953 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.953 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.953 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.953 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-06-06T04:35:10.953 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.953 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.953 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.954 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.954 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-06-06T04:35:10.954 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.954 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.954 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.954 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.954 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-06-06T04:35:10.954 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.955 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.955 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.955 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.955 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-06-06T04:35:10.955 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.955 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.955 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.955 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.955 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-06-06T04:35:10.956 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.956 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.956 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.956 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.956 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-06-06T04:35:10.956 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.956 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.956 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.956 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.957 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-06-06T04:35:10.957 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.957 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.957 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.957 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-06-06T04:35:10.957 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.957 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.957 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.958 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-06-06T04:35:10.958 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-06-06T04:35:10.958 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-06-06T04:35:10.958 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.958 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.958 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.958 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:10.958 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.958 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-06-06T04:35:10.959 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 1, 2024-06-06T04:35:10.959 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 2, 2024-06-06T04:35:10.959 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 1, 2024-06-06T04:35:10.959 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 2, 2024-06-06T04:35:10.959 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-06-06T04:35:10.959 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 0, 2024-06-06T04:35:10.959 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 0, 2024-06-06T04:35:10.959 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 5936, 2024-06-06T04:35:10.959 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-06-06T04:35:10.960 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.960 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-06-06T04:35:10.960 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-06-06T04:35:10.960 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-06-06T04:35:10.960 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T04:35:10.960 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-06-06T04:35:10.960 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T04:35:10.960 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-06-06T04:35:10.960 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T04:35:10.961 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T04:35:10.961 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T04:35:10.961 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T04:35:10.961 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T04:35:10.961 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-06-06T04:35:10.961 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T04:35:10.961 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T04:35:10.961 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-06-06T04:35:10.962 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-06-06T04:35:10.962 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-06-06T04:35:10.962 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-06-06T04:35:10.962 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T04:35:10.962 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T04:35:10.962 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T04:35:10.962 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T04:35:10.962 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T04:35:10.963 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-06-06T04:35:10.963 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-06-06T04:35:10.963 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-06-06T04:35:10.963 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T04:35:10.963 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T04:35:10.963 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-06-06T04:35:10.963 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-06-06T04:35:10.963 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-06-06T04:35:10.964 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-06-06T04:35:10.964 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T04:35:10.964 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-06-06T04:35:10.964 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T04:35:10.964 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-06-06T04:35:10.964 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 181320, 2024-06-06T04:35:10.964 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 47, 2024-06-06T04:35:10.964 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-06-06T04:35:10.965 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-06-06T04:35:10.965 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-06-06T04:35:10.965 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-06-06T04:35:10.965 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-06-06T04:35:10.965 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-06-06T04:35:10.965 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-06-06T04:35:10.965 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-06-06T04:35:10.965 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 8992, 2024-06-06T04:35:10.965 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 173, 2024-06-06T04:35:10.966 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T04:35:10.966 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-06-06T04:35:10.966 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-06-06T04:35:10.966 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-06-06T04:35:10.966 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 81742, 2024-06-06T04:35:10.966 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 348, 2024-06-06T04:35:10.966 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-06-06T04:35:10.966 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-06-06T04:35:10.967 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-06-06T04:35:10.967 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-06-06T04:35:10.967 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.967 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-06-06T04:35:10.967 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-06-06T04:35:10.967 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-06-06T04:35:10.967 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 1057, 2024-06-06T04:35:10.967 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 21, 2024-06-06T04:35:10.968 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 3, 2024-06-06T04:35:10.968 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 1057, 2024-06-06T04:35:10.968 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-06-06T04:35:10.968 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1057, 2024-06-06T04:35:10.968 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.530985346, 2024-06-06T04:35:10.968 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000502351 2024-06-06T04:35:10.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.968 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-06-06T04:35:10.969 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-06-06T04:35:10.969 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1057, 2024-06-06T04:35:10.969 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 1058 2024-06-06T04:35:10.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.969 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 1057, 2024-06-06T04:35:10.969 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 1057, 2024-06-06T04:35:10.969 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 0, 2024-06-06T04:35:10.969 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-06-06T04:35:10.970 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-06-06T04:35:10.970 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 634, 2024-06-06T04:35:10.970 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 0, 2024-06-06T04:35:10.970 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 421, 2024-06-06T04:35:10.970 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 0, 2024-06-06T04:35:10.970 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 0, 2024-06-06T04:35:10.970 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-06-06T04:35:10.970 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-06-06T04:35:10.971 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 0, 2024-06-06T04:35:10.971 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 0, 2024-06-06T04:35:10.971 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 0, 2024-06-06T04:35:10.971 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-06-06T04:35:10.971 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-06-06T04:35:10.971 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-06-06T04:35:10.971 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 1, 2024-06-06T04:35:10.971 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 0, 2024-06-06T04:35:10.971 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-06-06T04:35:10.972 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-06-06T04:35:10.972 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-06-06T04:35:10.972 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-06-06T04:35:10.972 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-06-06T04:35:10.972 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-06-06T04:35:10.972 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T04:35:10.972 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-06-06T04:35:10.972 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-06-06T04:35:10.973 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-06-06T04:35:10.973 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-06-06T04:35:10.973 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-06-06T04:35:10.973 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-06-06T04:35:10.973 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-06-06T04:35:10.973 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-06-06T04:35:10.973 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-06-06T04:35:10.973 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-06-06T04:35:10.973 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-06-06T04:35:10.974 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-06-06T04:35:10.974 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-06-06T04:35:10.974 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-06-06T04:35:10.974 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-06-06T04:35:10.974 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-06-06T04:35:10.974 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-06-06T04:35:10.974 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-06-06T04:35:10.974 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-06-06T04:35:10.975 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 100, 2024-06-06T04:35:10.975 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-06-06T04:35:10.975 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 2, 2024-06-06T04:35:10.975 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 4, 2024-06-06T04:35:10.975 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 4, 2024-06-06T04:35:10.975 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 0, 2024-06-06T04:35:10.975 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-06-06T04:35:10.975 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 0, 2024-06-06T04:35:10.975 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 2, 2024-06-06T04:35:10.976 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 0 2024-06-06T04:35:10.976 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.976 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-06-06T04:35:10.976 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 0, 2024-06-06T04:35:10.976 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-06-06T04:35:10.976 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-06-06T04:35:10.976 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-06-06T04:35:10.976 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.976 INFO:teuthology.orchestra.run.smithi154.stdout: "osd-slow-ops": { 2024-06-06T04:35:10.977 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 0 2024-06-06T04:35:10.977 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.977 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-06-06T04:35:10.977 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-06-06T04:35:10.977 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T04:35:10.977 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-06-06T04:35:10.977 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 0, 2024-06-06T04:35:10.977 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 0, 2024-06-06T04:35:10.978 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 0, 2024-06-06T04:35:10.978 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-06-06T04:35:10.978 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.978 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T04:35:10.978 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-06-06T04:35:10.978 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-06-06T04:35:10.978 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-06-06T04:35:10.978 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 1213, 2024-06-06T04:35:10.978 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 1065489, 2024-06-06T04:35:10.978 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:10.979 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 1213, 2024-06-06T04:35:10.979 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-06-06T04:35:10.979 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-06-06T04:35:10.979 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1213, 2024-06-06T04:35:10.979 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 1065489, 2024-06-06T04:35:10.979 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-06-06T04:35:10.979 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.979 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.980 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.980 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:10.980 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.980 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-06-06T04:35:10.980 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-06-06T04:35:10.980 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-06-06T04:35:10.980 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-06-06T04:35:10.980 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-06-06T04:35:10.980 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-06-06T04:35:10.981 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:10.981 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-06-06T04:35:10.981 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 1057, 2024-06-06T04:35:10.981 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 106627, 2024-06-06T04:35:10.981 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1057, 2024-06-06T04:35:10.981 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 106627, 2024-06-06T04:35:10.981 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-06-06T04:35:10.981 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.981 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.982 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.982 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:10.982 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.982 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-06-06T04:35:10.982 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-06-06T04:35:10.982 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-06-06T04:35:10.982 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-06-06T04:35:10.982 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-06-06T04:35:10.982 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-06-06T04:35:10.983 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:10.983 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-06-06T04:35:10.983 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 1057, 2024-06-06T04:35:10.983 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 1057, 2024-06-06T04:35:10.983 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1057, 2024-06-06T04:35:10.983 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 1057, 2024-06-06T04:35:10.983 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-06-06T04:35:10.983 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.983 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.983 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.984 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:10.984 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.984 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-06-06T04:35:10.984 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-06-06T04:35:10.984 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-06-06T04:35:10.984 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-06-06T04:35:10.984 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-06-06T04:35:10.984 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-06-06T04:35:10.984 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:10.985 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-06-06T04:35:10.985 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-06-06T04:35:10.985 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-06-06T04:35:10.985 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 0, 2024-06-06T04:35:10.985 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 0, 2024-06-06T04:35:10.985 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-06-06T04:35:10.985 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.985 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.985 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.985 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:10.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:10.986 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x55c9463d86e0": { 2024-06-06T04:35:10.986 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-06-06T04:35:10.986 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-06-06T04:35:10.986 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-06-06T04:35:10.986 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-06-06T04:35:10.986 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-06-06T04:35:10.986 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:10.986 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-06-06T04:35:10.987 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-06-06T04:35:10.987 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-06-06T04:35:10.987 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 0, 2024-06-06T04:35:10.987 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 0, 2024-06-06T04:35:10.987 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-06-06T04:35:10.987 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:10.987 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:10.987 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:10.987 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:10.987 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:10.988 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-06-06T04:35:11.688 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-06-06T04:35:11.689 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-06-06T04:35:11.689 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-06-06T04:35:11.689 DEBUG:tasks.check_counter:Getting stats from i 2024-06-06T04:35:11.689 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-06-06T04:35:11.934 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T04:35:12.331 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:11 smithi154 ceph-mon[36428]: pgmap v543: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:35:12.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:11 smithi086 ceph-mon[36053]: pgmap v543: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:35:12.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:11 smithi045 ceph-mon[32299]: pgmap v543: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:35:13.808 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-06-06T04:35:13.808 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T04:35:13.808 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 158, 2024-06-06T04:35:13.809 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 193, 2024-06-06T04:35:13.809 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 45425, 2024-06-06T04:35:13.809 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 273090, 2024-06-06T04:35:13.809 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 16, 2024-06-06T04:35:13.809 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 5, 2024-06-06T04:35:13.809 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.017133720, 2024-06-06T04:35:13.809 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.004592427, 2024-06-06T04:35:13.809 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.009880326, 2024-06-06T04:35:13.809 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.001522873, 2024-06-06T04:35:13.810 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:13.810 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 189, 2024-06-06T04:35:13.810 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.021143695, 2024-06-06T04:35:13.810 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000111871 2024-06-06T04:35:13.810 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.810 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:13.810 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 201, 2024-06-06T04:35:13.810 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000037092, 2024-06-06T04:35:13.811 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000184 2024-06-06T04:35:13.811 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.811 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T04:35:13.811 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T04:35:13.811 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.811 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T04:35:13.811 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 1023, 2024-06-06T04:35:13.811 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 779, 2024-06-06T04:35:13.812 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 1066397, 2024-06-06T04:35:13.812 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 265247, 2024-06-06T04:35:13.812 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 7, 2024-06-06T04:35:13.812 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 5, 2024-06-06T04:35:13.812 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.089357571, 2024-06-06T04:35:13.812 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.032318939, 2024-06-06T04:35:13.812 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.064234205, 2024-06-06T04:35:13.812 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.006561864, 2024-06-06T04:35:13.812 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:13.813 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 781, 2024-06-06T04:35:13.813 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.043303449, 2024-06-06T04:35:13.813 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000055446 2024-06-06T04:35:13.813 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.813 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:13.813 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1389, 2024-06-06T04:35:13.813 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000296516, 2024-06-06T04:35:13.813 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000213 2024-06-06T04:35:13.814 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.814 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 791552, 2024-06-06T04:35:13.814 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 89408 2024-06-06T04:35:13.814 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.814 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T04:35:13.814 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 54, 2024-06-06T04:35:13.814 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 155, 2024-06-06T04:35:13.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 14613, 2024-06-06T04:35:13.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 334288, 2024-06-06T04:35:13.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 14, 2024-06-06T04:35:13.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 6, 2024-06-06T04:35:13.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.013632408, 2024-06-06T04:35:13.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.005518626, 2024-06-06T04:35:13.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.005486509, 2024-06-06T04:35:13.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.001263942, 2024-06-06T04:35:13.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:13.816 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 153, 2024-06-06T04:35:13.816 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.017483553, 2024-06-06T04:35:13.816 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000114271 2024-06-06T04:35:13.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.816 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:13.816 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 13, 2024-06-06T04:35:13.816 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000002181, 2024-06-06T04:35:13.816 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000167 2024-06-06T04:35:13.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.817 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-06-06T04:35:13.817 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 236704 2024-06-06T04:35:13.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.817 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-06-06T04:35:13.817 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-06-06T04:35:13.817 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-06-06T04:35:13.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.818 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-06-06T04:35:13.818 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-06-06T04:35:13.818 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-06-06T04:35:13.818 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 110, 2024-06-06T04:35:13.818 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.020074073, 2024-06-06T04:35:13.818 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000182491 2024-06-06T04:35:13.818 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:13.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.819 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-06-06T04:35:13.819 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-06-06T04:35:13.819 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-06-06T04:35:13.819 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 37, 2024-06-06T04:35:13.819 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.002068893, 2024-06-06T04:35:13.819 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000055916 2024-06-06T04:35:13.819 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:13.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.820 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-06-06T04:35:13.820 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 48, 2024-06-06T04:35:13.820 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 47, 2024-06-06T04:35:13.820 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-06-06T04:35:13.820 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 47, 2024-06-06T04:35:13.820 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.122978229, 2024-06-06T04:35:13.820 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.002616558 2024-06-06T04:35:13.820 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.820 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-06-06T04:35:13.821 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 1, 2024-06-06T04:35:13.821 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 0, 2024-06-06T04:35:13.821 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 0, 2024-06-06T04:35:13.821 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 11, 2024-06-06T04:35:13.821 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 0, 2024-06-06T04:35:13.821 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 0, 2024-06-06T04:35:13.821 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 29, 2024-06-06T04:35:13.821 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 1, 2024-06-06T04:35:13.821 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 28, 2024-06-06T04:35:13.822 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 0, 2024-06-06T04:35:13.822 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 27, 2024-06-06T04:35:13.822 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 2, 2024-06-06T04:35:13.822 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 13, 2024-06-06T04:35:13.822 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 13, 2024-06-06T04:35:13.822 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 4, 2024-06-06T04:35:13.822 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 117, 2024-06-06T04:35:13.822 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 75, 2024-06-06T04:35:13.823 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 0, 2024-06-06T04:35:13.823 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 3, 2024-06-06T04:35:13.823 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 0, 2024-06-06T04:35:13.823 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-06-06T04:35:13.823 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 0, 2024-06-06T04:35:13.823 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 0, 2024-06-06T04:35:13.823 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-06-06T04:35:13.823 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 1, 2024-06-06T04:35:13.823 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 0, 2024-06-06T04:35:13.824 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 2, 2024-06-06T04:35:13.824 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 18, 2024-06-06T04:35:13.824 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-06-06T04:35:13.824 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 7, 2024-06-06T04:35:13.824 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-06-06T04:35:13.824 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 3381, 2024-06-06T04:35:13.824 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 24837330, 2024-06-06T04:35:13.824 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 0, 2024-06-06T04:35:13.824 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 7, 2024-06-06T04:35:13.825 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-06-06T04:35:13.825 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T04:35:13.825 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-06-06T04:35:13.825 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 0, 2024-06-06T04:35:13.825 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T04:35:13.825 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T04:35:13.825 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 7, 2024-06-06T04:35:13.825 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 0, 2024-06-06T04:35:13.825 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 10, 2024-06-06T04:35:13.826 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 11, 2024-06-06T04:35:13.826 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T04:35:13.826 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T04:35:13.826 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T04:35:13.826 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 35, 2024-06-06T04:35:13.826 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 18, 2024-06-06T04:35:13.826 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 2, 2024-06-06T04:35:13.826 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 24 2024-06-06T04:35:13.827 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.827 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-06-06T04:35:13.827 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 0, 2024-06-06T04:35:13.827 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 0, 2024-06-06T04:35:13.827 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 0, 2024-06-06T04:35:13.827 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 4, 2024-06-06T04:35:13.827 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 0, 2024-06-06T04:35:13.827 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 0, 2024-06-06T04:35:13.827 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-06-06T04:35:13.828 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-06-06T04:35:13.828 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 11, 2024-06-06T04:35:13.828 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 11, 2024-06-06T04:35:13.828 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-06-06T04:35:13.828 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-06-06T04:35:13.828 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-06-06T04:35:13.828 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-06-06T04:35:13.828 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-06-06T04:35:13.828 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-06-06T04:35:13.829 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-06-06T04:35:13.829 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 0, 2024-06-06T04:35:13.829 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 0, 2024-06-06T04:35:13.829 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T04:35:13.829 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 1, 2024-06-06T04:35:13.829 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-06-06T04:35:13.829 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 0, 2024-06-06T04:35:13.830 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-06-06T04:35:13.830 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0 2024-06-06T04:35:13.830 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.830 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-06-06T04:35:13.830 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 0, 2024-06-06T04:35:13.830 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 77, 2024-06-06T04:35:13.830 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 0, 2024-06-06T04:35:13.830 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 0, 2024-06-06T04:35:13.831 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 77, 2024-06-06T04:35:13.831 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-06-06T04:35:13.831 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 0, 2024-06-06T04:35:13.831 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 2, 2024-06-06T04:35:13.831 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 0, 2024-06-06T04:35:13.831 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 0, 2024-06-06T04:35:13.831 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 2, 2024-06-06T04:35:13.831 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 2, 2024-06-06T04:35:13.832 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-06-06T04:35:13.832 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 0, 2024-06-06T04:35:13.832 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 4194304, 2024-06-06T04:35:13.832 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 4300926, 2024-06-06T04:35:13.832 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 0, 2024-06-06T04:35:13.832 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-06-06T04:35:13.832 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 30, 2024-06-06T04:35:13.832 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.155776596, 2024-06-06T04:35:13.833 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.005192553 2024-06-06T04:35:13.833 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.833 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 0 2024-06-06T04:35:13.833 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.833 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-06-06T04:35:13.833 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 33, 2024-06-06T04:35:13.833 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 45, 2024-06-06T04:35:13.833 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 12, 2024-06-06T04:35:13.834 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 24, 2024-06-06T04:35:13.834 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 26, 2024-06-06T04:35:13.834 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 2, 2024-06-06T04:35:13.834 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 29, 2024-06-06T04:35:13.834 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 54, 2024-06-06T04:35:13.834 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 25, 2024-06-06T04:35:13.834 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 13, 2024-06-06T04:35:13.834 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 25, 2024-06-06T04:35:13.835 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 12, 2024-06-06T04:35:13.835 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 53860, 2024-06-06T04:35:13.835 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 223612 2024-06-06T04:35:13.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.835 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-06-06T04:35:13.835 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 85, 2024-06-06T04:35:13.835 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 1, 2024-06-06T04:35:13.835 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 48, 2024-06-06T04:35:13.835 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 9, 2024-06-06T04:35:13.836 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 15, 2024-06-06T04:35:13.836 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-06-06T04:35:13.836 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1, 2024-06-06T04:35:13.836 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.004206765, 2024-06-06T04:35:13.836 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.004206765 2024-06-06T04:35:13.836 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.836 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-06-06T04:35:13.836 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 4, 2024-06-06T04:35:13.836 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.001703681, 2024-06-06T04:35:13.837 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000425920 2024-06-06T04:35:13.837 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.837 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-06-06T04:35:13.837 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.837 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.837 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.837 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.837 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-06-06T04:35:13.837 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.838 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.838 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.838 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.838 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-06-06T04:35:13.838 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 29, 2024-06-06T04:35:13.838 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.012801099, 2024-06-06T04:35:13.838 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000441417 2024-06-06T04:35:13.838 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.839 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-06-06T04:35:13.839 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.839 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.839 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.839 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-06-06T04:35:13.839 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.839 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.839 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.840 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.840 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-06-06T04:35:13.840 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.840 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.840 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.840 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.840 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-06-06T04:35:13.840 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.841 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.841 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.841 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.841 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-06-06T04:35:13.841 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.841 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.841 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.841 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.842 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-06-06T04:35:13.842 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.842 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.842 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.842 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.842 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-06-06T04:35:13.843 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.843 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.843 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.843 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.843 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-06-06T04:35:13.844 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.844 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.844 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.844 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.844 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-06-06T04:35:13.844 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.844 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.844 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.844 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.845 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-06-06T04:35:13.845 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.845 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.845 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.845 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.845 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-06-06T04:35:13.845 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 2, 2024-06-06T04:35:13.845 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.001082542, 2024-06-06T04:35:13.845 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000541271 2024-06-06T04:35:13.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.846 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-06-06T04:35:13.846 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.846 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.846 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.846 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-06-06T04:35:13.846 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.847 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.847 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.847 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-06-06T04:35:13.847 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.847 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.847 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.847 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-06-06T04:35:13.848 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.848 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.848 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.848 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.848 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-06-06T04:35:13.848 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.848 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.848 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.849 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.849 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-06-06T04:35:13.849 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.849 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.849 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.849 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.849 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-06-06T04:35:13.849 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.849 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.850 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.850 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.850 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-06-06T04:35:13.850 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.850 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.850 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.850 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.850 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-06-06T04:35:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-06-06T04:35:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.851 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-06-06T04:35:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-06-06T04:35:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.852 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-06-06T04:35:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 11, 2024-06-06T04:35:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.103184142, 2024-06-06T04:35:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.009380376 2024-06-06T04:35:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-06-06T04:35:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-06-06T04:35:13.853 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-06-06T04:35:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-06-06T04:35:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 1, 2024-06-06T04:35:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 2, 2024-06-06T04:35:13.854 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 1, 2024-06-06T04:35:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 1, 2024-06-06T04:35:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-06-06T04:35:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 28, 2024-06-06T04:35:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 28, 2024-06-06T04:35:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 231, 2024-06-06T04:35:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-06-06T04:35:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-06-06T04:35:13.855 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-06-06T04:35:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-06-06T04:35:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T04:35:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-06-06T04:35:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T04:35:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-06-06T04:35:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T04:35:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T04:35:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T04:35:13.856 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T04:35:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T04:35:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-06-06T04:35:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T04:35:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T04:35:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-06-06T04:35:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-06-06T04:35:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-06-06T04:35:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-06-06T04:35:13.857 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T04:35:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T04:35:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T04:35:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T04:35:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T04:35:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-06-06T04:35:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-06-06T04:35:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-06-06T04:35:13.858 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T04:35:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T04:35:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-06-06T04:35:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-06-06T04:35:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-06-06T04:35:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-06-06T04:35:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T04:35:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-06-06T04:35:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T04:35:13.859 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-06-06T04:35:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 286865, 2024-06-06T04:35:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 107, 2024-06-06T04:35:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-06-06T04:35:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-06-06T04:35:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-06-06T04:35:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-06-06T04:35:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-06-06T04:35:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-06-06T04:35:13.860 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-06-06T04:35:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-06-06T04:35:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 8992, 2024-06-06T04:35:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 173, 2024-06-06T04:35:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T04:35:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-06-06T04:35:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-06-06T04:35:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-06-06T04:35:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 129854, 2024-06-06T04:35:13.861 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 686, 2024-06-06T04:35:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-06-06T04:35:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-06-06T04:35:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-06-06T04:35:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-06-06T04:35:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-06-06T04:35:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-06-06T04:35:13.862 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-06-06T04:35:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 123, 2024-06-06T04:35:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 121195, 2024-06-06T04:35:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 33, 2024-06-06T04:35:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 123, 2024-06-06T04:35:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-06-06T04:35:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 123, 2024-06-06T04:35:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 1.321963832, 2024-06-06T04:35:13.863 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.010747673 2024-06-06T04:35:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-06-06T04:35:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-06-06T04:35:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 123, 2024-06-06T04:35:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 156 2024-06-06T04:35:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 123, 2024-06-06T04:35:13.864 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 10, 2024-06-06T04:35:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 113, 2024-06-06T04:35:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-06-06T04:35:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-06-06T04:35:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 0, 2024-06-06T04:35:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 10, 2024-06-06T04:35:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 3, 2024-06-06T04:35:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 33, 2024-06-06T04:35:13.865 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 13, 2024-06-06T04:35:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-06-06T04:35:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-06-06T04:35:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 0, 2024-06-06T04:35:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 0, 2024-06-06T04:35:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 34, 2024-06-06T04:35:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-06-06T04:35:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-06-06T04:35:13.866 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-06-06T04:35:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 7, 2024-06-06T04:35:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 20, 2024-06-06T04:35:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-06-06T04:35:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-06-06T04:35:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-06-06T04:35:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-06-06T04:35:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-06-06T04:35:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-06-06T04:35:13.867 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T04:35:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-06-06T04:35:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-06-06T04:35:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-06-06T04:35:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-06-06T04:35:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-06-06T04:35:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-06-06T04:35:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-06-06T04:35:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-06-06T04:35:13.868 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-06-06T04:35:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-06-06T04:35:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-06-06T04:35:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-06-06T04:35:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-06-06T04:35:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-06-06T04:35:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-06-06T04:35:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-06-06T04:35:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-06-06T04:35:13.869 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-06-06T04:35:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-06-06T04:35:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 100, 2024-06-06T04:35:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-06-06T04:35:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 2, 2024-06-06T04:35:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 12, 2024-06-06T04:35:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 12, 2024-06-06T04:35:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 0, 2024-06-06T04:35:13.870 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-06-06T04:35:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 31, 2024-06-06T04:35:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 0, 2024-06-06T04:35:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 5 2024-06-06T04:35:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-06-06T04:35:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 1, 2024-06-06T04:35:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-06-06T04:35:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-06-06T04:35:13.871 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-06-06T04:35:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: "osd-slow-ops": { 2024-06-06T04:35:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 0 2024-06-06T04:35:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-06-06T04:35:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-06-06T04:35:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 16, 2024-06-06T04:35:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-06-06T04:35:13.872 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 8, 2024-06-06T04:35:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 22, 2024-06-06T04:35:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 11, 2024-06-06T04:35:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-06-06T04:35:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T04:35:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-06-06T04:35:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-06-06T04:35:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-06-06T04:35:13.873 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 1231, 2024-06-06T04:35:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 1003031, 2024-06-06T04:35:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 1231, 2024-06-06T04:35:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-06-06T04:35:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-06-06T04:35:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1231, 2024-06-06T04:35:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 1003031, 2024-06-06T04:35:13.874 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-06-06T04:35:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-06-06T04:35:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-06-06T04:35:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-06-06T04:35:13.875 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-06-06T04:35:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-06-06T04:35:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-06-06T04:35:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-06-06T04:35:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 123, 2024-06-06T04:35:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 122394, 2024-06-06T04:35:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 123, 2024-06-06T04:35:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 122394, 2024-06-06T04:35:13.876 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-06-06T04:35:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-06-06T04:35:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-06-06T04:35:13.877 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-06-06T04:35:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-06-06T04:35:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-06-06T04:35:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-06-06T04:35:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-06-06T04:35:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 123, 2024-06-06T04:35:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 123, 2024-06-06T04:35:13.878 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 123, 2024-06-06T04:35:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 123, 2024-06-06T04:35:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-06-06T04:35:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-06-06T04:35:13.879 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-06-06T04:35:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-06-06T04:35:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-06-06T04:35:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 11, 2024-06-06T04:35:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 1199, 2024-06-06T04:35:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 11, 2024-06-06T04:35:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-06-06T04:35:13.880 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-06-06T04:35:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 3, 2024-06-06T04:35:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 1199, 2024-06-06T04:35:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-06-06T04:35:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x5634bf09c500": { 2024-06-06T04:35:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-06-06T04:35:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-06-06T04:35:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-06-06T04:35:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 77, 2024-06-06T04:35:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 106622, 2024-06-06T04:35:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 77, 2024-06-06T04:35:13.882 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-06-06T04:35:13.883 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-06-06T04:35:13.883 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 30, 2024-06-06T04:35:13.883 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 106622, 2024-06-06T04:35:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-06-06T04:35:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:13.885 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:13.886 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:13.886 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:13.886 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-06-06T04:35:14.133 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:13 smithi154 ceph-mon[36428]: pgmap v544: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:35:14.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:13 smithi086 ceph-mon[36053]: pgmap v544: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:35:14.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:13 smithi045 ceph-mon[32299]: pgmap v544: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:35:14.573 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-06-06T04:35:14.573 INFO:tasks.check_counter:Daemon mds.i mds.imported=2 2024-06-06T04:35:14.573 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-06-06T04:35:14.574 DEBUG:tasks.check_counter:Getting stats from l 2024-06-06T04:35:14.574 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph daemon mds.l perf dump 2024-06-06T04:35:14.814 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T04:35:16.239 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:15 smithi154 ceph-mon[36428]: pgmap v545: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-06T04:35:16.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:15 smithi086 ceph-mon[36053]: pgmap v545: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-06T04:35:16.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:15 smithi045 ceph-mon[32299]: pgmap v545: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-06T04:35:16.502 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-06-06T04:35:16.503 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T04:35:16.503 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 151, 2024-06-06T04:35:16.503 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 144, 2024-06-06T04:35:16.503 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 735712, 2024-06-06T04:35:16.503 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 87200, 2024-06-06T04:35:16.503 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 4, 2024-06-06T04:35:16.503 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 0, 2024-06-06T04:35:16.503 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.028365872, 2024-06-06T04:35:16.504 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.009188565, 2024-06-06T04:35:16.504 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.018149028, 2024-06-06T04:35:16.504 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.003632681, 2024-06-06T04:35:16.504 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:16.504 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 144, 2024-06-06T04:35:16.504 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.008926357, 2024-06-06T04:35:16.504 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000061988 2024-06-06T04:35:16.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:16.504 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:16.505 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:16.505 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:16.505 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:16.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:16.505 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 735712, 2024-06-06T04:35:16.505 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 87200 2024-06-06T04:35:16.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:16.505 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T04:35:16.505 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 1, 2024-06-06T04:35:16.506 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 108, 2024-06-06T04:35:16.506 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 160, 2024-06-06T04:35:16.506 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 96784, 2024-06-06T04:35:16.506 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 3, 2024-06-06T04:35:16.506 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 2, 2024-06-06T04:35:16.506 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.007094189, 2024-06-06T04:35:16.506 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.004830012, 2024-06-06T04:35:16.506 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.000966094, 2024-06-06T04:35:16.507 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T04:35:16.507 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:16.507 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 106, 2024-06-06T04:35:16.507 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.007330083, 2024-06-06T04:35:16.507 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000069151 2024-06-06T04:35:16.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:16.507 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:16.507 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:16.508 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:16.508 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:16.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:16.508 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-06T04:35:16.508 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 96784 2024-06-06T04:35:16.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:16.508 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T04:35:16.508 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 0, 2024-06-06T04:35:16.509 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 2, 2024-06-06T04:35:16.509 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 0, 2024-06-06T04:35:16.509 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 0, 2024-06-06T04:35:16.509 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 2, 2024-06-06T04:35:16.509 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 1, 2024-06-06T04:35:16.509 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.000421887, 2024-06-06T04:35:16.509 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.000000000, 2024-06-06T04:35:16.509 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.000364449, 2024-06-06T04:35:16.510 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T04:35:16.510 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T04:35:16.510 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:16.510 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:16.510 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:16.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:16.510 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-06-06T04:35:16.511 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:16.511 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:16.511 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:16.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:16.511 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T04:35:16.511 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T04:35:16.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:16.511 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-06-06T04:35:16.511 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 0, 2024-06-06T04:35:16.512 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-06-06T04:35:16.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:16.512 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-06-06T04:35:16.512 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-06-06T04:35:16.512 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-06-06T04:35:16.512 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T04:35:16.512 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-06-06T04:35:16.512 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T04:35:16.512 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-06-06T04:35:16.513 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T04:35:16.513 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T04:35:16.513 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T04:35:16.513 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T04:35:16.513 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T04:35:16.513 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-06-06T04:35:16.513 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T04:35:16.513 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T04:35:16.514 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-06-06T04:35:16.514 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-06-06T04:35:16.514 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-06-06T04:35:16.514 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-06-06T04:35:16.514 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T04:35:16.514 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T04:35:16.514 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T04:35:16.514 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T04:35:16.515 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T04:35:16.515 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-06-06T04:35:16.515 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-06-06T04:35:16.515 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-06-06T04:35:16.515 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T04:35:16.515 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T04:35:16.515 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-06-06T04:35:16.515 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-06-06T04:35:16.515 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-06-06T04:35:16.516 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-06-06T04:35:16.516 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T04:35:16.516 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-06-06T04:35:16.516 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T04:35:16.516 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-06-06T04:35:16.516 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 148984, 2024-06-06T04:35:16.516 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 30, 2024-06-06T04:35:16.516 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-06-06T04:35:16.517 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-06-06T04:35:16.517 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-06-06T04:35:16.517 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-06-06T04:35:16.517 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-06-06T04:35:16.517 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-06-06T04:35:16.517 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-06-06T04:35:16.517 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-06-06T04:35:16.517 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 0, 2024-06-06T04:35:16.517 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 0, 2024-06-06T04:35:16.518 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T04:35:16.518 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-06-06T04:35:16.518 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-06-06T04:35:16.518 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-06-06T04:35:16.518 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 928, 2024-06-06T04:35:16.519 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 2, 2024-06-06T04:35:16.519 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-06-06T04:35:16.519 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-06-06T04:35:16.520 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-06-06T04:35:16.520 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-06-06T04:35:16.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-06-06T04:35:16.520 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T04:35:16.520 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-06-06T04:35:16.520 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-06-06T04:35:16.520 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-06-06T04:35:16.521 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 148, 2024-06-06T04:35:16.521 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 711992, 2024-06-06T04:35:16.521 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-06-06T04:35:16.521 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 148, 2024-06-06T04:35:16.521 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-06-06T04:35:16.521 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-06-06T04:35:16.521 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 148, 2024-06-06T04:35:16.521 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 711992, 2024-06-06T04:35:16.522 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-06-06T04:35:16.522 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-06-06T04:35:16.522 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-06-06T04:35:16.522 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-06-06T04:35:16.522 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:16.522 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-06-06T04:35:16.522 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-06-06T04:35:17.172 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-06-06T04:35:17.173 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-06-06T04:35:17.173 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-06-06T04:35:17.173 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-06-06T04:35:17.184 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-06-06T04:35:17.184 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:35:17.184 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T04:35:17.205 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:35:17.205 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T04:35:17.267 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-06T04:35:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:35:18.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:17 smithi045 ceph-mon[32299]: pgmap v546: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:35:18.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:17 smithi086 ceph-mon[36053]: pgmap v546: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:35:18.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:17 smithi154 ceph-mon[36428]: pgmap v546: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-06T04:35:19.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.447+0000 7f11e19ba700 1 -- 172.21.15.45:0/3797509344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11dc101380 msgr2=0x7f11dc101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:19.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.447+0000 7f11e19ba700 1 --2- 172.21.15.45:0/3797509344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11dc101380 0x7f11dc101800 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009990 tx=0x7f11d0009ca0 comp rx=0 tx=0).stop 2024-06-06T04:35:19.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.447+0000 7f11e19ba700 1 -- 172.21.15.45:0/3797509344 shutdown_connections 2024-06-06T04:35:19.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.447+0000 7f11e19ba700 1 --2- 172.21.15.45:0/3797509344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11dc101d40 0x7f11dc10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:19.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.447+0000 7f11e19ba700 1 --2- 172.21.15.45:0/3797509344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11dc1073f0 0x7f11dc1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:19.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.447+0000 7f11e19ba700 1 --2- 172.21.15.45:0/3797509344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11dc101380 0x7f11dc101800 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:19.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.447+0000 7f11e19ba700 1 -- 172.21.15.45:0/3797509344 >> 172.21.15.45:0/3797509344 conn(0x7f11dc078300 msgr2=0x7f11dc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:19.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.447+0000 7f11e19ba700 1 -- 172.21.15.45:0/3797509344 shutdown_connections 2024-06-06T04:35:19.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.447+0000 7f11e19ba700 1 -- 172.21.15.45:0/3797509344 wait complete. 2024-06-06T04:35:19.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.448+0000 7f11e19ba700 1 Processor -- start 2024-06-06T04:35:19.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.448+0000 7f11e19ba700 1 -- start start 2024-06-06T04:35:19.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.448+0000 7f11e19ba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11dc101380 0x7f11dc1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:19.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.448+0000 7f11e19ba700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11dc101d40 0x7f11dc199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:19.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.448+0000 7f11e19ba700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11dc1073f0 0x7f11dc1a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:19.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.448+0000 7f11e19ba700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc111540 con 0x7f11dc101380 2024-06-06T04:35:19.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.448+0000 7f11e19ba700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc1113c0 con 0x7f11dc1073f0 2024-06-06T04:35:19.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.448+0000 7f11e19ba700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc1116c0 con 0x7f11dc101d40 2024-06-06T04:35:19.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.448+0000 7f11daffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11dc101380 0x7f11dc1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:19.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.449+0000 7f11daffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11dc101380 0x7f11dc1999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57410/0 (socket says 172.21.15.45:57410) 2024-06-06T04:35:19.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.449+0000 7f11daffd700 1 -- 172.21.15.45:0/2458768499 learned_addr learned my addr 172.21.15.45:0/2458768499 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:35:19.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.449+0000 7f11da7fc700 1 --2- 172.21.15.45:0/2458768499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11dc101d40 0x7f11dc199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:19.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.449+0000 7f11db7fe700 1 --2- 172.21.15.45:0/2458768499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11dc1073f0 0x7f11dc1a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:19.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.449+0000 7f11daffd700 1 -- 172.21.15.45:0/2458768499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11dc101d40 msgr2=0x7f11dc199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:19.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.449+0000 7f11daffd700 1 --2- 172.21.15.45:0/2458768499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11dc101d40 0x7f11dc199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:19.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.449+0000 7f11daffd700 1 -- 172.21.15.45:0/2458768499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11dc1073f0 msgr2=0x7f11dc1a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:19.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.449+0000 7f11daffd700 1 --2- 172.21.15.45:0/2458768499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11dc1073f0 0x7f11dc1a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:19.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.449+0000 7f11daffd700 1 -- 172.21.15.45:0/2458768499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11dc1a4fa0 con 0x7f11dc101380 2024-06-06T04:35:19.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.449+0000 7f11da7fc700 1 --2- 172.21.15.45:0/2458768499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11dc101d40 0x7f11dc199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:35:19.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.449+0000 7f11daffd700 1 --2- 172.21.15.45:0/2458768499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11dc101380 0x7f11dc1999e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00e9a0 tx=0x7f11cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:19.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.449+0000 7f11e09b8700 1 -- 172.21.15.45:0/2458768499 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc00ca10 con 0x7f11dc101380 2024-06-06T04:35:19.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.449+0000 7f11e09b8700 1 -- 172.21.15.45:0/2458768499 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11cc00cbb0 con 0x7f11dc101380 2024-06-06T04:35:19.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.449+0000 7f11e19ba700 1 -- 172.21.15.45:0/2458768499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11dc1a5170 con 0x7f11dc101380 2024-06-06T04:35:19.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.449+0000 7f11e09b8700 1 -- 172.21.15.45:0/2458768499 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc018680 con 0x7f11dc101380 2024-06-06T04:35:19.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.450+0000 7f11e19ba700 1 -- 172.21.15.45:0/2458768499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11dc1a56d0 con 0x7f11dc101380 2024-06-06T04:35:19.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.451+0000 7f11e19ba700 1 -- 172.21.15.45:0/2458768499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11dc069180 con 0x7f11dc101380 2024-06-06T04:35:19.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.452+0000 7f11e09b8700 1 -- 172.21.15.45:0/2458768499 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f11cc010430 con 0x7f11dc101380 2024-06-06T04:35:19.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.453+0000 7f11e09b8700 1 --2- 172.21.15.45:0/2458768499 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f11c40a2ca0 0x7f11c40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.453+0000 7f11da7fc700 1 --2- 172.21.15.45:0/2458768499 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f11c40a2ca0 0x7f11c40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.453+0000 7f11e09b8700 1 -- 172.21.15.45:0/2458768499 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f11cc0cf4b0 con 0x7f11dc101380 2024-06-06T04:35:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.453+0000 7f11da7fc700 1 --2- 172.21.15.45:0/2458768499 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f11c40a2ca0 0x7f11c40a5160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f11d000b2e0 tx=0x7f11d001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:19.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.457+0000 7f11e09b8700 1 -- 172.21.15.45:0/2458768499 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f11cc09c050 con 0x7f11dc101380 2024-06-06T04:35:19.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.688+0000 7f11e19ba700 1 -- 172.21.15.45:0/2458768499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f11dc02de60 con 0x7f11dc101380 2024-06-06T04:35:19.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.688+0000 7f11e09b8700 1 -- 172.21.15.45:0/2458768499 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v100) ==== 81+0+589 (secure 0 0 0) 0x7f11cc09b950 con 0x7f11dc101380 2024-06-06T04:35:19.689 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/36732849 2024-06-07T04:20:04.267269+0000 2024-06-06T04:35:19.689 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1500318775 2024-06-07T04:20:04.267269+0000 2024-06-06T04:35:19.690 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/3465796147 2024-06-07T04:20:04.267269+0000 2024-06-06T04:35:19.690 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/3465796147 2024-06-07T04:20:04.267269+0000 2024-06-06T04:35:19.690 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2041380391 2024-06-07T04:19:37.026199+0000 2024-06-06T04:35:19.690 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2504003341 2024-06-07T04:19:37.026199+0000 2024-06-06T04:35:19.690 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3798552599 2024-06-07T04:20:04.267269+0000 2024-06-06T04:35:19.690 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/498952822 2024-06-07T04:19:37.026199+0000 2024-06-06T04:35:19.691 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/1562511056 2024-06-07T04:19:37.026199+0000 2024-06-06T04:35:19.691 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/1562511056 2024-06-07T04:19:37.026199+0000 2024-06-06T04:35:19.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.690+0000 7f11e19ba700 1 -- 172.21.15.45:0/2458768499 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f11c40a2ca0 msgr2=0x7f11c40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:19.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.690+0000 7f11e19ba700 1 --2- 172.21.15.45:0/2458768499 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f11c40a2ca0 0x7f11c40a5160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f11d000b2e0 tx=0x7f11d001a040 comp rx=0 tx=0).stop 2024-06-06T04:35:19.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.690+0000 7f11e19ba700 1 -- 172.21.15.45:0/2458768499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11dc101380 msgr2=0x7f11dc1999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:19.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.690+0000 7f11e19ba700 1 --2- 172.21.15.45:0/2458768499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11dc101380 0x7f11dc1999e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00e9a0 tx=0x7f11cc00ecb0 comp rx=0 tx=0).stop 2024-06-06T04:35:19.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.691+0000 7f11e19ba700 1 -- 172.21.15.45:0/2458768499 shutdown_connections 2024-06-06T04:35:19.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.691+0000 7f11e19ba700 1 --2- 172.21.15.45:0/2458768499 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f11c40a2ca0 0x7f11c40a5160 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:19.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.691+0000 7f11e19ba700 1 --2- 172.21.15.45:0/2458768499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11dc1073f0 0x7f11dc1a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:19.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.691+0000 7f11e19ba700 1 --2- 172.21.15.45:0/2458768499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11dc101380 0x7f11dc1999e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:19.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.691+0000 7f11e19ba700 1 --2- 172.21.15.45:0/2458768499 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11dc101d40 0x7f11dc199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:19.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.691+0000 7f11e19ba700 1 -- 172.21.15.45:0/2458768499 >> 172.21.15.45:0/2458768499 conn(0x7f11dc078300 msgr2=0x7f11dc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:19.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.691+0000 7f11e19ba700 1 -- 172.21.15.45:0/2458768499 shutdown_connections 2024-06-06T04:35:19.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:19.691+0000 7f11e19ba700 1 -- 172.21.15.45:0/2458768499 wait complete. 2024-06-06T04:35:19.694 INFO:teuthology.orchestra.run.smithi045.stderr:listed 10 entries 2024-06-06T04:35:20.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:20 smithi086 ceph-mon[36053]: pgmap v547: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:35:20.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:20 smithi086 ceph-mon[36053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:35:20.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:20 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2458768499' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T04:35:20.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:20 smithi154 ceph-mon[36428]: pgmap v547: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:35:20.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:20 smithi154 ceph-mon[36428]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:35:20.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:20 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2458768499' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T04:35:20.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:20 smithi045 ceph-mon[32299]: pgmap v547: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-06-06T04:35:20.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:20 smithi045 ceph-mon[32299]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:35:20.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:20 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2458768499' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T04:35:20.536 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-06-06T04:35:20.536 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T04:35:20.554 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-06T04:35:20.845 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:35:21.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:21 smithi045 ceph-mon[32299]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:35:21.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:21 smithi086 ceph-mon[36053]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:35:21.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:21 smithi154 ceph-mon[36428]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-06T04:35:22.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:22 smithi045 ceph-mon[32299]: pgmap v548: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:35:22.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:22 smithi086 ceph-mon[36053]: pgmap v548: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:35:22.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:22 smithi154 ceph-mon[36428]: pgmap v548: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-06T04:35:22.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.473+0000 7f43ea483700 1 -- 172.21.15.45:0/816169931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e4108500 msgr2=0x7f43e41088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:22.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.473+0000 7f43ea483700 1 --2- 172.21.15.45:0/816169931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e4108500 0x7f43e41088e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009990 tx=0x7f43d4009ca0 comp rx=0 tx=0).stop 2024-06-06T04:35:22.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.473+0000 7f43ea483700 1 -- 172.21.15.45:0/816169931 shutdown_connections 2024-06-06T04:35:22.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.473+0000 7f43ea483700 1 --2- 172.21.15.45:0/816169931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43e406b580 0x7f43e410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:22.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.473+0000 7f43ea483700 1 --2- 172.21.15.45:0/816169931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e4108500 0x7f43e41088e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:22.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.473+0000 7f43ea483700 1 --2- 172.21.15.45:0/816169931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43e406abc0 0x7f43e406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:22.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.473+0000 7f43ea483700 1 -- 172.21.15.45:0/816169931 >> 172.21.15.45:0/816169931 conn(0x7f43e4078340 msgr2=0x7f43e4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:22.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.473+0000 7f43ea483700 1 -- 172.21.15.45:0/816169931 shutdown_connections 2024-06-06T04:35:22.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.473+0000 7f43ea483700 1 -- 172.21.15.45:0/816169931 wait complete. 2024-06-06T04:35:22.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.474+0000 7f43ea483700 1 Processor -- start 2024-06-06T04:35:22.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.474+0000 7f43ea483700 1 -- start start 2024-06-06T04:35:22.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.474+0000 7f43ea483700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43e406abc0 0x7f43e4101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:22.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.474+0000 7f43ea483700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43e406b580 0x7f43e41019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:22.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.474+0000 7f43ea483700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e4108500 0x7f43e4101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:22.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.474+0000 7f43ea483700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4112690 con 0x7f43e4108500 2024-06-06T04:35:22.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.474+0000 7f43ea483700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4112510 con 0x7f43e406abc0 2024-06-06T04:35:22.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.474+0000 7f43ea483700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4112810 con 0x7f43e406b580 2024-06-06T04:35:22.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.475+0000 7f43e8a20700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e4108500 0x7f43e4101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:22.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.475+0000 7f43e8a20700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e4108500 0x7f43e4101f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57432/0 (socket says 172.21.15.45:57432) 2024-06-06T04:35:22.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.475+0000 7f43e8a20700 1 -- 172.21.15.45:0/1887004916 learned_addr learned my addr 172.21.15.45:0/1887004916 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:35:22.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.475+0000 7f43e37fe700 1 --2- 172.21.15.45:0/1887004916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43e406b580 0x7f43e41019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:22.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.475+0000 7f43e3fff700 1 --2- 172.21.15.45:0/1887004916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43e406abc0 0x7f43e4101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:22.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.475+0000 7f43e8a20700 1 -- 172.21.15.45:0/1887004916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43e406b580 msgr2=0x7f43e41019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:22.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.475+0000 7f43e8a20700 1 --2- 172.21.15.45:0/1887004916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43e406b580 0x7f43e41019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:22.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.475+0000 7f43e8a20700 1 -- 172.21.15.45:0/1887004916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43e406abc0 msgr2=0x7f43e4101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:22.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.475+0000 7f43e8a20700 1 --2- 172.21.15.45:0/1887004916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43e406abc0 0x7f43e4101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:22.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.475+0000 7f43e8a20700 1 -- 172.21.15.45:0/1887004916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e41a70a0 con 0x7f43e4108500 2024-06-06T04:35:22.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.475+0000 7f43e8a20700 1 --2- 172.21.15.45:0/1887004916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e4108500 0x7f43e4101f10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00b970 tx=0x7f43dc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:22.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.475+0000 7f43e17fa700 1 -- 172.21.15.45:0/1887004916 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00d860 con 0x7f43e4108500 2024-06-06T04:35:22.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.475+0000 7f43ea483700 1 -- 172.21.15.45:0/1887004916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e41a7390 con 0x7f43e4108500 2024-06-06T04:35:22.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.475+0000 7f43e17fa700 1 -- 172.21.15.45:0/1887004916 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43dc00e430 con 0x7f43e4108500 2024-06-06T04:35:22.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.475+0000 7f43e17fa700 1 -- 172.21.15.45:0/1887004916 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc017660 con 0x7f43e4108500 2024-06-06T04:35:22.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.475+0000 7f43ea483700 1 -- 172.21.15.45:0/1887004916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e41a7880 con 0x7f43e4108500 2024-06-06T04:35:22.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.476+0000 7f43ea483700 1 -- 172.21.15.45:0/1887004916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e410a9e0 con 0x7f43e4108500 2024-06-06T04:35:22.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.478+0000 7f43e17fa700 1 -- 172.21.15.45:0/1887004916 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f43dc00e5e0 con 0x7f43e4108500 2024-06-06T04:35:22.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.479+0000 7f43e17fa700 1 --2- 172.21.15.45:0/1887004916 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43cc0a2c20 0x7f43cc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:22.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.479+0000 7f43e3fff700 1 --2- 172.21.15.45:0/1887004916 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43cc0a2c20 0x7f43cc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:22.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.479+0000 7f43e17fa700 1 -- 172.21.15.45:0/1887004916 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f43dc0cf430 con 0x7f43e4108500 2024-06-06T04:35:22.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.480+0000 7f43e3fff700 1 --2- 172.21.15.45:0/1887004916 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43cc0a2c20 0x7f43cc0a50e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f43d40098e0 tx=0x7f43d4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:22.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.483+0000 7f43e17fa700 1 -- 172.21.15.45:0/1887004916 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f43dc013030 con 0x7f43e4108500 2024-06-06T04:35:22.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.709+0000 7f43ea483700 1 -- 172.21.15.45:0/1887004916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f43e410a3c0 con 0x7f43e4108500 2024-06-06T04:35:22.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.709+0000 7f43e17fa700 1 -- 172.21.15.45:0/1887004916 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v100) ==== 81+0+589 (secure 0 0 0) 0x7f43dc09bf70 con 0x7f43e4108500 2024-06-06T04:35:22.710 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/36732849 2024-06-07T04:20:04.267269+0000 2024-06-06T04:35:22.711 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1500318775 2024-06-07T04:20:04.267269+0000 2024-06-06T04:35:22.711 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/3465796147 2024-06-07T04:20:04.267269+0000 2024-06-06T04:35:22.711 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/3465796147 2024-06-07T04:20:04.267269+0000 2024-06-06T04:35:22.711 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2041380391 2024-06-07T04:19:37.026199+0000 2024-06-06T04:35:22.711 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2504003341 2024-06-07T04:19:37.026199+0000 2024-06-06T04:35:22.711 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3798552599 2024-06-07T04:20:04.267269+0000 2024-06-06T04:35:22.711 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/498952822 2024-06-07T04:19:37.026199+0000 2024-06-06T04:35:22.712 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/1562511056 2024-06-07T04:19:37.026199+0000 2024-06-06T04:35:22.712 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/1562511056 2024-06-07T04:19:37.026199+0000 2024-06-06T04:35:22.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.712+0000 7f43ea483700 1 -- 172.21.15.45:0/1887004916 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43cc0a2c20 msgr2=0x7f43cc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.712+0000 7f43ea483700 1 --2- 172.21.15.45:0/1887004916 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43cc0a2c20 0x7f43cc0a50e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f43d40098e0 tx=0x7f43d4005960 comp rx=0 tx=0).stop 2024-06-06T04:35:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.712+0000 7f43ea483700 1 -- 172.21.15.45:0/1887004916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e4108500 msgr2=0x7f43e4101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.712+0000 7f43ea483700 1 --2- 172.21.15.45:0/1887004916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e4108500 0x7f43e4101f10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00b970 tx=0x7f43dc00bc80 comp rx=0 tx=0).stop 2024-06-06T04:35:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.712+0000 7f43ea483700 1 -- 172.21.15.45:0/1887004916 shutdown_connections 2024-06-06T04:35:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.712+0000 7f43ea483700 1 --2- 172.21.15.45:0/1887004916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e4108500 0x7f43e4101f10 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.712+0000 7f43ea483700 1 --2- 172.21.15.45:0/1887004916 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f43cc0a2c20 0x7f43cc0a50e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.712+0000 7f43ea483700 1 --2- 172.21.15.45:0/1887004916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43e406abc0 0x7f43e4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.712+0000 7f43ea483700 1 --2- 172.21.15.45:0/1887004916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43e406b580 0x7f43e41019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:22.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.712+0000 7f43ea483700 1 -- 172.21.15.45:0/1887004916 >> 172.21.15.45:0/1887004916 conn(0x7f43e4078340 msgr2=0x7f43e4100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:22.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.712+0000 7f43ea483700 1 -- 172.21.15.45:0/1887004916 shutdown_connections 2024-06-06T04:35:22.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:22.712+0000 7f43ea483700 1 -- 172.21.15.45:0/1887004916 wait complete. 2024-06-06T04:35:22.714 INFO:teuthology.orchestra.run.smithi045.stderr:listed 10 entries 2024-06-06T04:35:23.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:23 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1887004916' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T04:35:23.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:23 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1887004916' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T04:35:23.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:23 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1887004916' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T04:35:23.499 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2024-06-06T04:35:23.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:35:23.499 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-06-06T04:35:23.530 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi045.stderr:ceph-fuse[91397]: fuse finished with error 0 and tester_r 0 2024-06-06T04:35:23.540 INFO:teuthology.orchestra.run:waiting for 300 2024-06-06T04:35:24.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:24 smithi045 ceph-mon[32299]: pgmap v549: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:35:24.272 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:24 smithi086 ceph-mon[36053]: pgmap v549: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:35:24.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:24 smithi154 ceph-mon[36428]: pgmap v549: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-06T04:35:26.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:26 smithi086 ceph-mon[36053]: pgmap v550: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-06-06T04:35:26.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:26 smithi154 ceph-mon[36428]: pgmap v550: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-06-06T04:35:26.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:26 smithi045 ceph-mon[32299]: pgmap v550: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-06-06T04:35:28.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:28 smithi086 ceph-mon[36053]: pgmap v551: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-06T04:35:28.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:28 smithi154 ceph-mon[36428]: pgmap v551: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-06T04:35:28.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:28 smithi045 ceph-mon[32299]: pgmap v551: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-06-06T04:35:29.642 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-06-06T04:35:29.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:35:29.642 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-06-06T04:35:29.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:35:29.664 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-06-06T04:35:29.724 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-06-06T04:35:29.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:35:29.725 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T04:35:29.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T04:35:29.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-06-06T04:35:29.901 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-06T04:35:29.913 INFO:tasks.cephadm:Teardown begin 2024-06-06T04:35:29.914 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-06-06T04:35:29.914 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:35:30.169 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:35:30.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:30 smithi045 ceph-mon[32299]: pgmap v552: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-06-06T04:35:30.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:30 smithi086 ceph-mon[36053]: pgmap v552: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-06-06T04:35:30.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:30 smithi154 ceph-mon[36428]: pgmap v552: 129 pgs: 129 active+clean; 30 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-06-06T04:35:31.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.935+0000 7fa415666700 1 -- 172.21.15.45:0/3001038771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa410107550 msgr2=0x7fa410107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:31.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.935+0000 7fa415666700 1 --2- 172.21.15.45:0/3001038771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa410107550 0x7fa410107930 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa400009990 tx=0x7fa400009ca0 comp rx=0 tx=0).stop 2024-06-06T04:35:31.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.935+0000 7fa415666700 1 -- 172.21.15.45:0/3001038771 shutdown_connections 2024-06-06T04:35:31.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.935+0000 7fa415666700 1 --2- 172.21.15.45:0/3001038771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4101017f0 0x7fa41010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:31.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.935+0000 7fa415666700 1 --2- 172.21.15.45:0/3001038771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa410107550 0x7fa410107930 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:31.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.935+0000 7fa415666700 1 --2- 172.21.15.45:0/3001038771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa410100e30 0x7fa4101012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:31.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.935+0000 7fa415666700 1 -- 172.21.15.45:0/3001038771 >> 172.21.15.45:0/3001038771 conn(0x7fa4100766b0 msgr2=0x7fa410076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:31.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.935+0000 7fa415666700 1 -- 172.21.15.45:0/3001038771 shutdown_connections 2024-06-06T04:35:31.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.935+0000 7fa415666700 1 -- 172.21.15.45:0/3001038771 wait complete. 2024-06-06T04:35:31.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.936+0000 7fa415666700 1 Processor -- start 2024-06-06T04:35:31.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.936+0000 7fa415666700 1 -- start start 2024-06-06T04:35:31.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.936+0000 7fa415666700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa410100e30 0x7fa41019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:31.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.936+0000 7fa415666700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4101017f0 0x7fa41019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:31.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.936+0000 7fa415666700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa410107550 0x7fa410198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:31.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.936+0000 7fa415666700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa410112740 con 0x7fa4101017f0 2024-06-06T04:35:31.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.936+0000 7fa415666700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4101125c0 con 0x7fa410107550 2024-06-06T04:35:31.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.936+0000 7fa415666700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa4101128c0 con 0x7fa410100e30 2024-06-06T04:35:31.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.936+0000 7fa40e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4101017f0 0x7fa41019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:31.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.936+0000 7fa40e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4101017f0 0x7fa41019eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56374/0 (socket says 172.21.15.45:56374) 2024-06-06T04:35:31.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.936+0000 7fa40e7fc700 1 -- 172.21.15.45:0/2559809715 learned_addr learned my addr 172.21.15.45:0/2559809715 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:35:31.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.937+0000 7fa40f7fe700 1 --2- 172.21.15.45:0/2559809715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa410107550 0x7fa410198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:31.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.937+0000 7fa40effd700 1 --2- 172.21.15.45:0/2559809715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa410100e30 0x7fa41019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:31.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.937+0000 7fa40e7fc700 1 -- 172.21.15.45:0/2559809715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa410100e30 msgr2=0x7fa41019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:31.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.937+0000 7fa40e7fc700 1 --2- 172.21.15.45:0/2559809715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa410100e30 0x7fa41019e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:31.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.937+0000 7fa40e7fc700 1 -- 172.21.15.45:0/2559809715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa410107550 msgr2=0x7fa410198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:31.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.937+0000 7fa40e7fc700 1 --2- 172.21.15.45:0/2559809715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa410107550 0x7fa410198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:31.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.937+0000 7fa40e7fc700 1 -- 172.21.15.45:0/2559809715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa410199180 con 0x7fa4101017f0 2024-06-06T04:35:31.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.937+0000 7fa40e7fc700 1 --2- 172.21.15.45:0/2559809715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4101017f0 0x7fa41019eda0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800b530 tx=0x7fa3f800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:31.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.937+0000 7fa407fff700 1 -- 172.21.15.45:0/2559809715 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8015020 con 0x7fa4101017f0 2024-06-06T04:35:31.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.937+0000 7fa407fff700 1 -- 172.21.15.45:0/2559809715 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3f8011ce0 con 0x7fa4101017f0 2024-06-06T04:35:31.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.937+0000 7fa415666700 1 -- 172.21.15.45:0/2559809715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa410199470 con 0x7fa4101017f0 2024-06-06T04:35:31.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.937+0000 7fa407fff700 1 -- 172.21.15.45:0/2559809715 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8010d60 con 0x7fa4101017f0 2024-06-06T04:35:31.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.937+0000 7fa415666700 1 -- 172.21.15.45:0/2559809715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4101a5730 con 0x7fa4101017f0 2024-06-06T04:35:31.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.938+0000 7fa415666700 1 -- 172.21.15.45:0/2559809715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa410069180 con 0x7fa4101017f0 2024-06-06T04:35:31.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.940+0000 7fa407fff700 1 -- 172.21.15.45:0/2559809715 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa3f8018020 con 0x7fa4101017f0 2024-06-06T04:35:31.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.940+0000 7fa407fff700 1 --2- 172.21.15.45:0/2559809715 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa3fc0a2c20 0x7fa3fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:31.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.940+0000 7fa40effd700 1 --2- 172.21.15.45:0/2559809715 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa3fc0a2c20 0x7fa3fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:31.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.940+0000 7fa407fff700 1 -- 172.21.15.45:0/2559809715 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fa3f80ceca0 con 0x7fa4101017f0 2024-06-06T04:35:31.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.941+0000 7fa40effd700 1 --2- 172.21.15.45:0/2559809715 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa3fc0a2c20 0x7fa3fc0a50e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa400006010 tx=0x7fa40000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:31.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:31.945+0000 7fa407fff700 1 -- 172.21.15.45:0/2559809715 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa3f801f020 con 0x7fa4101017f0 2024-06-06T04:35:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:32.197+0000 7fa415666700 1 -- 172.21.15.45:0/2559809715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa41002dca0 con 0x7fa4101017f0 2024-06-06T04:35:32.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:32.197+0000 7fa407fff700 1 -- 172.21.15.45:0/2559809715 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11858 (secure 0 0 0) 0x7fa3f809b840 con 0x7fa4101017f0 2024-06-06T04:35:32.198 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:35:32.198 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T04:27:24.462897+0000","modified":"2024-06-06T04:35:19.343059+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24451,24463,24473]},"id":1}]} 2024-06-06T04:35:32.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:32.200+0000 7fa415666700 1 -- 172.21.15.45:0/2559809715 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa3fc0a2c20 msgr2=0x7fa3fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:32.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:32.200+0000 7fa415666700 1 --2- 172.21.15.45:0/2559809715 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa3fc0a2c20 0x7fa3fc0a50e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa400006010 tx=0x7fa40000b540 comp rx=0 tx=0).stop 2024-06-06T04:35:32.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:32.200+0000 7fa415666700 1 -- 172.21.15.45:0/2559809715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4101017f0 msgr2=0x7fa41019eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:32.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:32.200+0000 7fa415666700 1 --2- 172.21.15.45:0/2559809715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4101017f0 0x7fa41019eda0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800b530 tx=0x7fa3f800b8f0 comp rx=0 tx=0).stop 2024-06-06T04:35:32.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:32.200+0000 7fa415666700 1 -- 172.21.15.45:0/2559809715 shutdown_connections 2024-06-06T04:35:32.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:32.200+0000 7fa415666700 1 --2- 172.21.15.45:0/2559809715 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fa3fc0a2c20 0x7fa3fc0a50e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:32.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:32.200+0000 7fa415666700 1 --2- 172.21.15.45:0/2559809715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa410107550 0x7fa410198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:32.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:32.200+0000 7fa415666700 1 --2- 172.21.15.45:0/2559809715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa410100e30 0x7fa41019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:32.200+0000 7fa415666700 1 --2- 172.21.15.45:0/2559809715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4101017f0 0x7fa41019eda0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:32.200+0000 7fa415666700 1 -- 172.21.15.45:0/2559809715 >> 172.21.15.45:0/2559809715 conn(0x7fa4100766b0 msgr2=0x7fa4100ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:32.200+0000 7fa415666700 1 -- 172.21.15.45:0/2559809715 shutdown_connections 2024-06-06T04:35:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:32.200+0000 7fa415666700 1 -- 172.21.15.45:0/2559809715 wait complete. 2024-06-06T04:35:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 33 2024-06-06T04:35:32.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:32 smithi086 ceph-mon[36053]: pgmap v553: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-06-06T04:35:32.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:32 smithi154 ceph-mon[36428]: pgmap v553: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-06-06T04:35:32.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:32 smithi045 ceph-mon[32299]: pgmap v553: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-06-06T04:35:33.013 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T04:35:33.273 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:35:33.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:33 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2559809715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:35:33.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:33 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2559809715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:35:33.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:33 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2559809715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:35:34.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:34 smithi086 ceph-mon[36053]: pgmap v554: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-06-06T04:35:34.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:34 smithi045 ceph-mon[32299]: pgmap v554: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-06-06T04:35:34.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:34 smithi154 ceph-mon[36428]: pgmap v554: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-06-06T04:35:35.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.140+0000 7fb494eb1700 1 -- 172.21.15.45:0/1614454235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb490107840 msgr2=0x7fb49010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:35.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.140+0000 7fb494eb1700 1 --2- 172.21.15.45:0/1614454235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb490107840 0x7fb49010f3c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb48800b0d0 tx=0x7fb48800b3e0 comp rx=0 tx=0).stop 2024-06-06T04:35:35.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.141+0000 7fb494eb1700 1 -- 172.21.15.45:0/1614454235 shutdown_connections 2024-06-06T04:35:35.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.141+0000 7fb494eb1700 1 --2- 172.21.15.45:0/1614454235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb490107840 0x7fb49010f3c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:35.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.141+0000 7fb494eb1700 1 --2- 172.21.15.45:0/1614454235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb490102740 0x7fb490102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:35.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.141+0000 7fb494eb1700 1 --2- 172.21.15.45:0/1614454235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4901030f0 0x7fb490107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:35.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.141+0000 7fb494eb1700 1 -- 172.21.15.45:0/1614454235 >> 172.21.15.45:0/1614454235 conn(0x7fb4900795f0 msgr2=0x7fb490079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:35.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.141+0000 7fb494eb1700 1 -- 172.21.15.45:0/1614454235 shutdown_connections 2024-06-06T04:35:35.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.141+0000 7fb494eb1700 1 -- 172.21.15.45:0/1614454235 wait complete. 2024-06-06T04:35:35.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.141+0000 7fb494eb1700 1 Processor -- start 2024-06-06T04:35:35.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.141+0000 7fb494eb1700 1 -- start start 2024-06-06T04:35:35.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb494eb1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb490102740 0x7fb490074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:35.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb494eb1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4901030f0 0x7fb490074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:35.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb494eb1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb490107840 0x7fb490075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:35.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb494eb1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb4901126a0 con 0x7fb4901030f0 2024-06-06T04:35:35.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb494eb1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb490112520 con 0x7fb490107840 2024-06-06T04:35:35.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb494eb1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb490112820 con 0x7fb490102740 2024-06-06T04:35:35.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb48dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4901030f0 0x7fb490074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:35.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb48dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4901030f0 0x7fb490074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56380/0 (socket says 172.21.15.45:56380) 2024-06-06T04:35:35.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb48dd9b700 1 -- 172.21.15.45:0/1874063802 learned_addr learned my addr 172.21.15.45:0/1874063802 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:35:35.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb48e59c700 1 --2- 172.21.15.45:0/1874063802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb490102740 0x7fb490074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:35.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb48ed9d700 1 --2- 172.21.15.45:0/1874063802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb490107840 0x7fb490075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:35.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb48dd9b700 1 -- 172.21.15.45:0/1874063802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb490102740 msgr2=0x7fb490074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:35.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb48dd9b700 1 --2- 172.21.15.45:0/1874063802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb490102740 0x7fb490074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:35.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb48dd9b700 1 -- 172.21.15.45:0/1874063802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb490107840 msgr2=0x7fb490075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:35.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb48dd9b700 1 --2- 172.21.15.45:0/1874063802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb490107840 0x7fb490075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:35.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb48dd9b700 1 -- 172.21.15.45:0/1874063802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4901a7000 con 0x7fb4901030f0 2024-06-06T04:35:35.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.142+0000 7fb48dd9b700 1 --2- 172.21.15.45:0/1874063802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4901030f0 0x7fb490074cd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb48400b530 tx=0x7fb48400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:35.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.143+0000 7fb47f7fe700 1 -- 172.21.15.45:0/1874063802 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484015020 con 0x7fb4901030f0 2024-06-06T04:35:35.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.143+0000 7fb494eb1700 1 -- 172.21.15.45:0/1874063802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4901a72f0 con 0x7fb4901030f0 2024-06-06T04:35:35.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.143+0000 7fb47f7fe700 1 -- 172.21.15.45:0/1874063802 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb484011ce0 con 0x7fb4901030f0 2024-06-06T04:35:35.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.143+0000 7fb47f7fe700 1 -- 172.21.15.45:0/1874063802 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484010dd0 con 0x7fb4901030f0 2024-06-06T04:35:35.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.143+0000 7fb494eb1700 1 -- 172.21.15.45:0/1874063802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4901a7880 con 0x7fb4901030f0 2024-06-06T04:35:35.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.144+0000 7fb494eb1700 1 -- 172.21.15.45:0/1874063802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb49010aa10 con 0x7fb4901030f0 2024-06-06T04:35:35.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.145+0000 7fb47f7fe700 1 -- 172.21.15.45:0/1874063802 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb484018020 con 0x7fb4901030f0 2024-06-06T04:35:35.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.145+0000 7fb47f7fe700 1 --2- 172.21.15.45:0/1874063802 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4780a2c20 0x7fb4780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:35.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.146+0000 7fb47f7fe700 1 -- 172.21.15.45:0/1874063802 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7fb4840ceb00 con 0x7fb4901030f0 2024-06-06T04:35:35.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.146+0000 7fb48e59c700 1 --2- 172.21.15.45:0/1874063802 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4780a2c20 0x7fb4780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:35.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.146+0000 7fb48e59c700 1 --2- 172.21.15.45:0/1874063802 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4780a2c20 0x7fb4780a50e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb480004630 tx=0x7fb48000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:35.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.150+0000 7fb47f7fe700 1 -- 172.21.15.45:0/1874063802 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb48401f020 con 0x7fb4901030f0 2024-06-06T04:35:35.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.409+0000 7fb494eb1700 1 -- 172.21.15.45:0/1874063802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb490075cc0 con 0x7fb4901030f0 2024-06-06T04:35:35.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.409+0000 7fb47f7fe700 1 -- 172.21.15.45:0/1874063802 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11858 (secure 0 0 0) 0x7fb48409b6a0 con 0x7fb4901030f0 2024-06-06T04:35:35.410 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:35:35.411 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1876588301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1876588301},{"type":"v1","addr":"172.21.15.45:6835","nonce":1876588301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14541,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/3532043360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":3532043360},{"type":"v1","addr":"172.21.15.45:6837","nonce":3532043360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2563890632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2563890632},{"type":"v1","addr":"172.21.15.45:6839","nonce":2563890632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/611193504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":611193504},{"type":"v1","addr":"172.21.15.45:6841","nonce":611193504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/452357683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":452357683},{"type":"v1","addr":"172.21.15.86:6833","nonce":452357683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2851773245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2851773245},{"type":"v1","addr":"172.21.15.154:6833","nonce":2851773245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T04:27:24.462897+0000","modified":"2024-06-06T04:35:19.343059+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24473,"mds_1":24463,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6835/4268654313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4268654313},{"type":"v1","addr":"172.21.15.86:6835","nonce":4268654313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/954391764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":954391764},{"type":"v1","addr":"172.21.15.154:6835","nonce":954391764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6837/414874753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":414874753},{"type":"v1","addr":"172.21.15.86:6837","nonce":414874753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6837/1140071744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1140071744},{"type":"v1","addr":"172.21.15.154:6837","nonce":1140071744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.154:6839/982197202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":982197202},{"type":"v1","addr":"172.21.15.154:6839","nonce":982197202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.86:6839/816791193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":816791193},{"type":"v1","addr":"172.21.15.86:6839","nonce":816791193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24451,24463,24473]},"id":1}]} 2024-06-06T04:35:35.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.411+0000 7fb494eb1700 1 -- 172.21.15.45:0/1874063802 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4780a2c20 msgr2=0x7fb4780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:35.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.411+0000 7fb494eb1700 1 --2- 172.21.15.45:0/1874063802 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4780a2c20 0x7fb4780a50e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb480004630 tx=0x7fb48000b3c0 comp rx=0 tx=0).stop 2024-06-06T04:35:35.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.411+0000 7fb494eb1700 1 -- 172.21.15.45:0/1874063802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4901030f0 msgr2=0x7fb490074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:35.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.411+0000 7fb494eb1700 1 --2- 172.21.15.45:0/1874063802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4901030f0 0x7fb490074cd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb48400b530 tx=0x7fb48400b840 comp rx=0 tx=0).stop 2024-06-06T04:35:35.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.412+0000 7fb494eb1700 1 -- 172.21.15.45:0/1874063802 shutdown_connections 2024-06-06T04:35:35.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.412+0000 7fb494eb1700 1 --2- 172.21.15.45:0/1874063802 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fb4780a2c20 0x7fb4780a50e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:35.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.412+0000 7fb494eb1700 1 --2- 172.21.15.45:0/1874063802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb490107840 0x7fb490075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:35.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.412+0000 7fb494eb1700 1 --2- 172.21.15.45:0/1874063802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb490102740 0x7fb490074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:35.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.412+0000 7fb494eb1700 1 --2- 172.21.15.45:0/1874063802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4901030f0 0x7fb490074cd0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:35.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.412+0000 7fb494eb1700 1 -- 172.21.15.45:0/1874063802 >> 172.21.15.45:0/1874063802 conn(0x7fb4900795f0 msgr2=0x7fb4901007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:35.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.412+0000 7fb494eb1700 1 -- 172.21.15.45:0/1874063802 shutdown_connections 2024-06-06T04:35:35.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:35.412+0000 7fb494eb1700 1 -- 172.21.15.45:0/1874063802 wait complete. 2024-06-06T04:35:35.414 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 33 2024-06-06T04:35:36.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T04:35:36.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:36 smithi086 ceph-mon[36053]: pgmap v555: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-06-06T04:35:36.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:36 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/1874063802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:35:36.360 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:35:36.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:36 smithi154 ceph-mon[36428]: pgmap v555: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-06-06T04:35:36.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:36 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/1874063802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:35:36.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:36 smithi045 ceph-mon[32299]: pgmap v555: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-06-06T04:35:36.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:36 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/1874063802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T04:35:38.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.166+0000 7f58fc7dd700 1 -- 172.21.15.45:0/1415892100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58f4102e80 msgr2=0x7f58f410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:38.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.166+0000 7f58fc7dd700 1 --2- 172.21.15.45:0/1415892100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58f4102e80 0x7f58f410f330 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00b0a0 tx=0x7f58ec00b3b0 comp rx=0 tx=0).stop 2024-06-06T04:35:38.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.166+0000 7f58fc7dd700 1 -- 172.21.15.45:0/1415892100 shutdown_connections 2024-06-06T04:35:38.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.166+0000 7f58fc7dd700 1 --2- 172.21.15.45:0/1415892100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58f4102e80 0x7f58f410f330 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:38.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.166+0000 7f58fc7dd700 1 --2- 172.21.15.45:0/1415892100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58f41084f0 0x7f58f41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:38.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.166+0000 7f58fc7dd700 1 --2- 172.21.15.45:0/1415892100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f41024e0 0x7f58f4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:38.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.166+0000 7f58fc7dd700 1 -- 172.21.15.45:0/1415892100 >> 172.21.15.45:0/1415892100 conn(0x7f58f4079530 msgr2=0x7f58f4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:38.168 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:38 smithi045 ceph-mon[32299]: pgmap v556: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-06-06T04:35:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.166+0000 7f58fc7dd700 1 -- 172.21.15.45:0/1415892100 shutdown_connections 2024-06-06T04:35:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.166+0000 7f58fc7dd700 1 -- 172.21.15.45:0/1415892100 wait complete. 2024-06-06T04:35:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.167+0000 7f58fc7dd700 1 Processor -- start 2024-06-06T04:35:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.167+0000 7f58fc7dd700 1 -- start start 2024-06-06T04:35:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.167+0000 7f58fc7dd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f41024e0 0x7f58f419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:38.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.167+0000 7f58fc7dd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58f4102e80 0x7f58f419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:38.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.167+0000 7f58fc7dd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58f41084f0 0x7f58f4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:38.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.167+0000 7f58fc7dd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f58f4112610 con 0x7f58f4102e80 2024-06-06T04:35:38.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.167+0000 7f58fc7dd700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58f4112490 con 0x7f58f41024e0 2024-06-06T04:35:38.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.167+0000 7f58fc7dd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f58f4112790 con 0x7f58f41084f0 2024-06-06T04:35:38.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.167+0000 7f58f9d78700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58f4102e80 0x7f58f419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:38.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.167+0000 7f58f9d78700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58f4102e80 0x7f58f419ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35264/0 (socket says 172.21.15.45:35264) 2024-06-06T04:35:38.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.167+0000 7f58f9d78700 1 -- 172.21.15.45:0/3941550966 learned_addr learned my addr 172.21.15.45:0/3941550966 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:35:38.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.167+0000 7f58fad7a700 1 --2- 172.21.15.45:0/3941550966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58f41084f0 0x7f58f4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:38.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.167+0000 7f58fa579700 1 --2- 172.21.15.45:0/3941550966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f41024e0 0x7f58f419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:38.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.168+0000 7f58f9d78700 1 -- 172.21.15.45:0/3941550966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58f41084f0 msgr2=0x7f58f4198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:38.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.168+0000 7f58f9d78700 1 --2- 172.21.15.45:0/3941550966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58f41084f0 0x7f58f4198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:38.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.168+0000 7f58f9d78700 1 -- 172.21.15.45:0/3941550966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f41024e0 msgr2=0x7f58f419e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:38.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.168+0000 7f58f9d78700 1 --2- 172.21.15.45:0/3941550966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f41024e0 0x7f58f419e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:38.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.168+0000 7f58f9d78700 1 -- 172.21.15.45:0/3941550966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f4199070 con 0x7f58f4102e80 2024-06-06T04:35:38.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.168+0000 7f58fa579700 1 --2- 172.21.15.45:0/3941550966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f41024e0 0x7f58f419e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:35:38.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.168+0000 7f58f9d78700 1 --2- 172.21.15.45:0/3941550966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58f4102e80 0x7f58f419ec90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f58e400d950 tx=0x7f58e400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:38.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.168+0000 7f58eb7fe700 1 -- 172.21.15.45:0/3941550966 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4016070 con 0x7f58f4102e80 2024-06-06T04:35:38.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.168+0000 7f58eb7fe700 1 -- 172.21.15.45:0/3941550966 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58e4009df0 con 0x7f58f4102e80 2024-06-06T04:35:38.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.168+0000 7f58fc7dd700 1 -- 172.21.15.45:0/3941550966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f4199360 con 0x7f58f4102e80 2024-06-06T04:35:38.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.168+0000 7f58eb7fe700 1 -- 172.21.15.45:0/3941550966 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e400f3c0 con 0x7f58f4102e80 2024-06-06T04:35:38.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.168+0000 7f58fc7dd700 1 -- 172.21.15.45:0/3941550966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f40746c0 con 0x7f58f4102e80 2024-06-06T04:35:38.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.169+0000 7f58fc7dd700 1 -- 172.21.15.45:0/3941550966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f4069180 con 0x7f58f4102e80 2024-06-06T04:35:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.171+0000 7f58eb7fe700 1 -- 172.21.15.45:0/3941550966 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f58e4019070 con 0x7f58f4102e80 2024-06-06T04:35:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.171+0000 7f58eb7fe700 1 --2- 172.21.15.45:0/3941550966 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f58e00a2c20 0x7f58e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.172+0000 7f58eb7fe700 1 -- 172.21.15.45:0/3941550966 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f58e40cf020 con 0x7f58f4102e80 2024-06-06T04:35:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.172+0000 7f58fa579700 1 --2- 172.21.15.45:0/3941550966 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f58e00a2c20 0x7f58e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:38.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.172+0000 7f58fa579700 1 --2- 172.21.15.45:0/3941550966 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f58e00a2c20 0x7f58e00a50e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f58f0005e10 tx=0x7f58f0005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:38.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.175+0000 7f58eb7fe700 1 -- 172.21.15.45:0/3941550966 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f58e409bb60 con 0x7f58f4102e80 2024-06-06T04:35:38.234 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:38 smithi154 ceph-mon[36428]: pgmap v556: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-06-06T04:35:38.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:38 smithi086 ceph-mon[36053]: pgmap v556: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-06-06T04:35:38.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.380+0000 7f58fc7dd700 1 -- 172.21.15.45:0/3941550966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58f402dca0 con 0x7f58f4102e80 2024-06-06T04:35:38.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.382+0000 7f58eb7fe700 1 -- 172.21.15.45:0/3941550966 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24789 (secure 0 0 0) 0x7f58e409b460 con 0x7f58f4102e80 2024-06-06T04:35:38.383 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-06-06T04:35:38.383 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":100,"fsid":"c4f6b938-23bb-11ef-bc9b-c7b262605968","created":"2024-06-06T04:19:02.626916+0000","modified":"2024-06-06T04:29:05.901370+0000","last_up_change":"2024-06-06T04:26:17.390429+0000","last_in_change":"2024-06-06T04:25:56.938388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T04:22:28.797137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T04:27:17.750625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T04:27:21.486917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T04:27:31.029818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7d3cee4a-5025-45b3-b217-082af37e79db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6803","nonce":2111759250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6805","nonce":2111759250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6809","nonce":2111759250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2111759250},{"type":"v1","addr":"172.21.15.45:6807","nonce":2111759250}]},"public_addr":"172.21.15.45:6803/2111759250","cluster_addr":"172.21.15.45:6805/2111759250","heartbeat_back_addr":"172.21.15.45:6809/2111759250","heartbeat_front_addr":"172.21.15.45:6807/2111759250","state":["exists","up"]},{"osd":1,"uuid":"75c1bf2b-1ef8-45a7-a6bc-89b2bf5e70ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6801","nonce":3732225287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6803","nonce":3732225287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6807","nonce":3732225287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":3732225287},{"type":"v1","addr":"172.21.15.86:6805","nonce":3732225287}]},"public_addr":"172.21.15.86:6801/3732225287","cluster_addr":"172.21.15.86:6803/3732225287","heartbeat_back_addr":"172.21.15.86:6807/3732225287","heartbeat_front_addr":"172.21.15.86:6805/3732225287","state":["exists","up"]},{"osd":2,"uuid":"bb20565f-f9ae-47e3-9604-c69f737cff4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6801","nonce":2898783039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6803","nonce":2898783039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6807","nonce":2898783039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2898783039},{"type":"v1","addr":"172.21.15.154:6805","nonce":2898783039}]},"public_addr":"172.21.15.154:6801/2898783039","cluster_addr":"172.21.15.154:6803/2898783039","heartbeat_back_addr":"172.21.15.154:6807/2898783039","heartbeat_front_addr":"172.21.15.154:6805/2898783039","state":["exists","up"]},{"osd":3,"uuid":"e1160799-7811-4326-9792-7e6c7d189c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6811","nonce":3436753135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6813","nonce":3436753135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6817","nonce":3436753135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3436753135},{"type":"v1","addr":"172.21.15.45:6815","nonce":3436753135}]},"public_addr":"172.21.15.45:6811/3436753135","cluster_addr":"172.21.15.45:6813/3436753135","heartbeat_back_addr":"172.21.15.45:6817/3436753135","heartbeat_front_addr":"172.21.15.45:6815/3436753135","state":["exists","up"]},{"osd":4,"uuid":"6b389bc8-e19a-47f3-83ff-7b782d146d0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6809","nonce":3030235796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6811","nonce":3030235796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6815","nonce":3030235796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3030235796},{"type":"v1","addr":"172.21.15.86:6813","nonce":3030235796}]},"public_addr":"172.21.15.86:6809/3030235796","cluster_addr":"172.21.15.86:6811/3030235796","heartbeat_back_addr":"172.21.15.86:6815/3030235796","heartbeat_front_addr":"172.21.15.86:6813/3030235796","state":["exists","up"]},{"osd":5,"uuid":"069de7b2-89bf-4150-bed7-7b8832573dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6809","nonce":2222268646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6811","nonce":2222268646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6815","nonce":2222268646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2222268646},{"type":"v1","addr":"172.21.15.154:6813","nonce":2222268646}]},"public_addr":"172.21.15.154:6809/2222268646","cluster_addr":"172.21.15.154:6811/2222268646","heartbeat_back_addr":"172.21.15.154:6815/2222268646","heartbeat_front_addr":"172.21.15.154:6813/2222268646","state":["exists","up"]},{"osd":6,"uuid":"4c70b627-eafa-4e27-ba2a-4d8e1e335962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6819","nonce":384734939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6821","nonce":384734939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6825","nonce":384734939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":384734939},{"type":"v1","addr":"172.21.15.45:6823","nonce":384734939}]},"public_addr":"172.21.15.45:6819/384734939","cluster_addr":"172.21.15.45:6821/384734939","heartbeat_back_addr":"172.21.15.45:6825/384734939","heartbeat_front_addr":"172.21.15.45:6823/384734939","state":["exists","up"]},{"osd":7,"uuid":"3b09b875-be52-4be0-b116-7fd9b0c26705","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6817","nonce":1119791005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6819","nonce":1119791005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6823","nonce":1119791005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1119791005},{"type":"v1","addr":"172.21.15.86:6821","nonce":1119791005}]},"public_addr":"172.21.15.86:6817/1119791005","cluster_addr":"172.21.15.86:6819/1119791005","heartbeat_back_addr":"172.21.15.86:6823/1119791005","heartbeat_front_addr":"172.21.15.86:6821/1119791005","state":["exists","up"]},{"osd":8,"uuid":"cdf1dd1b-5392-4836-8e65-4c7c5b688d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6817","nonce":3054812496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6819","nonce":3054812496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6823","nonce":3054812496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3054812496},{"type":"v1","addr":"172.21.15.154:6821","nonce":3054812496}]},"public_addr":"172.21.15.154:6817/3054812496","cluster_addr":"172.21.15.154:6819/3054812496","heartbeat_back_addr":"172.21.15.154:6823/3054812496","heartbeat_front_addr":"172.21.15.154:6821/3054812496","state":["exists","up"]},{"osd":9,"uuid":"acede92d-2cc1-4967-a357-7e5e201e99d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6827","nonce":1360655510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6829","nonce":1360655510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6833","nonce":1360655510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1360655510},{"type":"v1","addr":"172.21.15.45:6831","nonce":1360655510}]},"public_addr":"172.21.15.45:6827/1360655510","cluster_addr":"172.21.15.45:6829/1360655510","heartbeat_back_addr":"172.21.15.45:6833/1360655510","heartbeat_front_addr":"172.21.15.45:6831/1360655510","state":["exists","up"]},{"osd":10,"uuid":"a2d0c488-1892-4da4-adbf-952c22df0ce6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6825","nonce":1701691877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6827","nonce":1701691877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6831","nonce":1701691877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1701691877},{"type":"v1","addr":"172.21.15.86:6829","nonce":1701691877}]},"public_addr":"172.21.15.86:6825/1701691877","cluster_addr":"172.21.15.86:6827/1701691877","heartbeat_back_addr":"172.21.15.86:6831/1701691877","heartbeat_front_addr":"172.21.15.86:6829/1701691877","state":["exists","up"]},{"osd":11,"uuid":"6b349c4c-b672-480b-ac6a-8594dab0eaca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6825","nonce":2065648007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6827","nonce":2065648007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6831","nonce":2065648007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2065648007},{"type":"v1","addr":"172.21.15.154:6829","nonce":2065648007}]},"public_addr":"172.21.15.154:6825/2065648007","cluster_addr":"172.21.15.154:6827/2065648007","heartbeat_back_addr":"172.21.15.154:6831/2065648007","heartbeat_front_addr":"172.21.15.154:6829/2065648007","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:21:37.147394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:00.776617+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:24.786849+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:22:50.487455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:14.869272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:23:39.815158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:05.972619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:31.410063+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:24:56.795119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:23.114884+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:25:49.256504+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T04:26:15.251967+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/36732849":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/1500318775":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6801/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:6800/3465796147":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/2041380391":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/2504003341":"2024-06-07T04:19:37.026199+0000","172.21.15.45:0/3798552599":"2024-06-07T04:20:04.267269+0000","172.21.15.45:0/498952822":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6800/1562511056":"2024-06-07T04:19:37.026199+0000","172.21.15.45:6801/1562511056":"2024-06-07T04:19:37.026199+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T04:35:38.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.384+0000 7f58fc7dd700 1 -- 172.21.15.45:0/3941550966 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f58e00a2c20 msgr2=0x7f58e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:38.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.384+0000 7f58fc7dd700 1 --2- 172.21.15.45:0/3941550966 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f58e00a2c20 0x7f58e00a50e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f58f0005e10 tx=0x7f58f0005da0 comp rx=0 tx=0).stop 2024-06-06T04:35:38.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.384+0000 7f58fc7dd700 1 -- 172.21.15.45:0/3941550966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58f4102e80 msgr2=0x7f58f419ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:38.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.384+0000 7f58fc7dd700 1 --2- 172.21.15.45:0/3941550966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58f4102e80 0x7f58f419ec90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f58e400d950 tx=0x7f58e400dc60 comp rx=0 tx=0).stop 2024-06-06T04:35:38.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.384+0000 7f58fc7dd700 1 -- 172.21.15.45:0/3941550966 shutdown_connections 2024-06-06T04:35:38.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.384+0000 7f58fc7dd700 1 --2- 172.21.15.45:0/3941550966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58f41084f0 0x7f58f4198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:38.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.384+0000 7f58fc7dd700 1 --2- 172.21.15.45:0/3941550966 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f58e00a2c20 0x7f58e00a50e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:38.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.384+0000 7f58fc7dd700 1 --2- 172.21.15.45:0/3941550966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58f41024e0 0x7f58f419e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:38.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.384+0000 7f58fc7dd700 1 --2- 172.21.15.45:0/3941550966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58f4102e80 0x7f58f419ec90 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:38.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.384+0000 7f58fc7dd700 1 -- 172.21.15.45:0/3941550966 >> 172.21.15.45:0/3941550966 conn(0x7f58f4079530 msgr2=0x7f58f4100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:38.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.384+0000 7f58fc7dd700 1 -- 172.21.15.45:0/3941550966 shutdown_connections 2024-06-06T04:35:38.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:38.384+0000 7f58fc7dd700 1 -- 172.21.15.45:0/3941550966 wait complete. 2024-06-06T04:35:39.035 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs fail cephfs 2024-06-06T04:35:39.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:39 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/3941550966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:35:39.289 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:35:39.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:39 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/3941550966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:35:39.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:39 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/3941550966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T04:35:40.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:40 smithi086 ceph-mon[36053]: pgmap v557: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-06-06T04:35:40.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:40 smithi154 ceph-mon[36428]: pgmap v557: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-06-06T04:35:40.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:40 smithi045 ceph-mon[32299]: pgmap v557: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-06-06T04:35:41.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.098+0000 7f64ad2a6700 1 -- 172.21.15.45:0/3543250165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8107840 msgr2=0x7f64a810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:41.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.098+0000 7f64ad2a6700 1 --2- 172.21.15.45:0/3543250165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8107840 0x7f64a810f3c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f64a000b0d0 tx=0x7f64a000b3e0 comp rx=0 tx=0).stop 2024-06-06T04:35:41.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.098+0000 7f64ad2a6700 1 -- 172.21.15.45:0/3543250165 shutdown_connections 2024-06-06T04:35:41.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.098+0000 7f64ad2a6700 1 --2- 172.21.15.45:0/3543250165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8107840 0x7f64a810f3c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:41.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.098+0000 7f64ad2a6700 1 --2- 172.21.15.45:0/3543250165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64a8102740 0x7f64a8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:41.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.098+0000 7f64ad2a6700 1 --2- 172.21.15.45:0/3543250165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64a81030f0 0x7f64a8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:41.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.098+0000 7f64ad2a6700 1 -- 172.21.15.45:0/3543250165 >> 172.21.15.45:0/3543250165 conn(0x7f64a80795f0 msgr2=0x7f64a8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:41.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.098+0000 7f64ad2a6700 1 -- 172.21.15.45:0/3543250165 shutdown_connections 2024-06-06T04:35:41.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.098+0000 7f64ad2a6700 1 -- 172.21.15.45:0/3543250165 wait complete. 2024-06-06T04:35:41.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.099+0000 7f64ad2a6700 1 Processor -- start 2024-06-06T04:35:41.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.099+0000 7f64ad2a6700 1 -- start start 2024-06-06T04:35:41.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64ad2a6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64a8102740 0x7f64a81a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:41.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64ad2a6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64a81030f0 0x7f64a81a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:41.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64ad2a6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8107840 0x7f64a819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:41.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64ad2a6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f64a81126a0 con 0x7f64a8107840 2024-06-06T04:35:41.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64ad2a6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64a8112520 con 0x7f64a8102740 2024-06-06T04:35:41.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64ad2a6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f64a8112820 con 0x7f64a81030f0 2024-06-06T04:35:41.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64a77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8107840 0x7f64a819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:41.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64a77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8107840 0x7f64a819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35294/0 (socket says 172.21.15.45:35294) 2024-06-06T04:35:41.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64a77fe700 1 -- 172.21.15.45:0/591875880 learned_addr learned my addr 172.21.15.45:0/591875880 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:35:41.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64a6ffd700 1 --2- 172.21.15.45:0/591875880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64a8102740 0x7f64a81a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:41.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64a67fc700 1 --2- 172.21.15.45:0/591875880 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64a81030f0 0x7f64a81a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:41.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64a77fe700 1 -- 172.21.15.45:0/591875880 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64a81030f0 msgr2=0x7f64a81a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:41.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64a77fe700 1 --2- 172.21.15.45:0/591875880 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64a81030f0 0x7f64a81a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:41.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64a77fe700 1 -- 172.21.15.45:0/591875880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64a8102740 msgr2=0x7f64a81a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:41.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64a77fe700 1 --2- 172.21.15.45:0/591875880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64a8102740 0x7f64a81a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:41.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64a77fe700 1 -- 172.21.15.45:0/591875880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a819b2a0 con 0x7f64a8107840 2024-06-06T04:35:41.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.100+0000 7f64a67fc700 1 --2- 172.21.15.45:0/591875880 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64a81030f0 0x7f64a81a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:35:41.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.101+0000 7f64a77fe700 1 --2- 172.21.15.45:0/591875880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8107840 0x7f64a819ab40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f64a00095c0 tx=0x7f64a0007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:41.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.101+0000 7f648ffff700 1 -- 172.21.15.45:0/591875880 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a001f070 con 0x7f64a8107840 2024-06-06T04:35:41.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.101+0000 7f648ffff700 1 -- 172.21.15.45:0/591875880 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64a0003c00 con 0x7f64a8107840 2024-06-06T04:35:41.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.101+0000 7f64ad2a6700 1 -- 172.21.15.45:0/591875880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a819b530 con 0x7f64a8107840 2024-06-06T04:35:41.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.101+0000 7f648ffff700 1 -- 172.21.15.45:0/591875880 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a001b6a0 con 0x7f64a8107840 2024-06-06T04:35:41.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.101+0000 7f64ad2a6700 1 -- 172.21.15.45:0/591875880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a81a7800 con 0x7f64a8107840 2024-06-06T04:35:41.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.102+0000 7f64ad2a6700 1 -- 172.21.15.45:0/591875880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a8069180 con 0x7f64a8107840 2024-06-06T04:35:41.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.103+0000 7f648ffff700 1 -- 172.21.15.45:0/591875880 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f64a0022070 con 0x7f64a8107840 2024-06-06T04:35:41.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.104+0000 7f648ffff700 1 --2- 172.21.15.45:0/591875880 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f64900a2c20 0x7f64900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:41.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.104+0000 7f64a6ffd700 1 --2- 172.21.15.45:0/591875880 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f64900a2c20 0x7f64900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:41.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.104+0000 7f648ffff700 1 -- 172.21.15.45:0/591875880 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9187+0+0 (secure 0 0 0) 0x7f64a000f020 con 0x7f64a8107840 2024-06-06T04:35:41.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.104+0000 7f64a6ffd700 1 --2- 172.21.15.45:0/591875880 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f64900a2c20 0x7f64900a50e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6498004630 tx=0x7f649800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:41.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.108+0000 7f648ffff700 1 -- 172.21.15.45:0/591875880 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f64a0029020 con 0x7f64a8107840 2024-06-06T04:35:41.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.362+0000 7f64ad2a6700 1 -- 172.21.15.45:0/591875880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f64a81a79d0 con 0x7f64a8107840 2024-06-06T04:35:41.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.365+0000 7f648ffff700 1 -- 172.21.15.45:0/591875880 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) ==== 161+0+0 (secure 0 0 0) 0x7f64a009d240 con 0x7f64a8107840 2024-06-06T04:35:41.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.367+0000 7f64ad2a6700 1 -- 172.21.15.45:0/591875880 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f64900a2c20 msgr2=0x7f64900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:41.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.367+0000 7f64ad2a6700 1 --2- 172.21.15.45:0/591875880 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f64900a2c20 0x7f64900a50e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6498004630 tx=0x7f649800b3c0 comp rx=0 tx=0).stop 2024-06-06T04:35:41.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.367+0000 7f64ad2a6700 1 -- 172.21.15.45:0/591875880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8107840 msgr2=0x7f64a819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:41.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.367+0000 7f64ad2a6700 1 --2- 172.21.15.45:0/591875880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8107840 0x7f64a819ab40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f64a00095c0 tx=0x7f64a0007a20 comp rx=0 tx=0).stop 2024-06-06T04:35:41.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.367+0000 7f64ad2a6700 1 -- 172.21.15.45:0/591875880 shutdown_connections 2024-06-06T04:35:41.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.367+0000 7f64ad2a6700 1 --2- 172.21.15.45:0/591875880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8107840 0x7f64a819ab40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:41.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.367+0000 7f64ad2a6700 1 --2- 172.21.15.45:0/591875880 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f64900a2c20 0x7f64900a50e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:41.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.367+0000 7f64ad2a6700 1 --2- 172.21.15.45:0/591875880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64a8102740 0x7f64a81a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:41.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.367+0000 7f64ad2a6700 1 --2- 172.21.15.45:0/591875880 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64a81030f0 0x7f64a81a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:41.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.367+0000 7f64ad2a6700 1 -- 172.21.15.45:0/591875880 >> 172.21.15.45:0/591875880 conn(0x7f64a80795f0 msgr2=0x7f64a8100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:41.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.368+0000 7f64ad2a6700 1 -- 172.21.15.45:0/591875880 shutdown_connections 2024-06-06T04:35:41.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:41.368+0000 7f64ad2a6700 1 -- 172.21.15.45:0/591875880 wait complete. 2024-06-06T04:35:41.370 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-06-06T04:35:41.680 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:35:41 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-f[66373]: ignoring --setuser ceph since I am not root 2024-06-06T04:35:41.681 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:35:41 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-f[66373]: ignoring --setgroup ceph since I am not root 2024-06-06T04:35:41.681 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:35:41 smithi154 ceph-mds[66377]: -- 172.21.15.154:0/4162544122 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559bfc371a40 con 0x559bfc345000 2024-06-06T04:35:41.681 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:35:41 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-f[66373]: starting mds.f at 2024-06-06T04:35:41.681 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:35:41 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-c[66674]: ignoring --setuser ceph since I am not root 2024-06-06T04:35:41.681 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:35:41 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-c[66674]: ignoring --setgroup ceph since I am not root 2024-06-06T04:35:41.682 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:35:41 smithi154 ceph-mds[66678]: -- 172.21.15.154:0/1947363068 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b07c347a40 con 0x55b07c31a800 2024-06-06T04:35:41.682 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:35:41 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-c[66674]: starting mds.c at 2024-06-06T04:35:41.682 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:35:41 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-i[66974]: ignoring --setuser ceph since I am not root 2024-06-06T04:35:41.682 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:35:41 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-i[66974]: ignoring --setgroup ceph since I am not root 2024-06-06T04:35:41.682 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:35:41 smithi154 ceph-mds[66978]: -- 172.21.15.154:0/4265414915 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5645ef093a40 con 0x5645ef066800 2024-06-06T04:35:41.682 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:35:41 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-i[66974]: starting mds.i at 2024-06-06T04:35:41.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-a[32295]: 2024-06-06T04:35:41.363+0000 7fc9ae662700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-06T04:35:41.859 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:35:41 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-b[65935]: ignoring --setuser ceph since I am not root 2024-06-06T04:35:41.859 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:35:41 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-b[65935]: ignoring --setgroup ceph since I am not root 2024-06-06T04:35:41.859 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:35:41 smithi086 ceph-mds[65939]: -- 172.21.15.86:0/1606636297 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56158c7cda40 con 0x56158c7a0c00 2024-06-06T04:35:41.859 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:35:41 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-b[65935]: starting mds.b at 2024-06-06T04:35:41.860 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:35:41 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-h[65630]: ignoring --setuser ceph since I am not root 2024-06-06T04:35:41.860 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:35:41 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-h[65630]: ignoring --setgroup ceph since I am not root 2024-06-06T04:35:41.860 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:35:41 smithi086 ceph-mds[65634]: -- 172.21.15.86:0/3289925456 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aefa54ba40 con 0x55aefa51ec00 2024-06-06T04:35:41.860 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:35:41 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-h[65630]: starting mds.h at 2024-06-06T04:35:41.860 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:35:41 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-e[65320]: ignoring --setuser ceph since I am not root 2024-06-06T04:35:41.860 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:35:41 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-e[65320]: ignoring --setgroup ceph since I am not root 2024-06-06T04:35:41.861 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:35:41 smithi086 ceph-mds[65324]: -- 172.21.15.86:0/3736778844 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b692f67a40 con 0x55b692f3ac00 2024-06-06T04:35:41.861 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:35:41 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-e[65320]: starting mds.e at 2024-06-06T04:35:42.071 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-06-06T04:35:42.322 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:35:42.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:42 smithi086 ceph-mon[36053]: pgmap v558: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-06-06T04:35:42.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:42 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/591875880' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-06T04:35:42.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:42 smithi086 ceph-mon[36053]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-06T04:35:42.359 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:42 smithi086 ceph-mon[36053]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-06T04:35:42.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:42 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/591875880' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-06T04:35:42.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:42 smithi086 ceph-mon[36053]: osdmap e101: 12 total, 12 up, 12 in 2024-06-06T04:35:42.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:42 smithi086 ceph-mon[36053]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-06T04:35:42.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:42 smithi154 ceph-mon[36428]: pgmap v558: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-06-06T04:35:42.430 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:42 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/591875880' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-06T04:35:42.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:42 smithi154 ceph-mon[36428]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-06T04:35:42.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:42 smithi154 ceph-mon[36428]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-06T04:35:42.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:42 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/591875880' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-06T04:35:42.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:42 smithi154 ceph-mon[36428]: osdmap e101: 12 total, 12 up, 12 in 2024-06-06T04:35:42.431 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:42 smithi154 ceph-mon[36428]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-06T04:35:42.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:42 smithi045 ceph-mon[32299]: pgmap v558: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-06-06T04:35:42.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:42 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/591875880' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-06T04:35:42.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:42 smithi045 ceph-mon[32299]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-06T04:35:42.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:42 smithi045 ceph-mon[32299]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-06T04:35:42.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:42 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/591875880' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-06T04:35:42.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:42 smithi045 ceph-mon[32299]: osdmap e101: 12 total, 12 up, 12 in 2024-06-06T04:35:42.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:42 smithi045 ceph-mon[32299]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-06T04:35:43.658 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:43 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.86:6834/3021617529,v1:172.21.15.86:6837/3021617529] up:boot 2024-06-06T04:35:43.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:43 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.86:6835/2006388578,v1:172.21.15.86:6836/2006388578] up:boot 2024-06-06T04:35:43.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:43 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.86:6838/3292654112,v1:172.21.15.86:6839/3292654112] up:boot 2024-06-06T04:35:43.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:43 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.154:6835/1787601304,v1:172.21.15.154:6837/1787601304] up:boot 2024-06-06T04:35:43.659 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:43 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.154:6834/784189179,v1:172.21.15.154:6836/784189179] up:boot 2024-06-06T04:35:43.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:43 smithi045 ceph-mon[32299]: mds.? [v2:172.21.15.154:6838/1918167757,v1:172.21.15.154:6839/1918167757] up:boot 2024-06-06T04:35:43.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:43 smithi045 ceph-mon[32299]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-06T04:35:43.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:43 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T04:35:43.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:43 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T04:35:43.660 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:43 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T04:35:43.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:43 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T04:35:43.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:43 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T04:35:43.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:43 smithi045 ceph-mon[32299]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T04:35:43.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:43 smithi045 ceph-mon[32299]: pgmap v560: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-06T04:35:43.682 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:43 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.86:6834/3021617529,v1:172.21.15.86:6837/3021617529] up:boot 2024-06-06T04:35:43.683 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:43 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.86:6835/2006388578,v1:172.21.15.86:6836/2006388578] up:boot 2024-06-06T04:35:43.683 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:43 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.86:6838/3292654112,v1:172.21.15.86:6839/3292654112] up:boot 2024-06-06T04:35:43.683 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:43 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.154:6835/1787601304,v1:172.21.15.154:6837/1787601304] up:boot 2024-06-06T04:35:43.683 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:43 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.154:6834/784189179,v1:172.21.15.154:6836/784189179] up:boot 2024-06-06T04:35:43.683 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:43 smithi154 ceph-mon[36428]: mds.? [v2:172.21.15.154:6838/1918167757,v1:172.21.15.154:6839/1918167757] up:boot 2024-06-06T04:35:43.683 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:43 smithi154 ceph-mon[36428]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-06T04:35:43.683 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:43 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T04:35:43.683 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:43 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T04:35:43.684 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:43 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T04:35:43.684 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:43 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T04:35:43.684 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:43 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T04:35:43.684 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:43 smithi154 ceph-mon[36428]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T04:35:43.684 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:43 smithi154 ceph-mon[36428]: pgmap v560: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-06T04:35:43.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:43 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.86:6834/3021617529,v1:172.21.15.86:6837/3021617529] up:boot 2024-06-06T04:35:43.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:43 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.86:6835/2006388578,v1:172.21.15.86:6836/2006388578] up:boot 2024-06-06T04:35:43.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:43 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.86:6838/3292654112,v1:172.21.15.86:6839/3292654112] up:boot 2024-06-06T04:35:43.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:43 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.154:6835/1787601304,v1:172.21.15.154:6837/1787601304] up:boot 2024-06-06T04:35:43.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:43 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.154:6834/784189179,v1:172.21.15.154:6836/784189179] up:boot 2024-06-06T04:35:43.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:43 smithi086 ceph-mon[36053]: mds.? [v2:172.21.15.154:6838/1918167757,v1:172.21.15.154:6839/1918167757] up:boot 2024-06-06T04:35:43.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:43 smithi086 ceph-mon[36053]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-06T04:35:43.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:43 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T04:35:43.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:43 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T04:35:43.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:43 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T04:35:43.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:43 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T04:35:43.861 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:43 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T04:35:43.862 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:43 smithi086 ceph-mon[36053]: from='mgr.14152 172.21.15.45:0/254281267' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T04:35:43.862 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:43 smithi086 ceph-mon[36053]: pgmap v560: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-06T04:35:44.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.047+0000 7faffdbfc700 1 -- 172.21.15.45:0/4168889973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff806ac60 msgr2=0x7faff806b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:44.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.047+0000 7faffdbfc700 1 --2- 172.21.15.45:0/4168889973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff806ac60 0x7faff806b0e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fafec0099c0 tx=0x7fafec009cd0 comp rx=0 tx=0).stop 2024-06-06T04:35:44.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.047+0000 7faffdbfc700 1 -- 172.21.15.45:0/4168889973 shutdown_connections 2024-06-06T04:35:44.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.047+0000 7faffdbfc700 1 --2- 172.21.15.45:0/4168889973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faff806b620 0x7faff810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:44.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.047+0000 7faffdbfc700 1 --2- 172.21.15.45:0/4168889973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faff8108580 0x7faff8108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:44.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.047+0000 7faffdbfc700 1 --2- 172.21.15.45:0/4168889973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff806ac60 0x7faff806b0e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:44.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.047+0000 7faffdbfc700 1 -- 172.21.15.45:0/4168889973 >> 172.21.15.45:0/4168889973 conn(0x7faff8078430 msgr2=0x7faff8078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:44.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.047+0000 7faffdbfc700 1 -- 172.21.15.45:0/4168889973 shutdown_connections 2024-06-06T04:35:44.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.047+0000 7faffdbfc700 1 -- 172.21.15.45:0/4168889973 wait complete. 2024-06-06T04:35:44.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.048+0000 7faffdbfc700 1 Processor -- start 2024-06-06T04:35:44.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.048+0000 7faffdbfc700 1 -- start start 2024-06-06T04:35:44.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.048+0000 7faffdbfc700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faff806ac60 0x7faff81a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.048+0000 7faffdbfc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff806b620 0x7faff81a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.048+0000 7faffdbfc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faff8108580 0x7faff819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.048+0000 7faffdbfc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faff8112670 con 0x7faff806b620 2024-06-06T04:35:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.048+0000 7faffdbfc700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faff81124f0 con 0x7faff806ac60 2024-06-06T04:35:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.048+0000 7faffdbfc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faff81127f0 con 0x7faff8108580 2024-06-06T04:35:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.048+0000 7faff6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff806b620 0x7faff81a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.048+0000 7faff77fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faff806ac60 0x7faff81a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.048+0000 7faff6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff806b620 0x7faff81a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35316/0 (socket says 172.21.15.45:35316) 2024-06-06T04:35:44.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.048+0000 7faff6ffd700 1 -- 172.21.15.45:0/2479534489 learned_addr learned my addr 172.21.15.45:0/2479534489 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:35:44.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.049+0000 7faff7fff700 1 --2- 172.21.15.45:0/2479534489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faff8108580 0x7faff819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:44.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.049+0000 7faff6ffd700 1 -- 172.21.15.45:0/2479534489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faff8108580 msgr2=0x7faff819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:44.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.049+0000 7faff6ffd700 1 --2- 172.21.15.45:0/2479534489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faff8108580 0x7faff819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:44.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.049+0000 7faff6ffd700 1 -- 172.21.15.45:0/2479534489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faff806ac60 msgr2=0x7faff81a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:44.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.049+0000 7faff6ffd700 1 --2- 172.21.15.45:0/2479534489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faff806ac60 0x7faff81a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:44.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.049+0000 7faff6ffd700 1 -- 172.21.15.45:0/2479534489 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff819b400 con 0x7faff806b620 2024-06-06T04:35:44.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.049+0000 7faff77fe700 1 --2- 172.21.15.45:0/2479534489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faff806ac60 0x7faff81a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:35:44.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.049+0000 7faff7fff700 1 --2- 172.21.15.45:0/2479534489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faff8108580 0x7faff819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:35:44.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.049+0000 7faff6ffd700 1 --2- 172.21.15.45:0/2479534489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff806b620 0x7faff81a0f90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fafec006010 tx=0x7fafec0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:44.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.049+0000 7faff4ff9700 1 -- 172.21.15.45:0/2479534489 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec026070 con 0x7faff806b620 2024-06-06T04:35:44.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.049+0000 7faff4ff9700 1 -- 172.21.15.45:0/2479534489 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafec00f460 con 0x7faff806b620 2024-06-06T04:35:44.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.049+0000 7faff4ff9700 1 -- 172.21.15.45:0/2479534489 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec009e60 con 0x7faff806b620 2024-06-06T04:35:44.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.049+0000 7faffdbfc700 1 -- 172.21.15.45:0/2479534489 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff819b690 con 0x7faff806b620 2024-06-06T04:35:44.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.049+0000 7faffdbfc700 1 -- 172.21.15.45:0/2479534489 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff81a7800 con 0x7faff806b620 2024-06-06T04:35:44.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.050+0000 7faffdbfc700 1 -- 172.21.15.45:0/2479534489 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff8103bf0 con 0x7faff806b620 2024-06-06T04:35:44.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.051+0000 7faff4ff9700 1 -- 172.21.15.45:0/2479534489 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fafec029070 con 0x7faff806b620 2024-06-06T04:35:44.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.052+0000 7faff4ff9700 1 --2- 172.21.15.45:0/2479534489 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fafe00a2c20 0x7fafe00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:44.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.052+0000 7faff77fe700 1 --2- 172.21.15.45:0/2479534489 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fafe00a2c20 0x7fafe00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:44.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.052+0000 7faff4ff9700 1 -- 172.21.15.45:0/2479534489 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9445+0+0 (secure 0 0 0) 0x7fafec0cffb0 con 0x7faff806b620 2024-06-06T04:35:44.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.053+0000 7faff77fe700 1 --2- 172.21.15.45:0/2479534489 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fafe00a2c20 0x7fafe00a50e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009fd0 tx=0x7fafe80091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:44.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.056+0000 7faff4ff9700 1 -- 172.21.15.45:0/2479534489 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fafec030080 con 0x7faff806b620 2024-06-06T04:35:44.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.310+0000 7faffdbfc700 1 -- 172.21.15.45:0/2479534489 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7faff81a79d0 con 0x7faff806b620 2024-06-06T04:35:44.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.379+0000 7faff4ff9700 1 -- 172.21.15.45:0/2479534489 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) ==== 104+0+0 (secure 0 0 0) 0x7fafec013070 con 0x7faff806b620 2024-06-06T04:35:44.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.382+0000 7faffdbfc700 1 -- 172.21.15.45:0/2479534489 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fafe00a2c20 msgr2=0x7fafe00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:44.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.382+0000 7faffdbfc700 1 --2- 172.21.15.45:0/2479534489 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fafe00a2c20 0x7fafe00a50e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009fd0 tx=0x7fafe80091b0 comp rx=0 tx=0).stop 2024-06-06T04:35:44.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.382+0000 7faffdbfc700 1 -- 172.21.15.45:0/2479534489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff806b620 msgr2=0x7faff81a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.382+0000 7faffdbfc700 1 --2- 172.21.15.45:0/2479534489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff806b620 0x7faff81a0f90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fafec006010 tx=0x7fafec0045e0 comp rx=0 tx=0).stop 2024-06-06T04:35:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.382+0000 7faffdbfc700 1 -- 172.21.15.45:0/2479534489 shutdown_connections 2024-06-06T04:35:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.382+0000 7faffdbfc700 1 --2- 172.21.15.45:0/2479534489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faff8108580 0x7faff819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.382+0000 7faffdbfc700 1 --2- 172.21.15.45:0/2479534489 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7fafe00a2c20 0x7fafe00a50e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.382+0000 7faffdbfc700 1 --2- 172.21.15.45:0/2479534489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faff806ac60 0x7faff81a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.382+0000 7faffdbfc700 1 --2- 172.21.15.45:0/2479534489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff806b620 0x7faff81a0f90 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.382+0000 7faffdbfc700 1 -- 172.21.15.45:0/2479534489 >> 172.21.15.45:0/2479534489 conn(0x7faff8078430 msgr2=0x7faff810d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:44.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.382+0000 7faffdbfc700 1 -- 172.21.15.45:0/2479534489 shutdown_connections 2024-06-06T04:35:44.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:44.382+0000 7faffdbfc700 1 -- 172.21.15.45:0/2479534489 wait complete. 2024-06-06T04:35:44.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:44 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2479534489' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T04:35:44.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:44 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2479534489' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T04:35:44.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:44 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2479534489' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T04:35:45.082 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-06-06T04:35:45.335 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:35:45.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:45 smithi154 ceph-mon[36428]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-06T04:35:45.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:45 smithi154 ceph-mon[36428]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-06T04:35:45.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:45 smithi154 ceph-mon[36428]: Cluster is now healthy 2024-06-06T04:35:45.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:45 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2479534489' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-06T04:35:45.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:45 smithi154 ceph-mon[36428]: fsmap 12 up:standby 2024-06-06T04:35:45.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:45 smithi154 ceph-mon[36428]: pgmap v561: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 5 op/s 2024-06-06T04:35:45.698 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:45 smithi045 ceph-mon[32299]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-06T04:35:45.699 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:45 smithi045 ceph-mon[32299]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-06T04:35:45.699 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:45 smithi045 ceph-mon[32299]: Cluster is now healthy 2024-06-06T04:35:45.699 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:45 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2479534489' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-06T04:35:45.699 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:45 smithi045 ceph-mon[32299]: fsmap 12 up:standby 2024-06-06T04:35:45.699 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:45 smithi045 ceph-mon[32299]: pgmap v561: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 5 op/s 2024-06-06T04:35:45.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:45 smithi086 ceph-mon[36053]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-06T04:35:45.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:45 smithi086 ceph-mon[36053]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-06T04:35:45.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:45 smithi086 ceph-mon[36053]: Cluster is now healthy 2024-06-06T04:35:45.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:45 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2479534489' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-06T04:35:45.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:45 smithi086 ceph-mon[36053]: fsmap 12 up:standby 2024-06-06T04:35:45.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:45 smithi086 ceph-mon[36053]: pgmap v561: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 5 op/s 2024-06-06T04:35:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.143+0000 7f560303e700 1 -- 172.21.15.45:0/1556912760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55fc074fe0 msgr2=0x7f55fc10e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.143+0000 7f560303e700 1 --2- 172.21.15.45:0/1556912760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55fc074fe0 0x7f55fc10e950 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009990 tx=0x7f55e4009ca0 comp rx=0 tx=0).stop 2024-06-06T04:35:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.143+0000 7f560303e700 1 -- 172.21.15.45:0/1556912760 shutdown_connections 2024-06-06T04:35:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.143+0000 7f560303e700 1 --2- 172.21.15.45:0/1556912760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc10ee90 0x7f55fc111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.143+0000 7f560303e700 1 --2- 172.21.15.45:0/1556912760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55fc0746c0 0x7f55fc074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.143+0000 7f560303e700 1 --2- 172.21.15.45:0/1556912760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55fc074fe0 0x7f55fc10e950 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.143+0000 7f560303e700 1 -- 172.21.15.45:0/1556912760 >> 172.21.15.45:0/1556912760 conn(0x7f55fc079640 msgr2=0x7f55fc079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.143+0000 7f560303e700 1 -- 172.21.15.45:0/1556912760 shutdown_connections 2024-06-06T04:35:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.143+0000 7f560303e700 1 -- 172.21.15.45:0/1556912760 wait complete. 2024-06-06T04:35:47.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.144+0000 7f560303e700 1 Processor -- start 2024-06-06T04:35:47.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.144+0000 7f560303e700 1 -- start start 2024-06-06T04:35:47.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.144+0000 7f560303e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55fc0746c0 0x7f55fc19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:47.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.144+0000 7f560303e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55fc074fe0 0x7f55fc19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:47.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.144+0000 7f560303e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc10ee90 0x7f55fc19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:47.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.144+0000 7f560303e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc1148d0 con 0x7f55fc0746c0 2024-06-06T04:35:47.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.144+0000 7f560303e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc114750 con 0x7f55fc074fe0 2024-06-06T04:35:47.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.144+0000 7f560303e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc114a50 con 0x7f55fc10ee90 2024-06-06T04:35:47.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f5600dda700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55fc0746c0 0x7f55fc19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:47.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f5600dda700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55fc0746c0 0x7f55fc19adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35346/0 (socket says 172.21.15.45:35346) 2024-06-06T04:35:47.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f5600dda700 1 -- 172.21.15.45:0/4066803867 learned_addr learned my addr 172.21.15.45:0/4066803867 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:35:47.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f55fbfff700 1 --2- 172.21.15.45:0/4066803867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55fc074fe0 0x7f55fc19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:47.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f56015db700 1 --2- 172.21.15.45:0/4066803867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc10ee90 0x7f55fc19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:47.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f5600dda700 1 -- 172.21.15.45:0/4066803867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc10ee90 msgr2=0x7f55fc19f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:47.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f5600dda700 1 --2- 172.21.15.45:0/4066803867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc10ee90 0x7f55fc19f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:47.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f5600dda700 1 -- 172.21.15.45:0/4066803867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55fc074fe0 msgr2=0x7f55fc19b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:47.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f5600dda700 1 --2- 172.21.15.45:0/4066803867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55fc074fe0 0x7f55fc19b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:47.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f5600dda700 1 -- 172.21.15.45:0/4066803867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55fc19fdd0 con 0x7f55fc0746c0 2024-06-06T04:35:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f55fbfff700 1 --2- 172.21.15.45:0/4066803867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55fc074fe0 0x7f55fc19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:35:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f5600dda700 1 --2- 172.21.15.45:0/4066803867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55fc0746c0 0x7f55fc19adc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00ef10 tx=0x7f55ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f55f9ffb700 1 -- 172.21.15.45:0/4066803867 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00ca70 con 0x7f55fc0746c0 2024-06-06T04:35:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f560303e700 1 -- 172.21.15.45:0/4066803867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55fc1a0060 con 0x7f55fc0746c0 2024-06-06T04:35:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f55f9ffb700 1 -- 172.21.15.45:0/4066803867 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55ec019070 con 0x7f55fc0746c0 2024-06-06T04:35:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f55f9ffb700 1 -- 172.21.15.45:0/4066803867 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec014480 con 0x7f55fc0746c0 2024-06-06T04:35:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.145+0000 7f560303e700 1 -- 172.21.15.45:0/4066803867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55fc1a7940 con 0x7f55fc0746c0 2024-06-06T04:35:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.146+0000 7f560303e700 1 -- 172.21.15.45:0/4066803867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55fc069180 con 0x7f55fc0746c0 2024-06-06T04:35:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.147+0000 7f55f9ffb700 1 -- 172.21.15.45:0/4066803867 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f55ec012050 con 0x7f55fc0746c0 2024-06-06T04:35:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.148+0000 7f55f9ffb700 1 --2- 172.21.15.45:0/4066803867 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f55e80a2c20 0x7f55e80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.148+0000 7f55fbfff700 1 --2- 172.21.15.45:0/4066803867 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f55e80a2c20 0x7f55e80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:47.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.148+0000 7f55f9ffb700 1 -- 172.21.15.45:0/4066803867 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9445+0+0 (secure 0 0 0) 0x7f55ec0cf5f0 con 0x7f55fc0746c0 2024-06-06T04:35:47.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.149+0000 7f55fbfff700 1 --2- 172.21.15.45:0/4066803867 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f55e80a2c20 0x7f55e80a50e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f55e400b2e0 tx=0x7f55e401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:47.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.153+0000 7f55f9ffb700 1 -- 172.21.15.45:0/4066803867 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f55ec010040 con 0x7f55fc0746c0 2024-06-06T04:35:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.358+0000 7f560303e700 1 -- 172.21.15.45:0/4066803867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f55fc02dca0 con 0x7f55fc0746c0 2024-06-06T04:35:47.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.562+0000 7f55f9ffb700 1 -- 172.21.15.45:0/4066803867 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7f55ec09c030 con 0x7f55fc0746c0 2024-06-06T04:35:47.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.565+0000 7f560303e700 1 -- 172.21.15.45:0/4066803867 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f55e80a2c20 msgr2=0x7f55e80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:47.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.565+0000 7f560303e700 1 --2- 172.21.15.45:0/4066803867 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f55e80a2c20 0x7f55e80a50e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f55e400b2e0 tx=0x7f55e401a040 comp rx=0 tx=0).stop 2024-06-06T04:35:47.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.565+0000 7f560303e700 1 -- 172.21.15.45:0/4066803867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55fc0746c0 msgr2=0x7f55fc19adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:47.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.565+0000 7f560303e700 1 --2- 172.21.15.45:0/4066803867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55fc0746c0 0x7f55fc19adc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00ef10 tx=0x7f55ec00c3a0 comp rx=0 tx=0).stop 2024-06-06T04:35:47.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.566+0000 7f560303e700 1 -- 172.21.15.45:0/4066803867 shutdown_connections 2024-06-06T04:35:47.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.566+0000 7f560303e700 1 --2- 172.21.15.45:0/4066803867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc10ee90 0x7f55fc19f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:47.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.566+0000 7f560303e700 1 --2- 172.21.15.45:0/4066803867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55fc0746c0 0x7f55fc19adc0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:47.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.566+0000 7f560303e700 1 --2- 172.21.15.45:0/4066803867 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f55e80a2c20 0x7f55e80a50e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:47.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.566+0000 7f560303e700 1 --2- 172.21.15.45:0/4066803867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55fc074fe0 0x7f55fc19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:47.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.566+0000 7f560303e700 1 -- 172.21.15.45:0/4066803867 >> 172.21.15.45:0/4066803867 conn(0x7f55fc079640 msgr2=0x7f55fc0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:47.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.567+0000 7f560303e700 1 -- 172.21.15.45:0/4066803867 shutdown_connections 2024-06-06T04:35:47.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:47.567+0000 7f560303e700 1 -- 172.21.15.45:0/4066803867 wait complete. 2024-06-06T04:35:47.568 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-06-06T04:35:47.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:47 smithi086 ceph-mon[36053]: pgmap v562: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 5 op/s 2024-06-06T04:35:47.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:47 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/4066803867' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T04:35:47.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:47 smithi154 ceph-mon[36428]: pgmap v562: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 5 op/s 2024-06-06T04:35:47.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:47 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/4066803867' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T04:35:47.945 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:47 smithi045 ceph-mon[32299]: pgmap v562: 129 pgs: 129 active+clean; 30 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 5 op/s 2024-06-06T04:35:47.946 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:47 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/4066803867' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T04:35:48.362 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-06-06T04:35:48.619 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:35:48.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:48 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/4066803867' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T04:35:48.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:48 smithi045 ceph-mon[32299]: osdmap e102: 12 total, 12 up, 12 in 2024-06-06T04:35:48.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:48 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/4066803867' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T04:35:48.860 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:48 smithi086 ceph-mon[36053]: osdmap e102: 12 total, 12 up, 12 in 2024-06-06T04:35:48.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:48 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/4066803867' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T04:35:48.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:48 smithi154 ceph-mon[36428]: osdmap e102: 12 total, 12 up, 12 in 2024-06-06T04:35:49.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:49 smithi045 ceph-mon[32299]: pgmap v564: 97 pgs: 97 active+clean; 26 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:35:49.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:49 smithi086 ceph-mon[36053]: pgmap v564: 97 pgs: 97 active+clean; 26 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:35:49.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:49 smithi154 ceph-mon[36428]: pgmap v564: 97 pgs: 97 active+clean; 26 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:35:50.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.397+0000 7f36c6370700 1 -- 172.21.15.45:0/2240151038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c01030f0 msgr2=0x7f36c0107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:50.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.397+0000 7f36c6370700 1 --2- 172.21.15.45:0/2240151038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c01030f0 0x7f36c0107110 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f36b40099c0 tx=0x7f36b4009cd0 comp rx=0 tx=0).stop 2024-06-06T04:35:50.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.397+0000 7f36c6370700 1 -- 172.21.15.45:0/2240151038 shutdown_connections 2024-06-06T04:35:50.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.397+0000 7f36c6370700 1 --2- 172.21.15.45:0/2240151038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36c0107840 0x7f36c010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:50.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.397+0000 7f36c6370700 1 --2- 172.21.15.45:0/2240151038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36c0102740 0x7f36c0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:50.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.397+0000 7f36c6370700 1 --2- 172.21.15.45:0/2240151038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c01030f0 0x7f36c0107110 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:50.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.397+0000 7f36c6370700 1 -- 172.21.15.45:0/2240151038 >> 172.21.15.45:0/2240151038 conn(0x7f36c00795f0 msgr2=0x7f36c0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:50.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.397+0000 7f36c6370700 1 -- 172.21.15.45:0/2240151038 shutdown_connections 2024-06-06T04:35:50.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.397+0000 7f36c6370700 1 -- 172.21.15.45:0/2240151038 wait complete. 2024-06-06T04:35:50.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.398+0000 7f36c6370700 1 Processor -- start 2024-06-06T04:35:50.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.398+0000 7f36c6370700 1 -- start start 2024-06-06T04:35:50.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.398+0000 7f36c6370700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36c0102740 0x7f36c019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.398+0000 7f36c6370700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36c01030f0 0x7f36c019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.398+0000 7f36c6370700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c0107840 0x7f36c0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.398+0000 7f36c6370700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f36c0112700 con 0x7f36c0107840 2024-06-06T04:35:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.398+0000 7f36c6370700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36c0112580 con 0x7f36c01030f0 2024-06-06T04:35:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.398+0000 7f36c6370700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f36c0112880 con 0x7f36c0102740 2024-06-06T04:35:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.398+0000 7f36c490d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c0107840 0x7f36c0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.398+0000 7f36c490d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c0107840 0x7f36c0198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46258/0 (socket says 172.21.15.45:46258) 2024-06-06T04:35:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.398+0000 7f36c490d700 1 -- 172.21.15.45:0/2695207121 learned_addr learned my addr 172.21.15.45:0/2695207121 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:35:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.399+0000 7f36bffff700 1 --2- 172.21.15.45:0/2695207121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36c0102740 0x7f36c019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.399+0000 7f36bf7fe700 1 --2- 172.21.15.45:0/2695207121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36c01030f0 0x7f36c019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.399+0000 7f36c490d700 1 -- 172.21.15.45:0/2695207121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36c0102740 msgr2=0x7f36c019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.399+0000 7f36c490d700 1 --2- 172.21.15.45:0/2695207121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36c0102740 0x7f36c019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.399+0000 7f36c490d700 1 -- 172.21.15.45:0/2695207121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36c01030f0 msgr2=0x7f36c019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.399+0000 7f36c490d700 1 --2- 172.21.15.45:0/2695207121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36c01030f0 0x7f36c019ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:50.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.399+0000 7f36c490d700 1 -- 172.21.15.45:0/2695207121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c01991e0 con 0x7f36c0107840 2024-06-06T04:35:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.399+0000 7f36bffff700 1 --2- 172.21.15.45:0/2695207121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36c0102740 0x7f36c019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:35:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.399+0000 7f36c490d700 1 --2- 172.21.15.45:0/2695207121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c0107840 0x7f36c0198a80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f36b800b970 tx=0x7f36b800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.399+0000 7f36bd7fa700 1 -- 172.21.15.45:0/2695207121 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8019070 con 0x7f36c0107840 2024-06-06T04:35:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.399+0000 7f36c6370700 1 -- 172.21.15.45:0/2695207121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c01994d0 con 0x7f36c0107840 2024-06-06T04:35:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.400+0000 7f36bd7fa700 1 -- 172.21.15.45:0/2695207121 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36b8014440 con 0x7f36c0107840 2024-06-06T04:35:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.400+0000 7f36c6370700 1 -- 172.21.15.45:0/2695207121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c00746c0 con 0x7f36c0107840 2024-06-06T04:35:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.400+0000 7f36bd7fa700 1 -- 172.21.15.45:0/2695207121 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b800ce60 con 0x7f36c0107840 2024-06-06T04:35:50.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.401+0000 7f36c6370700 1 -- 172.21.15.45:0/2695207121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c0069180 con 0x7f36c0107840 2024-06-06T04:35:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.403+0000 7f36bd7fa700 1 -- 172.21.15.45:0/2695207121 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f36b801c070 con 0x7f36c0107840 2024-06-06T04:35:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.403+0000 7f36bd7fa700 1 --2- 172.21.15.45:0/2695207121 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f36a80a2ca0 0x7f36a80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.403+0000 7f36bd7fa700 1 -- 172.21.15.45:0/2695207121 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8912+0+0 (secure 0 0 0) 0x7f36b80ce6d0 con 0x7f36c0107840 2024-06-06T04:35:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.403+0000 7f36bffff700 1 --2- 172.21.15.45:0/2695207121 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f36a80a2ca0 0x7f36a80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:50.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.404+0000 7f36bffff700 1 --2- 172.21.15.45:0/2695207121 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f36a80a2ca0 0x7f36a80a5160 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f36c0199bc0 tx=0x7f36b0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:50.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.408+0000 7f36bd7fa700 1 -- 172.21.15.45:0/2695207121 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f36b809b390 con 0x7f36c0107840 2024-06-06T04:35:50.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:50.619+0000 7f36c6370700 1 -- 172.21.15.45:0/2695207121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f36c002dca0 con 0x7f36c0107840 2024-06-06T04:35:51.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:51.585+0000 7f36bd7fa700 1 -- 172.21.15.45:0/2695207121 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7f36b809ac90 con 0x7f36c0107840 2024-06-06T04:35:51.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:51.587+0000 7f36c6370700 1 -- 172.21.15.45:0/2695207121 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f36a80a2ca0 msgr2=0x7f36a80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:51.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:51.587+0000 7f36c6370700 1 --2- 172.21.15.45:0/2695207121 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f36a80a2ca0 0x7f36a80a5160 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f36c0199bc0 tx=0x7f36b0008040 comp rx=0 tx=0).stop 2024-06-06T04:35:51.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:51.587+0000 7f36c6370700 1 -- 172.21.15.45:0/2695207121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c0107840 msgr2=0x7f36c0198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:51.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:51.587+0000 7f36c6370700 1 --2- 172.21.15.45:0/2695207121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c0107840 0x7f36c0198a80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f36b800b970 tx=0x7f36b800bd30 comp rx=0 tx=0).stop 2024-06-06T04:35:51.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:51.587+0000 7f36c6370700 1 -- 172.21.15.45:0/2695207121 shutdown_connections 2024-06-06T04:35:51.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:51.587+0000 7f36c6370700 1 --2- 172.21.15.45:0/2695207121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c0107840 0x7f36c0198a80 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:51.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:51.587+0000 7f36c6370700 1 --2- 172.21.15.45:0/2695207121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36c0102740 0x7f36c019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:51.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:51.587+0000 7f36c6370700 1 --2- 172.21.15.45:0/2695207121 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f36a80a2ca0 0x7f36a80a5160 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:51.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:51.587+0000 7f36c6370700 1 --2- 172.21.15.45:0/2695207121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36c01030f0 0x7f36c019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:51.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:51.587+0000 7f36c6370700 1 -- 172.21.15.45:0/2695207121 >> 172.21.15.45:0/2695207121 conn(0x7f36c00795f0 msgr2=0x7f36c0100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:51.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:51.588+0000 7f36c6370700 1 -- 172.21.15.45:0/2695207121 shutdown_connections 2024-06-06T04:35:51.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:51.588+0000 7f36c6370700 1 -- 172.21.15.45:0/2695207121 wait complete. 2024-06-06T04:35:51.592 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-06-06T04:35:51.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:51 smithi086 ceph-mon[36053]: pgmap v565: 97 pgs: 97 active+clean; 26 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:35:51.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:51 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2695207121' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T04:35:51.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:51 smithi154 ceph-mon[36428]: pgmap v565: 97 pgs: 97 active+clean; 26 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:35:51.931 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:51 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2695207121' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T04:35:51.975 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:51 smithi045 ceph-mon[32299]: pgmap v565: 97 pgs: 97 active+clean; 26 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:35:51.975 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:51 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2695207121' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T04:35:52.417 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-06-06T04:35:52.673 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T04:35:52.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:52 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/2695207121' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T04:35:52.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:52 smithi045 ceph-mon[32299]: osdmap e103: 12 total, 12 up, 12 in 2024-06-06T04:35:52.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:52 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/2695207121' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T04:35:52.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:52 smithi086 ceph-mon[36053]: osdmap e103: 12 total, 12 up, 12 in 2024-06-06T04:35:52.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:52 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/2695207121' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T04:35:52.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:52 smithi154 ceph-mon[36428]: osdmap e103: 12 total, 12 up, 12 in 2024-06-06T04:35:53.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:53 smithi086 ceph-mon[36053]: pgmap v567: 33 pgs: 33 active+clean; 26 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:35:53.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:53 smithi045 ceph-mon[32299]: pgmap v567: 33 pgs: 33 active+clean; 26 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:35:53.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:53 smithi154 ceph-mon[36428]: pgmap v567: 33 pgs: 33 active+clean; 26 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:35:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.487+0000 7f2ac1e11700 1 -- 172.21.15.45:0/237121153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2abc101670 msgr2=0x7f2abc101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.487+0000 7f2ac1e11700 1 --2- 172.21.15.45:0/237121153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2abc101670 0x7f2abc101af0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009990 tx=0x7f2ab0009ca0 comp rx=0 tx=0).stop 2024-06-06T04:35:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.487+0000 7f2ac1e11700 1 -- 172.21.15.45:0/237121153 shutdown_connections 2024-06-06T04:35:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.487+0000 7f2ac1e11700 1 --2- 172.21.15.45:0/237121153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2abc102030 0x7f2abc10e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.487+0000 7f2ac1e11700 1 --2- 172.21.15.45:0/237121153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2abc1074e0 0x7f2abc1078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.487+0000 7f2ac1e11700 1 --2- 172.21.15.45:0/237121153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2abc101670 0x7f2abc101af0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.487+0000 7f2ac1e11700 1 -- 172.21.15.45:0/237121153 >> 172.21.15.45:0/237121153 conn(0x7f2abc078280 msgr2=0x7f2abc078690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.487+0000 7f2ac1e11700 1 -- 172.21.15.45:0/237121153 shutdown_connections 2024-06-06T04:35:54.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.487+0000 7f2ac1e11700 1 -- 172.21.15.45:0/237121153 wait complete. 2024-06-06T04:35:54.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.488+0000 7f2ac1e11700 1 Processor -- start 2024-06-06T04:35:54.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.488+0000 7f2ac1e11700 1 -- start start 2024-06-06T04:35:54.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2ac1e11700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2abc101670 0x7f2abc1a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:54.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2ac1e11700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2abc102030 0x7f2abc1a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:54.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2ac1e11700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2abc1074e0 0x7f2abc19f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:54.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2ac1e11700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc1115b0 con 0x7f2abc102030 2024-06-06T04:35:54.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2ac1e11700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc111430 con 0x7f2abc101670 2024-06-06T04:35:54.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2ac1e11700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc111730 con 0x7f2abc1074e0 2024-06-06T04:35:54.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2abb7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2abc101670 0x7f2abc1a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:54.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2abaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2abc102030 0x7f2abc1a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:54.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2abb7fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2abc101670 0x7f2abc1a3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.45:53504/0 (socket says 172.21.15.45:53504) 2024-06-06T04:35:54.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2abb7fe700 1 -- 172.21.15.45:0/931570167 learned_addr learned my addr 172.21.15.45:0/931570167 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-06-06T04:35:54.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2abbfff700 1 --2- 172.21.15.45:0/931570167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2abc1074e0 0x7f2abc19f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:54.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2abb7fe700 1 -- 172.21.15.45:0/931570167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2abc1074e0 msgr2=0x7f2abc19f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:54.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2abb7fe700 1 --2- 172.21.15.45:0/931570167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2abc1074e0 0x7f2abc19f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:54.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2abb7fe700 1 -- 172.21.15.45:0/931570167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2abc102030 msgr2=0x7f2abc1a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:54.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2abb7fe700 1 --2- 172.21.15.45:0/931570167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2abc102030 0x7f2abc1a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:54.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2abb7fe700 1 -- 172.21.15.45:0/931570167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2abc19fa20 con 0x7f2abc101670 2024-06-06T04:35:54.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2abaffd700 1 --2- 172.21.15.45:0/931570167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2abc102030 0x7f2abc1a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T04:35:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2abbfff700 1 --2- 172.21.15.45:0/931570167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2abc1074e0 0x7f2abc19f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T04:35:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.489+0000 7f2abb7fe700 1 --2- 172.21.15.45:0/931570167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2abc101670 0x7f2abc1a3d70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00c920 tx=0x7f2aac00cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.490+0000 7f2ab8ff9700 1 -- 172.21.15.45:0/931570167 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac007b00 con 0x7f2abc101670 2024-06-06T04:35:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.490+0000 7f2ac1e11700 1 -- 172.21.15.45:0/931570167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2abc19fd10 con 0x7f2abc101670 2024-06-06T04:35:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.490+0000 7f2ab8ff9700 1 -- 172.21.15.45:0/931570167 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2aac004ce0 con 0x7f2abc101670 2024-06-06T04:35:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.490+0000 7f2ab8ff9700 1 -- 172.21.15.45:0/931570167 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac005550 con 0x7f2abc101670 2024-06-06T04:35:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.490+0000 7f2ac1e11700 1 -- 172.21.15.45:0/931570167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2abc1abca0 con 0x7f2abc101670 2024-06-06T04:35:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.491+0000 7f2ac1e11700 1 -- 172.21.15.45:0/931570167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2abc102b00 con 0x7f2abc101670 2024-06-06T04:35:54.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.492+0000 7f2ab8ff9700 1 -- 172.21.15.45:0/931570167 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2aac004750 con 0x7f2abc101670 2024-06-06T04:35:54.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.492+0000 7f2ab8ff9700 1 --2- 172.21.15.45:0/931570167 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2aa40a2c20 0x7f2aa40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T04:35:54.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.494+0000 7f2abaffd700 1 --2- 172.21.15.45:0/931570167 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2aa40a2c20 0x7f2aa40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T04:35:54.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.494+0000 7f2ab8ff9700 1 -- 172.21.15.45:0/931570167 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8324+0+0 (secure 0 0 0) 0x7f2aac0cf140 con 0x7f2abc101670 2024-06-06T04:35:54.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.494+0000 7f2abaffd700 1 --2- 172.21.15.45:0/931570167 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2aa40a2c20 0x7f2aa40a50e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000b2e0 tx=0x7f2ab001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T04:35:54.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.497+0000 7f2ab8ff9700 1 -- 172.21.15.45:0/931570167 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2aac014030 con 0x7f2abc101670 2024-06-06T04:35:54.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:54.705+0000 7f2ac1e11700 1 -- 172.21.15.45:0/931570167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2abc1abf30 con 0x7f2abc101670 2024-06-06T04:35:55.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:55.608+0000 7f2ab8ff9700 1 -- 172.21.15.45:0/931570167 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v104) ==== 178+0+0 (secure 0 0 0) 0x7f2aac09bfe0 con 0x7f2abc101670 2024-06-06T04:35:55.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:55.612+0000 7f2ac1e11700 1 -- 172.21.15.45:0/931570167 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2aa40a2c20 msgr2=0x7f2aa40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:55.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:55.612+0000 7f2ac1e11700 1 --2- 172.21.15.45:0/931570167 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2aa40a2c20 0x7f2aa40a50e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000b2e0 tx=0x7f2ab001a040 comp rx=0 tx=0).stop 2024-06-06T04:35:55.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:55.612+0000 7f2ac1e11700 1 -- 172.21.15.45:0/931570167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2abc101670 msgr2=0x7f2abc1a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T04:35:55.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:55.612+0000 7f2ac1e11700 1 --2- 172.21.15.45:0/931570167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2abc101670 0x7f2abc1a3d70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00c920 tx=0x7f2aac00cc30 comp rx=0 tx=0).stop 2024-06-06T04:35:55.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:55.612+0000 7f2ac1e11700 1 -- 172.21.15.45:0/931570167 shutdown_connections 2024-06-06T04:35:55.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:55.612+0000 7f2ac1e11700 1 --2- 172.21.15.45:0/931570167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2abc1074e0 0x7f2abc19f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:55.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:55.612+0000 7f2ac1e11700 1 --2- 172.21.15.45:0/931570167 >> [v2:172.21.15.45:6800/1718508550,v1:172.21.15.45:6801/1718508550] conn(0x7f2aa40a2c20 0x7f2aa40a50e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:55.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:55.612+0000 7f2ac1e11700 1 --2- 172.21.15.45:0/931570167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2abc101670 0x7f2abc1a3d70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:55.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:55.612+0000 7f2ac1e11700 1 --2- 172.21.15.45:0/931570167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2abc102030 0x7f2abc1a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T04:35:55.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:55.612+0000 7f2ac1e11700 1 -- 172.21.15.45:0/931570167 >> 172.21.15.45:0/931570167 conn(0x7f2abc078280 msgr2=0x7f2abc06ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T04:35:55.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:55.612+0000 7f2ac1e11700 1 -- 172.21.15.45:0/931570167 shutdown_connections 2024-06-06T04:35:55.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-06-06T04:35:55.612+0000 7f2ac1e11700 1 -- 172.21.15.45:0/931570167 wait complete. 2024-06-06T04:35:55.615 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data_ec' removed 2024-06-06T04:35:55.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:55 smithi086 ceph-mon[36053]: pgmap v568: 33 pgs: 33 active+clean; 26 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:35:55.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:55 smithi086 ceph-mon[36053]: from='client.? 172.21.15.45:0/931570167' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T04:35:55.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:55 smithi086 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T04:35:55.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:55 smithi154 ceph-mon[36428]: pgmap v568: 33 pgs: 33 active+clean; 26 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:35:55.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:55 smithi154 ceph-mon[36428]: from='client.? 172.21.15.45:0/931570167' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T04:35:55.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:55 smithi154 ceph-mon[36428]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T04:35:55.946 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:55 smithi045 ceph-mon[32299]: pgmap v568: 33 pgs: 33 active+clean; 26 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T04:35:55.946 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:55 smithi045 ceph-mon[32299]: from='client.? 172.21.15.45:0/931570167' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T04:35:55.946 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:55 smithi045 ceph-mon[32299]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-06T04:35:56.364 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T04:35:56.395 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T04:35:56.429 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T04:35:56.464 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-06T04:35:56.464 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-06T04:35:56.481 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-06T04:35:56.499 DEBUG:teuthology.orchestra.run.smithi154:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-06T04:35:56.521 INFO:tasks.cephadm:Stopping all daemons... 2024-06-06T04:35:56.522 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-06T04:35:56.522 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mon.a 2024-06-06T04:35:56.606 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:56 smithi045 systemd[1]: Stopping Ceph mon.a for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:35:56.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:56 smithi086 ceph-mon[36053]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T04:35:56.859 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:56 smithi086 ceph-mon[36053]: osdmap e104: 12 total, 12 up, 12 in 2024-06-06T04:35:56.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:56 smithi154 ceph-mon[36428]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T04:35:56.930 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:56 smithi154 ceph-mon[36428]: osdmap e104: 12 total, 12 up, 12 in 2024-06-06T04:35:56.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:56 smithi045 ceph-mon[32299]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-06T04:35:56.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:56 smithi045 ceph-mon[32299]: osdmap e104: 12 total, 12 up, 12 in 2024-06-06T04:35:56.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:56 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-a[32295]: 2024-06-06T04:35:56.826+0000 7fc9b3e6d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:35:56.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:56 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-a[32295]: 2024-06-06T04:35:56.826+0000 7fc9b3e6d700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-06T04:35:56.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:56 smithi045 podman[107347]: 2024-06-06 04:35:56.848678145 +0000 UTC m=+0.121514659 container died 3a75fd6d5b466f236b111a97443e544e9fcd4ab5f8ed62d756fe9640754c0f9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-06T04:35:57.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:57 smithi045 podman[107347]: 2024-06-06 04:35:57.29438461 +0000 UTC m=+0.567221123 container cleanup 3a75fd6d5b466f236b111a97443e544e9fcd4ab5f8ed62d756fe9640754c0f9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-06T04:35:57.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:57 smithi045 bash[107347]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-a 2024-06-06T04:35:57.728 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mon.a.service' 2024-06-06T04:35:57.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:57 smithi045 podman[107360]: 2024-06-06 04:35:57.487947458 +0000 UTC m=+0.630211436 container remove 3a75fd6d5b466f236b111a97443e544e9fcd4ab5f8ed62d756fe9640754c0f9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-06T04:35:57.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:57 smithi045 systemd[1]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-06T04:35:57.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:57 smithi045 systemd[1]: Stopped Ceph mon.a for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:35:57.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Jun 06 04:35:57 smithi045 systemd[1]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mon.a.service: Consumed 17.861s CPU time. 2024-06-06T04:35:57.785 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:35:57.785 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-06T04:35:57.786 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-06-06T04:35:57.786 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mon.b 2024-06-06T04:35:58.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:57 smithi086 systemd[1]: Stopping Ceph mon.b for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:35:58.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:58 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-b[36049]: 2024-06-06T04:35:58.111+0000 7fc719b74700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:35:58.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:58 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-b[36049]: 2024-06-06T04:35:58.111+0000 7fc719b74700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-06T04:35:58.514 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:58 smithi086 podman[69283]: 2024-06-06 04:35:58.203912226 +0000 UTC m=+0.212626332 container died b0595627ff8975d6e7b8bd62f43545bc1db08327eef53419625b8ec9e3be3cf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-06T04:35:58.816 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:58 smithi086 podman[69283]: 2024-06-06 04:35:58.614307854 +0000 UTC m=+0.623021951 container cleanup b0595627ff8975d6e7b8bd62f43545bc1db08327eef53419625b8ec9e3be3cf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T04:35:58.816 INFO:journalctl@ceph.mon.b.smithi086.stdout:Jun 06 04:35:58 smithi086 bash[69283]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-b 2024-06-06T04:35:58.965 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mon.b.service' 2024-06-06T04:35:59.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:35:59.011 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-06-06T04:35:59.011 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-06-06T04:35:59.011 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mon.c 2024-06-06T04:35:59.404 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:59 smithi154 systemd[1]: Stopping Ceph mon.c for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:35:59.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:59 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-c[36413]: 2024-06-06T04:35:59.403+0000 7fc94e953700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:35:59.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:59 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-c[36413]: 2024-06-06T04:35:59.403+0000 7fc94e953700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-06T04:35:59.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:59 smithi154 podman[70361]: 2024-06-06 04:35:59.598355082 +0000 UTC m=+0.387069449 container died 6742d6b94ee13aef48cb8aa3fa5547780cbde1f68dd6126f26a35c94111931db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc) 2024-06-06T04:36:00.180 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:59 smithi154 podman[70361]: 2024-06-06 04:35:59.996820826 +0000 UTC m=+0.785535184 container cleanup 6742d6b94ee13aef48cb8aa3fa5547780cbde1f68dd6126f26a35c94111931db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-06T04:36:00.181 INFO:journalctl@ceph.mon.c.smithi154.stdout:Jun 06 04:35:59 smithi154 bash[70361]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mon-c 2024-06-06T04:36:00.428 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mon.c.service' 2024-06-06T04:36:00.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:36:00.474 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-06-06T04:36:00.474 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-06T04:36:00.475 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mgr.x 2024-06-06T04:36:00.841 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:36:00 smithi045 systemd[1]: Stopping Ceph mgr.x for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:36:01.133 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:36:00 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:36:00.840+0000 7f129d72a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:36:01.133 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:36:00 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T04:36:00.840+0000 7f129d72a700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-06-06T04:36:01.134 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:36:00 smithi045 podman[107468]: 2024-06-06 04:36:00.872670666 +0000 UTC m=+0.194076286 container died 8bc2882f94572fb416ce4f76ecfc4b219074c07bd8188e377a0f415aae5cba6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:36:01.477 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:36:01 smithi045 podman[107468]: 2024-06-06 04:36:01.292097619 +0000 UTC m=+0.613503268 container cleanup 8bc2882f94572fb416ce4f76ecfc4b219074c07bd8188e377a0f415aae5cba6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:36:01.477 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:36:01 smithi045 bash[107468]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x 2024-06-06T04:36:01.732 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mgr.x.service' 2024-06-06T04:36:01.755 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:36:01 smithi045 podman[107481]: 2024-06-06 04:36:01.502237287 +0000 UTC m=+0.627664438 container remove 8bc2882f94572fb416ce4f76ecfc4b219074c07bd8188e377a0f415aae5cba6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:36:01.756 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:36:01 smithi045 systemd[1]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mgr.x.service: Deactivated successfully. 2024-06-06T04:36:01.756 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:36:01 smithi045 systemd[1]: Stopped Ceph mgr.x for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:36:01.756 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Jun 06 04:36:01 smithi045 systemd[1]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mgr.x.service: Consumed 1min 8.480s CPU time. 2024-06-06T04:36:01.769 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:36:01.769 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-06T04:36:01.769 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-06T04:36:01.770 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mgr.y 2024-06-06T04:36:02.107 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:36:01 smithi086 systemd[1]: Stopping Ceph mgr.y for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:36:02.359 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:36:02 smithi086 podman[69408]: 2024-06-06 04:36:02.107364701 +0000 UTC m=+0.138963410 container died 98975c069f1fe69928a1f2b9933f41118e84b541f957ff09ee0934250198e586 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-06T04:36:02.755 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:36:02 smithi086 podman[69408]: 2024-06-06 04:36:02.488074706 +0000 UTC m=+0.519673406 container cleanup 98975c069f1fe69928a1f2b9933f41118e84b541f957ff09ee0934250198e586 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:36:02.756 INFO:journalctl@ceph.mgr.y.smithi086.stdout:Jun 06 04:36:02 smithi086 bash[69408]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-y 2024-06-06T04:36:02.917 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mgr.y.service' 2024-06-06T04:36:02.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:36:02.952 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-06T04:36:02.952 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-06-06T04:36:02.952 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mgr.z 2024-06-06T04:36:03.297 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:36:03 smithi154 systemd[1]: Stopping Ceph mgr.z for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:36:03.602 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:36:03 smithi154 podman[70486]: 2024-06-06 04:36:03.297622619 +0000 UTC m=+0.145835488 container died c34767ed5a646046d08c5709b65c1c7d5d4da759a27fadbe3e625b04625b3781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-06T04:36:03.930 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:36:03 smithi154 podman[70486]: 2024-06-06 04:36:03.702282658 +0000 UTC m=+0.550495518 container cleanup c34767ed5a646046d08c5709b65c1c7d5d4da759a27fadbe3e625b04625b3781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-06T04:36:03.930 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Jun 06 04:36:03 smithi154 bash[70486]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mgr-z 2024-06-06T04:36:04.124 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mgr.z.service' 2024-06-06T04:36:04.158 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:36:04.159 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-06-06T04:36:04.159 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-06T04:36:04.159 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.0 2024-06-06T04:36:04.472 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:36:04 smithi045 systemd[1]: Stopping Ceph osd.0 for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:36:04.726 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:36:04 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-0[47449]: 2024-06-06T04:36:04.472+0000 7f45edb7c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:36:04.727 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:36:04 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-0[47449]: 2024-06-06T04:36:04.472+0000 7f45edb7c700 -1 osd.0 104 *** Got signal Terminated *** 2024-06-06T04:36:04.727 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:36:04 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-0[47449]: 2024-06-06T04:36:04.472+0000 7f45edb7c700 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T04:36:10.132 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:36:09 smithi045 podman[107590]: 2024-06-06 04:36:09.849394884 +0000 UTC m=+5.484787342 container died 073ab2c9831f86af86e7dfd576ad4e4125a2ecaf64054a457c1fb268909c9bc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-06T04:36:10.477 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:36:10 smithi045 podman[107590]: 2024-06-06 04:36:10.306083678 +0000 UTC m=+5.941476131 container cleanup 073ab2c9831f86af86e7dfd576ad4e4125a2ecaf64054a457c1fb268909c9bc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-0, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-06T04:36:10.477 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:36:10 smithi045 bash[107590]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-0 2024-06-06T04:36:10.945 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:36:10 smithi045 podman[107608]: 2024-06-06 04:36:10.599685456 +0000 UTC m=+0.749291016 container remove 073ab2c9831f86af86e7dfd576ad4e4125a2ecaf64054a457c1fb268909c9bc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-0, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T04:36:11.227 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:36:10 smithi045 podman[107698]: 2024-06-06 04:36:10.849056146 +0000 UTC m=+0.017843103 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:36:11.227 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:36:11 smithi045 podman[107698]: 2024-06-06 04:36:11.050741699 +0000 UTC m=+0.219528644 container create 6a97dfada474364a78bae0c769ca543990d1849f1723fe39374171c4e0acc3b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=) 2024-06-06T04:36:11.565 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:36:11 smithi045 podman[107698]: 2024-06-06 04:36:11.245537587 +0000 UTC m=+0.414324547 container init 6a97dfada474364a78bae0c769ca543990d1849f1723fe39374171c4e0acc3b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-06T04:36:11.565 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:36:11 smithi045 podman[107698]: 2024-06-06 04:36:11.249598458 +0000 UTC m=+0.418385410 container start 6a97dfada474364a78bae0c769ca543990d1849f1723fe39374171c4e0acc3b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-0-deactivate, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-06T04:36:11.566 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:36:11 smithi045 podman[107698]: 2024-06-06 04:36:11.353563469 +0000 UTC m=+0.522350458 container attach 6a97dfada474364a78bae0c769ca543990d1849f1723fe39374171c4e0acc3b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:36:11.923 INFO:journalctl@ceph.osd.0.smithi045.stdout:Jun 06 04:36:11 smithi045 podman[107698]: 2024-06-06 04:36:11.56576424 +0000 UTC m=+0.734551198 container died 6a97dfada474364a78bae0c769ca543990d1849f1723fe39374171c4e0acc3b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-06-06T04:36:12.336 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-06T04:36:12.382 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:36:12.382 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-06T04:36:12.382 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-06T04:36:12.382 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.1 2024-06-06T04:36:12.692 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:36:12 smithi086 systemd[1]: Stopping Ceph osd.1 for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:36:13.108 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:36:12 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-1[40608]: 2024-06-06T04:36:12.691+0000 7ff41a99d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:36:13.109 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:36:12 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-1[40608]: 2024-06-06T04:36:12.691+0000 7ff41a99d700 -1 osd.1 104 *** Got signal Terminated *** 2024-06-06T04:36:13.109 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:36:12 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-1[40608]: 2024-06-06T04:36:12.691+0000 7ff41a99d700 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T04:36:18.537 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:36:18 smithi086 podman[69537]: 2024-06-06 04:36:18.22670544 +0000 UTC m=+5.651788833 container died d341ee4b1d678047ae396c40d5b34947679ddd1a964a6df20ec0a03db7e0aba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-1, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-06T04:36:18.859 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:36:18 smithi086 podman[69537]: 2024-06-06 04:36:18.636388871 +0000 UTC m=+6.061472263 container cleanup d341ee4b1d678047ae396c40d5b34947679ddd1a964a6df20ec0a03db7e0aba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T04:36:18.859 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:36:18 smithi086 bash[69537]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-1 2024-06-06T04:36:19.316 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:36:18 smithi086 podman[69554]: 2024-06-06 04:36:18.871284975 +0000 UTC m=+0.646621478 container remove d341ee4b1d678047ae396c40d5b34947679ddd1a964a6df20ec0a03db7e0aba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-1, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-06T04:36:19.609 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:36:19 smithi086 podman[69642]: 2024-06-06 04:36:19.219233456 +0000 UTC m=+0.017464909 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:36:19.609 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:36:19 smithi086 podman[69642]: 2024-06-06 04:36:19.347134185 +0000 UTC m=+0.145365625 container create 39b59102c0621587e0e64c0db2fab440e37b9bf8ea8ec96299a6b6dc161c7f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-06T04:36:19.912 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:36:19 smithi086 podman[69642]: 2024-06-06 04:36:19.616946289 +0000 UTC m=+0.415177730 container init 39b59102c0621587e0e64c0db2fab440e37b9bf8ea8ec96299a6b6dc161c7f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:36:19.912 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:36:19 smithi086 podman[69642]: 2024-06-06 04:36:19.620665023 +0000 UTC m=+0.418896457 container start 39b59102c0621587e0e64c0db2fab440e37b9bf8ea8ec96299a6b6dc161c7f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True) 2024-06-06T04:36:19.912 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:36:19 smithi086 podman[69642]: 2024-06-06 04:36:19.72652249 +0000 UTC m=+0.524753928 container attach 39b59102c0621587e0e64c0db2fab440e37b9bf8ea8ec96299a6b6dc161c7f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD) 2024-06-06T04:36:20.195 INFO:journalctl@ceph.osd.1.smithi086.stdout:Jun 06 04:36:19 smithi086 podman[69642]: 2024-06-06 04:36:19.913048254 +0000 UTC m=+0.711279692 container died 39b59102c0621587e0e64c0db2fab440e37b9bf8ea8ec96299a6b6dc161c7f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-06-06T04:36:20.521 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-06T04:36:20.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:36:20.563 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-06T04:36:20.563 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-06T04:36:20.563 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.2 2024-06-06T04:36:20.882 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:36:20 smithi154 systemd[1]: Stopping Ceph osd.2 for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:36:21.180 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:36:20 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-2[41536]: 2024-06-06T04:36:20.881+0000 7fac1ff80700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:36:21.180 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:36:20 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-2[41536]: 2024-06-06T04:36:20.881+0000 7fac1ff80700 -1 osd.2 104 *** Got signal Terminated *** 2024-06-06T04:36:21.181 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:36:20 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-2[41536]: 2024-06-06T04:36:20.881+0000 7fac1ff80700 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T04:36:26.633 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:36:26 smithi154 podman[70623]: 2024-06-06 04:36:26.332546024 +0000 UTC m=+5.570213199 container died 48ae4757700972c7f64bfdb0a2cddd56446b808d3920a752e166908018071c65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-2, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-06T04:36:26.930 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:36:26 smithi154 podman[70623]: 2024-06-06 04:36:26.723088086 +0000 UTC m=+5.960755259 container cleanup 48ae4757700972c7f64bfdb0a2cddd56446b808d3920a752e166908018071c65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:36:26.930 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:36:26 smithi154 bash[70623]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-2 2024-06-06T04:36:27.337 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:36:26 smithi154 podman[70639]: 2024-06-06 04:36:26.982757176 +0000 UTC m=+0.648682839 container remove 48ae4757700972c7f64bfdb0a2cddd56446b808d3920a752e166908018071c65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc) 2024-06-06T04:36:27.628 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:36:27 smithi154 podman[70727]: 2024-06-06 04:36:27.241149925 +0000 UTC m=+0.017327333 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:36:27.628 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:36:27 smithi154 podman[70727]: 2024-06-06 04:36:27.367173676 +0000 UTC m=+0.143351068 container create 654031aee37618e535167138836129a67881a0a0856b4bf60aba4a6c92237f69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-06T04:36:27.930 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:36:27 smithi154 podman[70727]: 2024-06-06 04:36:27.628279861 +0000 UTC m=+0.404457255 container init 654031aee37618e535167138836129a67881a0a0856b4bf60aba4a6c92237f69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-06T04:36:27.930 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:36:27 smithi154 podman[70727]: 2024-06-06 04:36:27.631590008 +0000 UTC m=+0.407767408 container start 654031aee37618e535167138836129a67881a0a0856b4bf60aba4a6c92237f69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-06T04:36:27.930 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:36:27 smithi154 podman[70727]: 2024-06-06 04:36:27.745085887 +0000 UTC m=+0.521263291 container attach 654031aee37618e535167138836129a67881a0a0856b4bf60aba4a6c92237f69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-06T04:36:28.180 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:36:27 smithi154 podman[70727]: 2024-06-06 04:36:27.930193261 +0000 UTC m=+0.706370666 container died 654031aee37618e535167138836129a67881a0a0856b4bf60aba4a6c92237f69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-06T04:36:28.453 INFO:journalctl@ceph.osd.2.smithi154.stdout:Jun 06 04:36:28 smithi154 podman[70753]: 2024-06-06 04:36:28.440672085 +0000 UTC m=+0.500089099 container remove 654031aee37618e535167138836129a67881a0a0856b4bf60aba4a6c92237f69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-2-deactivate, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-06-06T04:36:28.459 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-06T04:36:28.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:36:28.501 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-06T04:36:28.501 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-06T04:36:28.502 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.3 2024-06-06T04:36:28.827 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:36:28 smithi045 systemd[1]: Stopping Ceph osd.3 for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:36:29.115 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:36:28 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-3[54073]: 2024-06-06T04:36:28.826+0000 7f7ffdfe2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:36:29.115 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:36:28 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-3[54073]: 2024-06-06T04:36:28.826+0000 7f7ffdfe2700 -1 osd.3 104 *** Got signal Terminated *** 2024-06-06T04:36:29.115 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:36:28 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-3[54073]: 2024-06-06T04:36:28.826+0000 7f7ffdfe2700 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T04:36:31.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:30 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:30.659+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:31.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:31 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:31.284+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:32.044 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:31 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:31.661+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:32.431 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:32 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:32.070+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:32.726 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:32 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:32.309+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:32.978 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:32 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:32.613+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:33.227 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:36:32 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-3[54073]: 2024-06-06T04:36:32.956+0000 7f7ff65ce700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.112744+0000 front 2024-06-06T04:36:08.112769+0000 (oldest deadline 2024-06-06T04:36:32.813028+0000) 2024-06-06T04:36:33.430 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:33 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:33.092+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:33.726 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:33 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:33.350+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:34.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:33 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:33.645+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:34.430 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:34 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:34.057+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:34.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:34 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:34.149+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:34.477 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:34 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:34.369+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:34.477 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:36:34 smithi045 podman[107806]: 2024-06-06 04:36:34.197460775 +0000 UTC m=+5.490896623 container died a5661291dc161f2dae61ba09b2397a53a3ac8191b234199f50ff708be3772672 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-06T04:36:34.977 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:36:34 smithi045 podman[107806]: 2024-06-06 04:36:34.6028679 +0000 UTC m=+5.896303734 container cleanup a5661291dc161f2dae61ba09b2397a53a3ac8191b234199f50ff708be3772672 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-06T04:36:34.977 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:36:34 smithi045 bash[107806]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-3 2024-06-06T04:36:34.977 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:36:34 smithi045 podman[107835]: 2024-06-06 04:36:34.796178288 +0000 UTC m=+0.597470630 container remove a5661291dc161f2dae61ba09b2397a53a3ac8191b234199f50ff708be3772672 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-3, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T04:36:35.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:34 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:34.670+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:35.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:35 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:35.197+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:35.431 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:35 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:35.037+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:35.477 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:35 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:35.377+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:35.477 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:36:35 smithi045 podman[107925]: 2024-06-06 04:36:35.125592059 +0000 UTC m=+0.018002493 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:36:35.477 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:36:35 smithi045 podman[107925]: 2024-06-06 04:36:35.280511277 +0000 UTC m=+0.172921745 container create f087e6c7c24b96939d61e0a450459622781c38cf7e4bbd39fd10bef15d327c24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-06T04:36:35.861 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:36:35 smithi045 podman[107925]: 2024-06-06 04:36:35.550563365 +0000 UTC m=+0.442973792 container init f087e6c7c24b96939d61e0a450459622781c38cf7e4bbd39fd10bef15d327c24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T04:36:35.862 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:36:35 smithi045 podman[107925]: 2024-06-06 04:36:35.554090794 +0000 UTC m=+0.446501231 container start f087e6c7c24b96939d61e0a450459622781c38cf7e4bbd39fd10bef15d327c24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-06T04:36:35.862 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:36:35 smithi045 podman[107925]: 2024-06-06 04:36:35.633570023 +0000 UTC m=+0.525980457 container attach f087e6c7c24b96939d61e0a450459622781c38cf7e4bbd39fd10bef15d327c24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-3-deactivate, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-06T04:36:36.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:35 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:35.644+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:36.144 INFO:journalctl@ceph.osd.3.smithi045.stdout:Jun 06 04:36:35 smithi045 podman[107925]: 2024-06-06 04:36:35.862742327 +0000 UTC m=+0.755152758 container died f087e6c7c24b96939d61e0a450459622781c38cf7e4bbd39fd10bef15d327c24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T04:36:36.430 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:36 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:36.023+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:36.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:36 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:36.193+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:36.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:36 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:36.173+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:36.483 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-06T04:36:36.528 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:36:36.528 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-06T04:36:36.528 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-06T04:36:36.528 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.4 2024-06-06T04:36:36.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:36 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:36.425+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:36.857 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:36 smithi086 systemd[1]: Stopping Ceph osd.4 for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:36:36.857 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:36 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:36.688+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:37.109 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:36 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4[47207]: 2024-06-06T04:36:36.856+0000 7f6b5860b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:36:37.109 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:36 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4[47207]: 2024-06-06T04:36:36.856+0000 7f6b5860b700 -1 osd.4 104 *** Got signal Terminated *** 2024-06-06T04:36:37.109 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:36 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4[47207]: 2024-06-06T04:36:36.856+0000 7f6b5860b700 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T04:36:37.430 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:36 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:36.992+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:37.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:37 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:37.164+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:37.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:37 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:37.158+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:37.726 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:37 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:37.464+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:38.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:37 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:37.676+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:38.430 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:37 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:37.972+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:38.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:38 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:38.172+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:38.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:38 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:38.115+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:38.976 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:38 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:38.506+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:39.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:38 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:38.720+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:39.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:38 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:38.720+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:39.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:39 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:39.161+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:39.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:39 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:39.144+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:39.431 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:38 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:38.937+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:39.726 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:39 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:39.460+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:40.109 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:39 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4[47207]: 2024-06-06T04:36:39.665+0000 7f6b54bff700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.108136+0000 front 2024-06-06T04:36:13.108183+0000 (oldest deadline 2024-06-06T04:36:39.008294+0000) 2024-06-06T04:36:40.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:39 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:39.700+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:40.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:39 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:39.700+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:40.141 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:39 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:39.899+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:40.141 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:39 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:39.899+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.506916+0000 front 2024-06-06T04:36:16.507033+0000 (oldest deadline 2024-06-06T04:36:39.406946+0000) 2024-06-06T04:36:40.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:40 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:40.140+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:40.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:40 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:40.151+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:40.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:40 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:40.151+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:40.477 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:40 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:40.324+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:40.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:40 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:40.494+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:41.109 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:40 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4[47207]: 2024-06-06T04:36:40.636+0000 7f6b54bff700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.108136+0000 front 2024-06-06T04:36:13.108183+0000 (oldest deadline 2024-06-06T04:36:39.008294+0000) 2024-06-06T04:36:41.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:40 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:40.739+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:41.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:40 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:40.739+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:41.332 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:41 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:41.132+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:41.332 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:40 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:40.937+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:41.332 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:40 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:40.937+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.506916+0000 front 2024-06-06T04:36:16.507033+0000 (oldest deadline 2024-06-06T04:36:39.406946+0000) 2024-06-06T04:36:41.333 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:41 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:41.165+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:41.333 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:41 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:41.165+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:41.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:41.296+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:41.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:41.490+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:41.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:41 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:41.490+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:41.859 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:41 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4[47207]: 2024-06-06T04:36:41.595+0000 7f6b54bff700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.108136+0000 front 2024-06-06T04:36:13.108183+0000 (oldest deadline 2024-06-06T04:36:39.008294+0000) 2024-06-06T04:36:41.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:41 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:41.761+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:41.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:41 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:41.761+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:42.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:42 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:42.087+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:42.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:42 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:42.123+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:42.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:42 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:42.123+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:42.432 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:41 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:41.951+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:42.432 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:41 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:41.951+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.506916+0000 front 2024-06-06T04:36:16.507033+0000 (oldest deadline 2024-06-06T04:36:39.406946+0000) 2024-06-06T04:36:42.540 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:42 smithi086 podman[69762]: 2024-06-06 04:36:42.243537998 +0000 UTC m=+5.510742379 container stop ef5f3da54c6c868f8a3adb70ff8f1c2c1dca734d75066ba0e19ede2abf58f4c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-06T04:36:42.541 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:42 smithi086 podman[69762]: 2024-06-06 04:36:42.24591971 +0000 UTC m=+5.513124102 container died ef5f3da54c6c868f8a3adb70ff8f1c2c1dca734d75066ba0e19ede2abf58f4c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-06-06T04:36:42.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:42 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:42.329+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:42.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:42 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:42.470+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:42.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:42 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:42.470+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:42.805 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:42 smithi086 podman[69762]: 2024-06-06 04:36:42.640905161 +0000 UTC m=+5.908109543 container cleanup ef5f3da54c6c868f8a3adb70ff8f1c2c1dca734d75066ba0e19ede2abf58f4c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-06-06T04:36:42.805 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:42 smithi086 bash[69762]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4 2024-06-06T04:36:43.109 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:42 smithi086 podman[69780]: 2024-06-06 04:36:42.917088009 +0000 UTC m=+0.658333899 container remove ef5f3da54c6c868f8a3adb70ff8f1c2c1dca734d75066ba0e19ede2abf58f4c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-06T04:36:43.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:42 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:42.805+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:43.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:42 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:42.805+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:43.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:43 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:43.052+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:43.431 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:42 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:42.933+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:43.431 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:42 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:42.933+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.506916+0000 front 2024-06-06T04:36:16.507033+0000 (oldest deadline 2024-06-06T04:36:39.406946+0000) 2024-06-06T04:36:43.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:43 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:43.153+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:43.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:43 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:43.153+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:43.519 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:43 smithi086 podman[69868]: 2024-06-06 04:36:43.171789705 +0000 UTC m=+0.016786676 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:36:43.520 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:43 smithi086 podman[69868]: 2024-06-06 04:36:43.276203252 +0000 UTC m=+0.121200207 container create 85a43905997b61358ff7f2457b1041f222d8224514f5cfd4737e20b4466d6ac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4-deactivate, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:36:43.726 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:43 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:43.321+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:43.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:43 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:43.431+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:43.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:43 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:43.431+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:43.813 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:43 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:43.532+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:43.814 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:43 smithi086 podman[69868]: 2024-06-06 04:36:43.520263712 +0000 UTC m=+0.365260678 container init 85a43905997b61358ff7f2457b1041f222d8224514f5cfd4737e20b4466d6ac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-06T04:36:43.814 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:43 smithi086 podman[69868]: 2024-06-06 04:36:43.523628962 +0000 UTC m=+0.368625917 container start 85a43905997b61358ff7f2457b1041f222d8224514f5cfd4737e20b4466d6ac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2) 2024-06-06T04:36:43.814 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:43 smithi086 podman[69868]: 2024-06-06 04:36:43.620409674 +0000 UTC m=+0.465406637 container attach 85a43905997b61358ff7f2457b1041f222d8224514f5cfd4737e20b4466d6ac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-06T04:36:44.109 INFO:journalctl@ceph.osd.4.smithi086.stdout:Jun 06 04:36:43 smithi086 podman[69868]: 2024-06-06 04:36:43.814251445 +0000 UTC m=+0.659248410 container died 85a43905997b61358ff7f2457b1041f222d8224514f5cfd4737e20b4466d6ac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-06T04:36:44.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:43 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:43.826+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:44.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:43 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:43.826+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:44.180 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:43 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:43.917+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:44.180 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:43 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:43.917+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.506916+0000 front 2024-06-06T04:36:16.507033+0000 (oldest deadline 2024-06-06T04:36:39.406946+0000) 2024-06-06T04:36:44.181 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:44 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:44.018+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:44.411 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-06T04:36:44.450 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:36:44.450 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-06T04:36:44.451 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-06T04:36:44.451 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.5 2024-06-06T04:36:44.478 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:44 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:44.193+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:44.479 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:44 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:44.193+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:44.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:44 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:44.301+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:44.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:44 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:44.458+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:44.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:44 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:44.458+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:44.759 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:44 smithi154 systemd[1]: Stopping Ceph osd.5 for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:36:44.806 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:44 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:44.536+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:45.011 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:44 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:44.759+0000 7ffa97b69700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:36:45.011 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:44 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:44.759+0000 7ffa97b69700 -1 osd.5 104 *** Got signal Terminated *** 2024-06-06T04:36:45.011 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:44 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:44.759+0000 7ffa97b69700 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T04:36:45.011 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:44 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:44.932+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:45.011 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:44 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:44.932+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.506916+0000 front 2024-06-06T04:36:16.507033+0000 (oldest deadline 2024-06-06T04:36:39.406946+0000) 2024-06-06T04:36:45.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:44 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:44.806+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:45.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:44 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:44.806+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:45.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:45 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:45.010+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:45.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:45 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:45.150+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:45.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:45 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:45.150+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:45.726 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:45 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:45.306+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:45.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:45 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:45.479+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:45.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:45 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:45.479+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:45.758 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:45 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:45.491+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:46.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:45 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:45.758+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:46.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:45 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:45.758+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:46.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:46 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:46.059+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:46.431 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:45 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:45.969+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:46.431 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:45 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:45.969+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.506916+0000 front 2024-06-06T04:36:16.507033+0000 (oldest deadline 2024-06-06T04:36:39.406946+0000) 2024-06-06T04:36:46.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:46 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:46.147+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:46.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:46 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:46.147+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:46.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:46 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:46.350+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:46.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:46 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:46.479+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:46.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:46 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:46.479+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:46.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:46 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:46.734+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:46.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:46 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:46.734+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:46.859 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:46 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:46.494+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:47.180 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:47 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:47.050+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:47.181 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:46 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:46.923+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:47.181 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:46 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:46.923+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.506916+0000 front 2024-06-06T04:36:16.507033+0000 (oldest deadline 2024-06-06T04:36:39.406946+0000) 2024-06-06T04:36:47.680 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:47 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:47.183+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:47.680 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:47 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:47.183+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:47.704 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:47 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:47.445+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:47.726 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:47 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:47.320+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:47.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:47 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:47.494+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:47.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:47 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:47.494+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:47.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:47 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:47.494+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:36:48.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:47 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:47.705+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:48.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:47 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:47.705+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:48.430 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:47 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:47.946+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:48.430 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:47 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:47.946+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.506916+0000 front 2024-06-06T04:36:16.507033+0000 (oldest deadline 2024-06-06T04:36:39.406946+0000) 2024-06-06T04:36:48.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:48 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:48.018+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:48.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:48 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:48.018+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:36:48.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:48 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:48.142+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:48.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:48 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:48.142+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:48.432 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:48 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:48.142+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:36:48.705 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:48 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:48.430+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:48.726 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:48 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:48.367+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:48.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:48 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:48.475+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:48.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:48 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:48.475+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:48.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:48 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:48.475+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:36:49.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:48 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:48.705+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:49.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:48 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:48.705+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:49.180 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:48 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:48.928+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:07.905085+0000 front 2024-06-06T04:36:07.904905+0000 (oldest deadline 2024-06-06T04:36:31.404808+0000) 2024-06-06T04:36:49.181 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:48 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5[48159]: 2024-06-06T04:36:48.928+0000 7ffa90155700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.506916+0000 front 2024-06-06T04:36:16.507033+0000 (oldest deadline 2024-06-06T04:36:39.406946+0000) 2024-06-06T04:36:49.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:49 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:49.127+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:49.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:49 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:49.127+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:49.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:49 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:49.127+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:36:49.181 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:49 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:49.025+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:49.182 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:49 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:49.025+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:36:49.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:49 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:49.377+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:49.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:49 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:49.377+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:22.915632+0000 front 2024-06-06T04:36:22.915656+0000 (oldest deadline 2024-06-06T04:36:48.815814+0000) 2024-06-06T04:36:49.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:49 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:49.500+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:49.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:49 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:49.500+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:49.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:49 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:49.500+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:36:49.859 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:49 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:49.475+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:49.859 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:49 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:49.475+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:36:49.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:49 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:49.687+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:49.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:49 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:49.687+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:49.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:49 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:49.687+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:36:50.394 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:50 smithi154 podman[70845]: 2024-06-06 04:36:50.107639108 +0000 UTC m=+5.458964822 container died 2b985706de73a6eb11cc6e2b035fc4ef40fa60d14fa86ef2b5a9dee8242e35a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-06-06T04:36:50.394 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:50 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:50.074+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:50.394 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:50 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:50.074+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:36:50.395 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:50 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:50.157+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:50.395 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:50 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:50.157+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:50.395 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:50 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:50.157+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:36:50.680 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:50 smithi154 podman[70845]: 2024-06-06 04:36:50.492847019 +0000 UTC m=+5.844172731 container cleanup 2b985706de73a6eb11cc6e2b035fc4ef40fa60d14fa86ef2b5a9dee8242e35a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-06T04:36:50.680 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:50 smithi154 bash[70845]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5 2024-06-06T04:36:50.729 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:50 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:50.374+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:50.729 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:50 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:50.374+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:22.915632+0000 front 2024-06-06T04:36:22.915656+0000 (oldest deadline 2024-06-06T04:36:48.815814+0000) 2024-06-06T04:36:50.730 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:50 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:50.495+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:50.730 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:50 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:50.495+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:50.730 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:50 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:50.495+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:36:50.859 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:50 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:50.511+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:50.860 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:50 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:50.511+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:36:50.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:50 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:50.720+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:50.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:50 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:50.720+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:50.861 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:50 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:50.720+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:36:51.106 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:50 smithi154 podman[70859]: 2024-06-06 04:36:50.76909378 +0000 UTC m=+0.656272742 container remove 2b985706de73a6eb11cc6e2b035fc4ef40fa60d14fa86ef2b5a9dee8242e35a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-06T04:36:51.398 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:51 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:51.105+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:51.398 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:51 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:51.105+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:36:51.398 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:51 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:51.192+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:51.399 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:51 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:51.192+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:51.399 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:51 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:51.192+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:36:51.399 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:51 smithi154 podman[70950]: 2024-06-06 04:36:51.026602003 +0000 UTC m=+0.017173235 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:36:51.399 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:51 smithi154 podman[70950]: 2024-06-06 04:36:51.12823268 +0000 UTC m=+0.118803897 container create bd481e176423457e5019ffc1421bb0d2887620621748fa69dbcba4355f41590b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-06T04:36:51.399 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:51 smithi154 podman[70950]: 2024-06-06 04:36:51.323412594 +0000 UTC m=+0.313983827 container init bd481e176423457e5019ffc1421bb0d2887620621748fa69dbcba4355f41590b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc) 2024-06-06T04:36:51.399 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:51 smithi154 podman[70950]: 2024-06-06 04:36:51.326800574 +0000 UTC m=+0.317371798 container start bd481e176423457e5019ffc1421bb0d2887620621748fa69dbcba4355f41590b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-06T04:36:51.400 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:51 smithi154 podman[70950]: 2024-06-06 04:36:51.398264114 +0000 UTC m=+0.388835337 container attach bd481e176423457e5019ffc1421bb0d2887620621748fa69dbcba4355f41590b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux ) 2024-06-06T04:36:51.681 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:51 smithi154 podman[70950]: 2024-06-06 04:36:51.622919607 +0000 UTC m=+0.613490823 container died bd481e176423457e5019ffc1421bb0d2887620621748fa69dbcba4355f41590b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-06T04:36:51.726 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:51 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:51.415+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:51.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:51 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:51.415+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:22.915632+0000 front 2024-06-06T04:36:22.915656+0000 (oldest deadline 2024-06-06T04:36:48.815814+0000) 2024-06-06T04:36:51.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:51 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:51.538+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:51.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:51 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:51.538+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:51.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:51 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:51.538+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:36:51.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:51 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:51.737+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:51.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:51 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:51.737+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:51.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:51 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:51.737+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:36:51.859 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:51 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:51.552+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:51.860 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:51 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:51.552+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:36:52.196 INFO:journalctl@ceph.osd.5.smithi154.stdout:Jun 06 04:36:52 smithi154 podman[70977]: 2024-06-06 04:36:52.17680844 +0000 UTC m=+0.548449830 container remove bd481e176423457e5019ffc1421bb0d2887620621748fa69dbcba4355f41590b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-06T04:36:52.196 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:52 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:52.142+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:52.196 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:52 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:52.142+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:36:52.201 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-06T04:36:52.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:36:52.280 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-06T04:36:52.280 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-06T04:36:52.280 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.6 2024-06-06T04:36:52.580 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:52 smithi045 systemd[1]: Stopping Ceph osd.6 for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:36:52.580 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:52 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:52.443+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:52.580 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:52 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:52.443+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:22.915632+0000 front 2024-06-06T04:36:22.915656+0000 (oldest deadline 2024-06-06T04:36:48.815814+0000) 2024-06-06T04:36:52.680 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:52 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:52.212+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:52.680 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:52 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:52.212+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:52.681 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:52 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:52.212+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:36:52.859 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:52 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:52.546+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:52.859 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:52 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:52.546+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:36:52.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:52 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:52.762+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:52.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:52 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:52.762+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:52.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:52 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:52.762+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:36:52.976 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:52 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:52.614+0000 7fcab65f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:36:52.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:52 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:52.614+0000 7fcab65f5700 -1 osd.6 104 *** Got signal Terminated *** 2024-06-06T04:36:52.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:52 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:52.614+0000 7fcab65f5700 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T04:36:52.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:52 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:52.580+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:52.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:52 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:52.580+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:52.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:52 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:52.580+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:36:53.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:53 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:53.135+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:53.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:53 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:53.135+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:36:53.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:53 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:53.190+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:53.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:53 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:53.190+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:53.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:53 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:53.190+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:36:53.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:53 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:53.433+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:53.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:53 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:53.433+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:22.915632+0000 front 2024-06-06T04:36:22.915656+0000 (oldest deadline 2024-06-06T04:36:48.815814+0000) 2024-06-06T04:36:53.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:53 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:53.613+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:53.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:53 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:53.613+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:53.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:53 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:53.613+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:36:53.859 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:53 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:53.499+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:53.859 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:53 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:53.499+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:36:53.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:53 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:53.734+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:53.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:53 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:53.734+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:53.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:53 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:53.734+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:36:54.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:54 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:54.089+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:54.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:54 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:54.089+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:36:54.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:54 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:54.151+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:54.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:54 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:54.151+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:54.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:54 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:54.151+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:36:54.726 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:54 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:54.425+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:54.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:54 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:54.425+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:22.915632+0000 front 2024-06-06T04:36:22.915656+0000 (oldest deadline 2024-06-06T04:36:48.815814+0000) 2024-06-06T04:36:54.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:54 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:54.622+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:54.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:54 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:54.622+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:54.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:54 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:54.622+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:36:54.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:54 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:54.622+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:36:54.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:54 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:54.720+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:54.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:54 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:54.720+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:54.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:54 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:54.720+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:36:54.859 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:54 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:54.476+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:54.860 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:54 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:54.476+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:36:55.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:55 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:55.110+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:55.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:55 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:55.110+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:36:55.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:55 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:55.130+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:55.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:55 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:55.130+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:55.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:55 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:55.130+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:36:55.726 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:55 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:55.412+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:55.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:55 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:55.412+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:22.915632+0000 front 2024-06-06T04:36:22.915656+0000 (oldest deadline 2024-06-06T04:36:48.815814+0000) 2024-06-06T04:36:55.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:55 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:55.591+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:55.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:55 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:55.591+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:55.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:55 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:55.591+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:36:55.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:55 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:55.591+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:36:55.859 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:55 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:55.507+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:55.859 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:55 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:55.507+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:36:55.859 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:55 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:55.507+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:33.769386+0000 front 2024-06-06T04:36:33.769367+0000 (oldest deadline 2024-06-06T04:36:54.869230+0000) 2024-06-06T04:36:55.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:55 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:55.753+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:55.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:55 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:55.753+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:55.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:55 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:55.753+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:36:55.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:55 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:55.753+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:36:56.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:56 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:56.148+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:56.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:56 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:56.148+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:36:56.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:56 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:56.148+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:36:56.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:56 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:56.148+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:56.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:56 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:56.148+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:56.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:56 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:56.148+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:36:56.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:56 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:56.390+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:56.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:56 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:56.390+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:22.915632+0000 front 2024-06-06T04:36:22.915656+0000 (oldest deadline 2024-06-06T04:36:48.815814+0000) 2024-06-06T04:36:56.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:56 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:56.586+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:56.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:56 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:56.586+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:56.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:56 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:56.586+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:36:56.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:56 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:56.586+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:36:56.795 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:56 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:56.487+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:56.796 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:56 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:56.487+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:36:56.796 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:56 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:56.487+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:33.769386+0000 front 2024-06-06T04:36:33.769367+0000 (oldest deadline 2024-06-06T04:36:54.869230+0000) 2024-06-06T04:36:57.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:56 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:56.794+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:57.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:56 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:56.794+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:57.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:56 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:56.794+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:36:57.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:56 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:56.794+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:36:57.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:57 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:57.141+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:57.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:57 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:57.141+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:57.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:57 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:57.141+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:36:57.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:57 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:57.103+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:57.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:57 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:57.103+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:36:57.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:57 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:57.103+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:36:57.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:57 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:57.392+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:14.814097+0000 front 2024-06-06T04:36:14.814175+0000 (oldest deadline 2024-06-06T04:36:39.513983+0000) 2024-06-06T04:36:57.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:57 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6[60579]: 2024-06-06T04:36:57.392+0000 7fcaaebe1700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:22.915632+0000 front 2024-06-06T04:36:22.915656+0000 (oldest deadline 2024-06-06T04:36:48.815814+0000) 2024-06-06T04:36:57.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:57 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:57.617+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:57.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:57 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:57.617+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:57.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:57 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:57.617+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:36:57.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:57 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:57.617+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:36:57.791 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:57 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:57.499+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:57.791 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:57 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:57.499+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:36:57.791 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:57 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:57.499+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:33.769386+0000 front 2024-06-06T04:36:33.769367+0000 (oldest deadline 2024-06-06T04:36:54.869230+0000) 2024-06-06T04:36:58.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:57 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:57.790+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:58.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:57 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:57.790+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:58.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:57 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:57.790+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:36:58.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:57 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:57.790+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:36:58.283 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:58 smithi045 podman[108031]: 2024-06-06 04:36:58.007134225 +0000 UTC m=+5.521681720 container died cfd33efb5f7f7b6d77b5c86b3ca5eaef4496efe4bc47dd0df7b88e36353b0a35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-06-06T04:36:58.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:58 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:58.145+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:58.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:58 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:58.145+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:36:58.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:58 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:58.145+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:36:58.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:58 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:58.105+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:58.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:58 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:58.105+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:58.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:58 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:58.105+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:36:58.533 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:58 smithi045 podman[108031]: 2024-06-06 04:36:58.357073879 +0000 UTC m=+5.871621377 container cleanup cfd33efb5f7f7b6d77b5c86b3ca5eaef4496efe4bc47dd0df7b88e36353b0a35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T04:36:58.533 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:58 smithi045 bash[108031]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6 2024-06-06T04:36:58.778 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:58 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:58.489+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:58.779 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:58 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:58.489+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:36:58.779 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:58 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:58.489+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:33.769386+0000 front 2024-06-06T04:36:33.769367+0000 (oldest deadline 2024-06-06T04:36:54.869230+0000) 2024-06-06T04:36:58.933 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:58 smithi045 podman[108048]: 2024-06-06 04:36:58.533280103 +0000 UTC m=+0.519118533 container remove cfd33efb5f7f7b6d77b5c86b3ca5eaef4496efe4bc47dd0df7b88e36353b0a35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-06T04:36:58.934 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:58 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:58.622+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:58.934 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:58 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:58.622+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:58.934 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:58 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:58.622+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:36:58.934 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:58 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:58.622+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:36:59.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:58 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:58.778+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:36:59.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:58 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:58.778+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:36:59.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:58 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:58.778+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:36:59.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:58 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:58.778+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:36:59.211 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:58 smithi045 podman[108136]: 2024-06-06 04:36:58.934086441 +0000 UTC m=+0.101405520 container create 1944ab566ba0836a551e1fa85f8bd6ec87b07422f83533d16ebf4af2f09f0224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-06T04:36:59.211 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:58 smithi045 podman[108136]: 2024-06-06 04:36:58.850599624 +0000 UTC m=+0.017918718 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:36:59.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:59 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:59.117+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:36:59.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:59 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:59.117+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:36:59.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:36:59 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:36:59.117+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:36:59.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:59 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:59.086+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:36:59.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:59 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:59.086+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:36:59.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:36:59 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:36:59.086+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:36:59.477 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:59 smithi045 podman[108136]: 2024-06-06 04:36:59.211659164 +0000 UTC m=+0.378978239 container init 1944ab566ba0836a551e1fa85f8bd6ec87b07422f83533d16ebf4af2f09f0224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-06T04:36:59.477 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:59 smithi045 podman[108136]: 2024-06-06 04:36:59.215076085 +0000 UTC m=+0.382395160 container start 1944ab566ba0836a551e1fa85f8bd6ec87b07422f83533d16ebf4af2f09f0224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T04:36:59.477 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:59 smithi045 podman[108136]: 2024-06-06 04:36:59.294981236 +0000 UTC m=+0.462300323 container attach 1944ab566ba0836a551e1fa85f8bd6ec87b07422f83533d16ebf4af2f09f0224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6-deactivate, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-06T04:36:59.814 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:59 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:59.449+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:36:59.814 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:59 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:59.449+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:36:59.814 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:36:59 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:36:59.449+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:33.769386+0000 front 2024-06-06T04:36:33.769367+0000 (oldest deadline 2024-06-06T04:36:54.869230+0000) 2024-06-06T04:36:59.881 INFO:journalctl@ceph.osd.6.smithi045.stdout:Jun 06 04:36:59 smithi045 podman[108136]: 2024-06-06 04:36:59.518342776 +0000 UTC m=+0.685661851 container died 1944ab566ba0836a551e1fa85f8bd6ec87b07422f83533d16ebf4af2f09f0224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T04:36:59.881 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:59 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:59.600+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:36:59.882 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:59 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:59.600+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:36:59.882 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:59 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:59.600+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:36:59.882 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:36:59 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:36:59.600+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:00.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:59 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:59.813+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:00.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:59 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:59.813+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:00.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:59 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:59.813+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:00.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:36:59 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:36:59.813+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:00.197 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.6.service' 2024-06-06T04:37:00.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:37:00.240 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-06T04:37:00.241 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-06T04:37:00.241 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.7 2024-06-06T04:37:00.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:00 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:00.150+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:37:00.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:00 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:00.150+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:37:00.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:00 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:00.150+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:37:00.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:00 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:00.047+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:00.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:00 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:00.047+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:00.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:00 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:00.047+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:00.543 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:00 smithi086 systemd[1]: Stopping Ceph osd.7 for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:37:00.543 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:00 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:00.492+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:37:00.544 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:00 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:00.492+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:37:00.544 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:00 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:00.492+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:33.769386+0000 front 2024-06-06T04:36:33.769367+0000 (oldest deadline 2024-06-06T04:36:54.869230+0000) 2024-06-06T04:37:00.544 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:00 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:00.542+0000 7f2b6aab3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:37:00.544 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:00 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:00.542+0000 7f2b6aab3700 -1 osd.7 104 *** Got signal Terminated *** 2024-06-06T04:37:00.544 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:00 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:00.542+0000 7f2b6aab3700 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T04:37:00.726 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:00 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:00.560+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:00.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:00 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:00.560+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:00.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:00 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:00.560+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:00.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:00 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:00.560+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:01.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:00 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:00.836+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:01.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:00 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:00.836+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:01.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:00 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:00.836+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:01.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:00 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:00.836+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:01.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:01 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:01.118+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:37:01.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:01 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:01.118+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:37:01.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:01 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:01.118+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:37:01.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:01 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:01.021+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:01.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:01 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:01.021+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:01.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:01 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:01.021+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:01.726 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:01 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:01.604+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:01.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:01 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:01.604+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:01.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:01 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:01.604+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:01.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:01 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:01.604+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:01.734 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:01 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:01.443+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:37:01.734 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:01 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:01.443+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:37:01.734 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:01 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:01.443+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:33.769386+0000 front 2024-06-06T04:36:33.769367+0000 (oldest deadline 2024-06-06T04:36:54.869230+0000) 2024-06-06T04:37:02.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:01 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:01.801+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:02.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:01 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:01.801+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:02.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:01 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:01.801+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:02.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:01 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:01.801+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:02.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:01 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:01.971+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:02.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:01 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:01.971+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:02.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:01 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:01.971+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:02.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:02 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:02.166+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:37:02.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:02 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:02.166+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:37:02.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:02 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:02.166+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:37:02.832 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:02 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:02.445+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:37:02.832 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:02 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:02.445+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:37:02.832 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:02 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:02.445+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:33.769386+0000 front 2024-06-06T04:36:33.769367+0000 (oldest deadline 2024-06-06T04:36:54.869230+0000) 2024-06-06T04:37:02.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:02 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:02.587+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:02.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:02 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:02.587+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:02.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:02 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:02.587+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:02.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:02 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:02.587+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:03.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:02 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:02.831+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:03.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:02 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:02.831+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:03.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:02 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:02.831+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:03.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:02 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:02.831+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:03.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:03 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:03.174+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:37:03.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:03 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:03.174+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:37:03.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:03 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:03.174+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:37:03.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:03 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:03.002+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:03.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:03 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:03.002+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:03.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:03 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:03.002+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:03.825 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:03 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:03.410+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:37:03.825 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:03 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:03.410+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:37:03.825 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:03 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:03.410+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:33.769386+0000 front 2024-06-06T04:36:33.769367+0000 (oldest deadline 2024-06-06T04:36:54.869230+0000) 2024-06-06T04:37:03.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:03 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:03.608+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:03.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:03 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:03.608+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:03.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:03 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:03.608+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:03.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:03 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:03.608+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:04.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:03 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:03.824+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:04.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:03 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:03.824+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:04.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:03 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:03.824+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:04.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:03 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:03.824+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:04.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:03 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:03.824+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:04.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:04 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:04.151+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:37:04.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:04 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:04.151+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:37:04.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:04 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:04.151+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:37:04.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:03 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:03.973+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:04.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:03 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:03.973+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:04.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:03 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:03.973+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:04.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:03 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:03.973+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:04.843 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:04 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:04.434+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:37:04.843 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:04 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:04.434+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:37:04.844 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:04 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:04.434+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:33.769386+0000 front 2024-06-06T04:36:33.769367+0000 (oldest deadline 2024-06-06T04:36:54.869230+0000) 2024-06-06T04:37:04.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:04 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:04.583+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:04.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:04 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:04.583+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:04.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:04 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:04.583+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:04.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:04 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:04.583+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:04.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:04 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:04.583+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:05.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:04 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:04.843+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:05.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:04 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:04.843+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:05.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:04 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:04.843+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:05.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:04 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:04.843+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:05.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:04 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:04.843+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:05.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:05 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:05.109+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:37:05.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:05 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:05.109+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:37:05.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:05 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:05.109+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:37:05.432 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:04 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:04.940+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:05.432 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:04 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:04.940+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:05.432 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:04 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:04.940+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:05.432 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:04 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:04.940+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:05.815 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:05 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:05.398+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:13.766590+0000 front 2024-06-06T04:36:13.766627+0000 (oldest deadline 2024-06-06T04:36:43.167514+0000) 2024-06-06T04:37:05.816 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:05 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:05.398+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.168041+0000 front 2024-06-06T04:36:23.168074+0000 (oldest deadline 2024-06-06T04:36:48.468233+0000) 2024-06-06T04:37:05.816 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:05 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7[53723]: 2024-06-06T04:37:05.398+0000 7f2b6309f700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:33.769386+0000 front 2024-06-06T04:36:33.769367+0000 (oldest deadline 2024-06-06T04:36:54.869230+0000) 2024-06-06T04:37:05.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:05 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:05.600+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:05.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:05 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:05.600+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:05.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:05 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:05.600+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:05.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:05 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:05.600+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:05.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:05 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:05.600+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:06.109 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:05 smithi086 podman[69973]: 2024-06-06 04:37:05.908886559 +0000 UTC m=+5.471542787 container died 539d7e11074d0577578b818bd32228377f92dec8c588628c062a1c4d31f3f994 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-06T04:37:06.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:05 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:05.815+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:06.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:05 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:05.815+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:06.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:05 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:05.815+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:06.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:05 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:05.815+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:06.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:05 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:05.815+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:06.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:05 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:05.899+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:06.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:05 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:05.899+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:06.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:05 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:05.899+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:06.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:05 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:05.899+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:06.181 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:06 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:06.128+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:37:06.181 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:06 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:06.128+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:37:06.181 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:06 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:06.128+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:37:06.181 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:06 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:06.128+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.589036+0000 front 2024-06-06T04:36:40.589088+0000 (oldest deadline 2024-06-06T04:37:05.889072+0000) 2024-06-06T04:37:06.569 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:06 smithi086 podman[69973]: 2024-06-06 04:37:06.260020567 +0000 UTC m=+5.822676790 container cleanup 539d7e11074d0577578b818bd32228377f92dec8c588628c062a1c4d31f3f994 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T04:37:06.569 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:06 smithi086 bash[69973]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7 2024-06-06T04:37:06.859 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:06 smithi086 podman[70002]: 2024-06-06 04:37:06.569569085 +0000 UTC m=+0.657535301 container remove 539d7e11074d0577578b818bd32228377f92dec8c588628c062a1c4d31f3f994 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:37:06.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:06 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:06.766+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:06.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:06 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:06.766+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:06.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:06 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:06.766+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:06.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:06 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:06.766+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:06.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:06 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:06.766+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:06.976 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:06 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:06.560+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:06.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:06 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:06.560+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:06.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:06 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:06.560+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:06.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:06 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:06.560+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:06.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:06 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:06.560+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:07.148 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:06 smithi086 podman[70090]: 2024-06-06 04:37:06.795618804 +0000 UTC m=+0.017030553 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:37:07.148 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:06 smithi086 podman[70090]: 2024-06-06 04:37:06.912780366 +0000 UTC m=+0.134192167 container create 7d6d6d18b9600562b989e83f3d3abda0f4f009e928bed54acd1ea33383fb75f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7-deactivate, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-06-06T04:37:07.148 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:07 smithi086 podman[70090]: 2024-06-06 04:37:07.148545992 +0000 UTC m=+0.369957725 container init 7d6d6d18b9600562b989e83f3d3abda0f4f009e928bed54acd1ea33383fb75f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-06-06T04:37:07.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:06 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:06.881+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:07.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:06 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:06.881+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:07.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:06 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:06.881+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:07.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:06 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:06.881+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:07.181 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:07 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:07.085+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:37:07.181 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:07 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:07.085+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:37:07.182 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:07 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:07.085+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:37:07.182 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:07 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:07.085+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.589036+0000 front 2024-06-06T04:36:40.589088+0000 (oldest deadline 2024-06-06T04:37:05.889072+0000) 2024-06-06T04:37:07.442 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:07 smithi086 podman[70090]: 2024-06-06 04:37:07.151959367 +0000 UTC m=+0.373371109 container start 7d6d6d18b9600562b989e83f3d3abda0f4f009e928bed54acd1ea33383fb75f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-06T04:37:07.442 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:07 smithi086 podman[70090]: 2024-06-06 04:37:07.248380204 +0000 UTC m=+0.469791975 container attach 7d6d6d18b9600562b989e83f3d3abda0f4f009e928bed54acd1ea33383fb75f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7-deactivate, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-06T04:37:07.767 INFO:journalctl@ceph.osd.7.smithi086.stdout:Jun 06 04:37:07 smithi086 podman[70090]: 2024-06-06 04:37:07.442849987 +0000 UTC m=+0.664261738 container died 7d6d6d18b9600562b989e83f3d3abda0f4f009e928bed54acd1ea33383fb75f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-06T04:37:07.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:07 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:07.579+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:07.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:07 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:07.579+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:07.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:07 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:07.579+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:07.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:07 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:07.579+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:07.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:07 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:07.579+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:08.043 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:07 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:07.787+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:08.043 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:07 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:07.787+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:08.043 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:07 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:07.787+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:08.044 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:07 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:07.787+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:08.044 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:07 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:07.787+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:08.071 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.7.service' 2024-06-06T04:37:08.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:37:08.116 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-06T04:37:08.116 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-06-06T04:37:08.117 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.8 2024-06-06T04:37:08.166 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:08 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:08.105+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:37:08.166 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:08 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:08.105+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:37:08.166 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:08 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:08.105+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:37:08.166 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:08 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:08.105+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.589036+0000 front 2024-06-06T04:36:40.589088+0000 (oldest deadline 2024-06-06T04:37:05.889072+0000) 2024-06-06T04:37:08.167 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:07 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:07.904+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:08.167 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:07 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:07.904+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:08.167 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:07 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:07.904+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:08.167 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:07 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:07.904+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:08.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:08 smithi154 systemd[1]: Stopping Ceph osd.8 for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:37:08.888 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:08 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:08.437+0000 7ffb7bc50700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:37:08.889 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:08 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:08.437+0000 7ffb7bc50700 -1 osd.8 104 *** Got signal Terminated *** 2024-06-06T04:37:08.889 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:08 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:08.437+0000 7ffb7bc50700 -1 osd.8 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T04:37:08.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:08 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:08.617+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:08.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:08 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:08.617+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:08.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:08 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:08.617+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:08.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:08 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:08.617+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:08.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:08 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:08.617+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:09.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:08 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:08.801+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:09.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:08 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:08.801+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:09.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:08 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:08.801+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:09.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:08 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:08.801+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:09.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:08 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:08.801+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:09.180 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:09 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:09.131+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:37:09.180 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:09 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:09.131+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:37:09.180 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:09 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:09.131+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:37:09.180 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:09 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:09.131+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.589036+0000 front 2024-06-06T04:36:40.589088+0000 (oldest deadline 2024-06-06T04:37:05.889072+0000) 2024-06-06T04:37:09.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:08 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:08.888+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:09.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:08 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:08.888+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:09.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:08 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:08.888+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:09.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:08 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:08.888+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:09.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:09 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:09.609+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:09.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:09 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:09.609+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:09.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:09 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:09.609+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:09.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:09 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:09.609+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:09.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:09 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:09.609+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:10.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:09 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:09.806+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:10.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:09 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:09.806+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:10.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:09 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:09.806+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:10.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:09 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:09.806+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:10.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:09 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:09.806+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:10.140 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:09 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:09.878+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:10.140 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:09 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:09.878+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:10.141 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:09 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:09.878+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:10.141 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:09 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:09.878+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:10.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:10 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:10.139+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:37:10.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:10 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:10.139+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:37:10.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:10 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:10.139+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:37:10.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:10 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:10.139+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.589036+0000 front 2024-06-06T04:36:40.589088+0000 (oldest deadline 2024-06-06T04:37:05.889072+0000) 2024-06-06T04:37:10.919 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:10 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:10.643+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:10.919 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:10 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:10.643+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:10.920 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:10 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:10.643+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:10.920 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:10 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:10.643+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:10.920 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:10 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:10.643+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:11.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:10 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:10.758+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:11.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:10 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:10.758+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:11.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:10 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:10.758+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:11.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:10 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:10.758+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:11.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:10 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:10.758+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:11.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:10 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:10.838+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:11.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:10 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:10.838+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:11.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:10 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:10.838+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:11.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:10 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:10.838+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:11.680 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:11 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:11.181+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:37:11.681 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:11 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:11.181+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:37:11.681 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:11 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:11.181+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:37:11.681 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:11 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:11.181+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.589036+0000 front 2024-06-06T04:36:40.589088+0000 (oldest deadline 2024-06-06T04:37:05.889072+0000) 2024-06-06T04:37:11.681 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:11 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:11.181+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.790094+0000 front 2024-06-06T04:36:48.790134+0000 (oldest deadline 2024-06-06T04:37:11.090314+0000) 2024-06-06T04:37:11.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:11 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:11.613+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:11.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:11 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:11.613+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:11.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:11 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:11.613+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:11.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:11 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:11.613+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:11.978 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:11 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:11.613+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:11.978 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:11 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:11.613+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.028387+0000 front 2024-06-06T04:36:48.028336+0000 (oldest deadline 2024-06-06T04:37:11.528420+0000) 2024-06-06T04:37:12.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:11 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:11.799+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:12.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:11 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:11.799+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:12.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:11 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:11.799+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:12.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:11 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:11.799+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:12.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:11 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:11.799+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:12.144 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:11 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:11.818+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:12.144 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:11 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:11.818+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:12.144 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:11 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:11.818+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:12.144 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:11 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:11.818+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:12.144 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:11 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:11.818+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:12.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:12 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:12.143+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:37:12.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:12 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:12.143+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:37:12.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:12 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:12.143+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:37:12.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:12 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:12.143+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.589036+0000 front 2024-06-06T04:36:40.589088+0000 (oldest deadline 2024-06-06T04:37:05.889072+0000) 2024-06-06T04:37:12.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:12 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:12.143+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.790094+0000 front 2024-06-06T04:36:48.790134+0000 (oldest deadline 2024-06-06T04:37:11.090314+0000) 2024-06-06T04:37:12.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:12 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:12.633+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:12.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:12 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:12.633+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:12.978 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:12 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:12.633+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:12.978 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:12 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:12.633+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:12.978 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:12 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:12.633+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:12.978 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:12 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:12.633+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.028387+0000 front 2024-06-06T04:36:48.028336+0000 (oldest deadline 2024-06-06T04:37:11.528420+0000) 2024-06-06T04:37:13.099 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:12 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:12.792+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:13.100 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:12 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:12.792+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:13.100 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:12 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:12.792+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:13.100 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:12 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:12.792+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:13.100 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:12 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:12.792+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:13.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:12 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:12.810+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:13.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:12 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:12.810+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:13.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:12 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:12.810+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:13.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:12 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:12.810+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:13.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:12 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:12.810+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:13.385 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:13 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:13.099+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:06.183888+0000 front 2024-06-06T04:36:06.183798+0000 (oldest deadline 2024-06-06T04:36:35.584205+0000) 2024-06-06T04:37:13.385 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:13 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:13.099+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:23.185635+0000 front 2024-06-06T04:36:23.185667+0000 (oldest deadline 2024-06-06T04:36:47.885898+0000) 2024-06-06T04:37:13.386 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:13 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:13.099+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:32.586942+0000 front 2024-06-06T04:36:32.586928+0000 (oldest deadline 2024-06-06T04:36:56.086940+0000) 2024-06-06T04:37:13.386 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:13 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:13.099+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.589036+0000 front 2024-06-06T04:36:40.589088+0000 (oldest deadline 2024-06-06T04:37:05.889072+0000) 2024-06-06T04:37:13.386 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:13 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8[54675]: 2024-06-06T04:37:13.099+0000 7ffb78244700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.790094+0000 front 2024-06-06T04:36:48.790134+0000 (oldest deadline 2024-06-06T04:37:11.090314+0000) 2024-06-06T04:37:13.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:13 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:13.591+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:13.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:13 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:13.591+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:13.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:13 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:13.591+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:13.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:13 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:13.591+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:13.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:13 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:13.591+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:13.978 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:13 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:13.591+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.028387+0000 front 2024-06-06T04:36:48.028336+0000 (oldest deadline 2024-06-06T04:37:11.528420+0000) 2024-06-06T04:37:14.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:13 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:13.781+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:14.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:13 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:13.781+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:14.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:13 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:13.781+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:14.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:13 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:13.781+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:14.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:13 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:13.781+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:14.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:13 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:13.781+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:14.154 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:13 smithi154 podman[71056]: 2024-06-06 04:37:13.858798069 +0000 UTC m=+5.542527586 container died 5c829d3bf712627df52a7bf91fc8d4ecfb9b59a0bccdcda30a18657993912680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-06T04:37:14.154 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:13 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:13.769+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:14.154 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:13 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:13.769+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:14.155 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:13 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:13.769+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:14.155 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:13 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:13.769+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:14.155 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:13 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:13.769+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:14.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:14 smithi154 podman[71056]: 2024-06-06 04:37:14.262235676 +0000 UTC m=+5.945965237 container cleanup 5c829d3bf712627df52a7bf91fc8d4ecfb9b59a0bccdcda30a18657993912680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, org.label-schema.vendor=CentOS) 2024-06-06T04:37:14.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:14 smithi154 bash[71056]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8 2024-06-06T04:37:14.795 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:14 smithi154 podman[71085]: 2024-06-06 04:37:14.513346091 +0000 UTC m=+0.651610456 container remove 5c829d3bf712627df52a7bf91fc8d4ecfb9b59a0bccdcda30a18657993912680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-06T04:37:14.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:14 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:14.602+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:14.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:14 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:14.602+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:14.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:14 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:14.602+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:14.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:14 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:14.602+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:14.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:14 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:14.602+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:14.978 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:14 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:14.602+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.028387+0000 front 2024-06-06T04:36:48.028336+0000 (oldest deadline 2024-06-06T04:37:11.528420+0000) 2024-06-06T04:37:15.053 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:14 smithi154 podman[71175]: 2024-06-06 04:37:14.836352454 +0000 UTC m=+0.017101248 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:37:15.054 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:14 smithi154 podman[71175]: 2024-06-06 04:37:14.955800653 +0000 UTC m=+0.136549430 container create 072e4dc1d45795b512cfdc2023f4755cd88782af9deb23fdff8fb04b660a813f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-06T04:37:15.054 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:14 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:14.794+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:15.054 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:14 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:14.794+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:15.054 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:14 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:14.794+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:15.054 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:14 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:14.794+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:15.055 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:14 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:14.794+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:15.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:14 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:14.746+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:15.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:14 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:14.746+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:15.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:14 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:14.746+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:15.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:14 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:14.746+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:15.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:14 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:14.746+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:15.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:14 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:14.746+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:15.430 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:15 smithi154 podman[71175]: 2024-06-06 04:37:15.174985147 +0000 UTC m=+0.355733924 container init 072e4dc1d45795b512cfdc2023f4755cd88782af9deb23fdff8fb04b660a813f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-06T04:37:15.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:15 smithi154 podman[71175]: 2024-06-06 04:37:15.179120776 +0000 UTC m=+0.359869553 container start 072e4dc1d45795b512cfdc2023f4755cd88782af9deb23fdff8fb04b660a813f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True) 2024-06-06T04:37:15.431 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:15 smithi154 podman[71175]: 2024-06-06 04:37:15.258141264 +0000 UTC m=+0.438890061 container attach 072e4dc1d45795b512cfdc2023f4755cd88782af9deb23fdff8fb04b660a813f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-06T04:37:15.750 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:15 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:15.747+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:15.750 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:15 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:15.747+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:15.750 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:15 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:15.747+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:15.751 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:15 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:15.748+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:15.751 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:15 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:15.748+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:15.751 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:15 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:15.748+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:15.786 INFO:journalctl@ceph.osd.8.smithi154.stdout:Jun 06 04:37:15 smithi154 podman[71175]: 2024-06-06 04:37:15.487800422 +0000 UTC m=+0.668549205 container died 072e4dc1d45795b512cfdc2023f4755cd88782af9deb23fdff8fb04b660a813f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-06T04:37:15.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:15 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:15.575+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:15.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:15 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:15.575+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:15.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:15 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:15.575+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:15.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:15 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:15.575+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:15.978 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:15 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:15.575+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:15.978 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:15 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:15.575+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.028387+0000 front 2024-06-06T04:36:48.028336+0000 (oldest deadline 2024-06-06T04:37:11.528420+0000) 2024-06-06T04:37:16.061 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:15 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:15.812+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:16.062 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:15 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:15.812+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:16.062 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:15 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:15.812+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:16.062 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:15 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:15.812+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:16.062 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:15 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:15.812+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:16.081 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.8.service' 2024-06-06T04:37:16.120 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:37:16.120 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-06-06T04:37:16.120 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-06-06T04:37:16.121 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.9 2024-06-06T04:37:16.460 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:16 smithi045 systemd[1]: Stopping Ceph osd.9 for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:37:16.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:16 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:16.459+0000 7f08918b2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:37:16.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:16 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:16.459+0000 7f08918b2700 -1 osd.9 104 *** Got signal Terminated *** 2024-06-06T04:37:16.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:16 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:16.459+0000 7f08918b2700 -1 osd.9 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T04:37:16.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:16 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:16.564+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:16.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:16 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:16.564+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:16.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:16 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:16.564+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:16.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:16 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:16.564+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:16.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:16 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:16.564+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:16.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:16 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:16.564+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.028387+0000 front 2024-06-06T04:36:48.028336+0000 (oldest deadline 2024-06-06T04:37:11.528420+0000) 2024-06-06T04:37:17.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:16 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:16.788+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:17.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:16 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:16.788+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:17.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:16 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:16.788+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:17.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:16 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:16.788+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:17.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:16 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:16.788+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:17.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:16 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:16.788+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:17.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:16 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:16.785+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:17.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:16 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:16.785+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:17.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:16 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:16.785+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:17.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:16 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:16.785+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:17.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:16 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:16.785+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:17.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:17 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:17.515+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:17.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:17 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:17.515+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:17.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:17 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:17.515+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:17.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:17 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:17.515+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:17.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:17 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:17.515+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:17.978 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:17 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:17.515+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.028387+0000 front 2024-06-06T04:36:48.028336+0000 (oldest deadline 2024-06-06T04:37:11.528420+0000) 2024-06-06T04:37:18.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:17 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:17.762+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:18.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:17 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:17.762+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:18.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:17 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:17.762+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:18.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:17 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:17.762+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:18.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:17 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:17.762+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:18.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:17 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:17.762+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:18.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:17 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:17.796+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:18.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:17 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:17.796+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:18.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:17 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:17.796+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:18.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:17 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:17.796+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:18.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:17 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:17.796+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:18.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:18 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:18.502+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:18.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:18 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:18.502+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:18.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:18 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:18.502+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:18.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:18 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:18.502+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:18.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:18 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:18.502+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:18.978 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:18 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:18.502+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.028387+0000 front 2024-06-06T04:36:48.028336+0000 (oldest deadline 2024-06-06T04:37:11.528420+0000) 2024-06-06T04:37:19.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:18 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:18.720+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:19.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:18 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:18.720+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:19.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:18 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:18.720+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:19.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:18 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:18.720+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:19.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:18 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:18.720+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:19.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:18 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:18.720+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:19.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:18 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:18.751+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:19.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:18 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:18.751+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:19.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:18 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:18.751+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:19.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:18 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:18.751+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:19.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:18 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:18.751+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:19.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:19 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:19.507+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:19.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:19 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:19.507+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:19.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:19 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:19.507+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:19.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:19 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:19.507+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:19.978 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:19 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:19.507+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:19.978 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:19 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:19.507+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.028387+0000 front 2024-06-06T04:36:48.028336+0000 (oldest deadline 2024-06-06T04:37:11.528420+0000) 2024-06-06T04:37:20.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:19 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:19.674+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:20.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:19 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:19.674+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:20.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:19 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:19.674+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:20.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:19 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:19.674+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:20.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:19 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:19.674+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:20.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:19 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:19.674+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:20.111 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:19 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:19.674+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:53.145361+0000 front 2024-06-06T04:36:53.145413+0000 (oldest deadline 2024-06-06T04:37:19.045267+0000) 2024-06-06T04:37:20.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:19 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:19.790+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:20.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:19 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:19.790+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:20.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:19 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:19.790+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:20.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:19 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:19.790+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:20.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:19 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:19.790+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:20.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:20 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:20.545+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:05.520884+0000 front 2024-06-06T04:36:05.520906+0000 (oldest deadline 2024-06-06T04:36:30.821025+0000) 2024-06-06T04:37:20.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:20 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:20.545+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:17.222687+0000 front 2024-06-06T04:36:17.222781+0000 (oldest deadline 2024-06-06T04:36:41.322791+0000) 2024-06-06T04:37:20.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:20 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:20.545+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:21.323387+0000 front 2024-06-06T04:36:21.323401+0000 (oldest deadline 2024-06-06T04:36:46.623364+0000) 2024-06-06T04:37:20.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:20 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:20.545+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:31.924285+0000 front 2024-06-06T04:36:31.924339+0000 (oldest deadline 2024-06-06T04:36:54.224369+0000) 2024-06-06T04:37:20.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:20 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:20.545+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.526770+0000 front 2024-06-06T04:36:40.526673+0000 (oldest deadline 2024-06-06T04:37:04.026728+0000) 2024-06-06T04:37:20.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:20 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9[67578]: 2024-06-06T04:37:20.545+0000 7f088dea6700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.028387+0000 front 2024-06-06T04:36:48.028336+0000 (oldest deadline 2024-06-06T04:37:11.528420+0000) 2024-06-06T04:37:21.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:20 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:20.642+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:21.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:20 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:20.642+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:21.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:20 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:20.642+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:21.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:20 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:20.642+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:21.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:20 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:20.642+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:21.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:20 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:20.642+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:21.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:20 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:20.642+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:53.145361+0000 front 2024-06-06T04:36:53.145413+0000 (oldest deadline 2024-06-06T04:37:19.045267+0000) 2024-06-06T04:37:21.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:20 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:20.817+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:21.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:20 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:20.817+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:21.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:20 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:20.817+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:21.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:20 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:20.817+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:21.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:20 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:20.817+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:21.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:20 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:20.817+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:22.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:21 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:21.625+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:22.168 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:21 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:21.625+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:22.168 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:21 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:21.625+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:22.168 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:21 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:21.625+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:22.168 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:21 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:21.625+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:22.168 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:21 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:21.625+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:22.168 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:21 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:21.625+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:53.145361+0000 front 2024-06-06T04:36:53.145413+0000 (oldest deadline 2024-06-06T04:37:19.045267+0000) 2024-06-06T04:37:22.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:21 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:21.790+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:22.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:21 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:21.790+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:22.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:21 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:21.790+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:22.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:21 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:21.790+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:22.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:21 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:21.790+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:22.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:21 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:21.790+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:22.201 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:21 smithi045 podman[108252]: 2024-06-06 04:37:21.923762111 +0000 UTC m=+5.591961932 container died 9c168550f5733286e2a2784ea96b8365826756bfbae52c299c4c0fdd7fcf5671 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-06T04:37:22.477 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:22 smithi045 podman[108252]: 2024-06-06 04:37:22.409686574 +0000 UTC m=+6.077886386 container cleanup 9c168550f5733286e2a2784ea96b8365826756bfbae52c299c4c0fdd7fcf5671 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:37:22.477 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:22 smithi045 bash[108252]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9 2024-06-06T04:37:22.968 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:22 smithi045 podman[108267]: 2024-06-06 04:37:22.64437081 +0000 UTC m=+0.713011833 container remove 9c168550f5733286e2a2784ea96b8365826756bfbae52c299c4c0fdd7fcf5671 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-06-06T04:37:23.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:22 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:22.628+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:23.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:22 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:22.628+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:23.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:22 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:22.628+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:23.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:22 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:22.628+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:23.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:22 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:22.628+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:23.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:22 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:22.628+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:23.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:22 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:22.628+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:53.145361+0000 front 2024-06-06T04:36:53.145413+0000 (oldest deadline 2024-06-06T04:37:19.045267+0000) 2024-06-06T04:37:23.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:22 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:22.837+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:23.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:22 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:22.837+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:23.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:22 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:22.837+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:23.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:22 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:22.837+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:23.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:22 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:22.837+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:23.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:22 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:22.837+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:23.227 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:22 smithi045 podman[108356]: 2024-06-06 04:37:22.871117681 +0000 UTC m=+0.017327437 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:37:23.227 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:22 smithi045 podman[108356]: 2024-06-06 04:37:22.978414524 +0000 UTC m=+0.124624262 container create 5f335ed3daa4dc54a0d38a3a6a2fca1998b8450e72d7140c785f9d635c953ce3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T04:37:23.227 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:23 smithi045 podman[108356]: 2024-06-06 04:37:23.205777932 +0000 UTC m=+0.351987675 container init 5f335ed3daa4dc54a0d38a3a6a2fca1998b8450e72d7140c785f9d635c953ce3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , ceph=True) 2024-06-06T04:37:23.228 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:23 smithi045 podman[108356]: 2024-06-06 04:37:23.209073641 +0000 UTC m=+0.355283387 container start 5f335ed3daa4dc54a0d38a3a6a2fca1998b8450e72d7140c785f9d635c953ce3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-06T04:37:23.514 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:23 smithi045 podman[108356]: 2024-06-06 04:37:23.314259281 +0000 UTC m=+0.460469040 container attach 5f335ed3daa4dc54a0d38a3a6a2fca1998b8450e72d7140c785f9d635c953ce3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-06T04:37:23.808 INFO:journalctl@ceph.osd.9.smithi045.stdout:Jun 06 04:37:23 smithi045 podman[108356]: 2024-06-06 04:37:23.514452082 +0000 UTC m=+0.660661824 container died 5f335ed3daa4dc54a0d38a3a6a2fca1998b8450e72d7140c785f9d635c953ce3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-06T04:37:24.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:23 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:23.676+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:24.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:23 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:23.676+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:24.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:23 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:23.676+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:24.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:23 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:23.676+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:24.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:23 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:23.676+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:24.111 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:23 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:23.676+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:24.111 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:23 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:23.676+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:53.145361+0000 front 2024-06-06T04:36:53.145413+0000 (oldest deadline 2024-06-06T04:37:19.045267+0000) 2024-06-06T04:37:24.176 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.9.service' 2024-06-06T04:37:24.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:23 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:23.867+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:24.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:23 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:23.867+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:24.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:23 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:23.867+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:24.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:23 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:23.867+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:24.182 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:23 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:23.867+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:24.182 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:23 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:23.867+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:24.225 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:37:24.225 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-06-06T04:37:24.225 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-06-06T04:37:24.226 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.10 2024-06-06T04:37:24.546 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:24 smithi086 systemd[1]: Stopping Ceph osd.10 for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:37:24.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:24 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:24.545+0000 7f24530ec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:37:24.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:24 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:24.545+0000 7f24530ec700 -1 osd.10 104 *** Got signal Terminated *** 2024-06-06T04:37:24.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:24 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:24.545+0000 7f24530ec700 -1 osd.10 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T04:37:24.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:24 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:24.643+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:24.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:24 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:24.643+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:24.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:24 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:24.643+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:24.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:24 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:24.643+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:24.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:24 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:24.643+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:24.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:24 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:24.643+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:24.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:24 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:24.643+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:53.145361+0000 front 2024-06-06T04:36:53.145413+0000 (oldest deadline 2024-06-06T04:37:19.045267+0000) 2024-06-06T04:37:25.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:24 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:24.903+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:25.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:24 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:24.903+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:25.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:24 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:24.903+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:25.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:24 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:24.903+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:25.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:24 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:24.903+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:25.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:24 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:24.903+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:26.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:25 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:25.624+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:26.109 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:25 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:25.624+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:26.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:25 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:25.624+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:26.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:25 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:25.624+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:26.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:25 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:25.624+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:26.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:25 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:25.624+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:26.110 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:25 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:25.624+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:53.145361+0000 front 2024-06-06T04:36:53.145413+0000 (oldest deadline 2024-06-06T04:37:19.045267+0000) 2024-06-06T04:37:26.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:25 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:25.941+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:26.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:25 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:25.941+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:26.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:25 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:25.941+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:26.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:25 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:25.941+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:26.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:25 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:25.941+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:26.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:25 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:25.941+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:26.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:26 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:26.589+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:26.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:26 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:26.589+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:26.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:26 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:26.589+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:26.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:26 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:26.589+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:26.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:26 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:26.589+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:26.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:26 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:26.589+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:26.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:26 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:26.589+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:53.145361+0000 front 2024-06-06T04:36:53.145413+0000 (oldest deadline 2024-06-06T04:37:19.045267+0000) 2024-06-06T04:37:27.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:26 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:26.928+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:27.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:26 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:26.928+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:27.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:26 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:26.928+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:27.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:26 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:26.928+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:27.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:26 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:26.928+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:27.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:26 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:26.928+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:27.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:27 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:27.582+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:27.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:27 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:27.582+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:27.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:27 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:27.582+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:27.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:27 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:27.582+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:27.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:27 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:27.582+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:27.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:27 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:27.582+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:27.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:27 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:27.582+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:53.145361+0000 front 2024-06-06T04:36:53.145413+0000 (oldest deadline 2024-06-06T04:37:19.045267+0000) 2024-06-06T04:37:28.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:27 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:27.962+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:28.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:27 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:27.962+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:28.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:27 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:27.962+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:28.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:27 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:27.962+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:28.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:27 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:27.962+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:28.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:27 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:27.962+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:28.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:27 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:27.962+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-06-06T04:37:04.762784+0000 front 2024-06-06T04:37:04.762816+0000 (oldest deadline 2024-06-06T04:37:27.062676+0000) 2024-06-06T04:37:28.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:28 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:28.581+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:28.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:28 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:28.581+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:28.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:28 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:28.581+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:28.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:28 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:28.581+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:28.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:28 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:28.581+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:28.861 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:28 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:28.581+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:28.861 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:28 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:28.581+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:53.145361+0000 front 2024-06-06T04:36:53.145413+0000 (oldest deadline 2024-06-06T04:37:19.045267+0000) 2024-06-06T04:37:29.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:28 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:28.979+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:29.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:28 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:28.979+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:29.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:28 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:28.979+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:29.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:28 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:28.979+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:29.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:28 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:28.979+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:29.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:28 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:28.979+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:29.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:28 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:28.979+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-06-06T04:37:04.762784+0000 front 2024-06-06T04:37:04.762816+0000 (oldest deadline 2024-06-06T04:37:27.062676+0000) 2024-06-06T04:37:29.859 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:29 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:29.542+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.338073+0000 front 2024-06-06T04:36:08.337879+0000 (oldest deadline 2024-06-06T04:36:30.638057+0000) 2024-06-06T04:37:29.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:29 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:29.542+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:12.939124+0000 front 2024-06-06T04:36:12.939148+0000 (oldest deadline 2024-06-06T04:36:38.239244+0000) 2024-06-06T04:37:29.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:29 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:29.542+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:24.641072+0000 front 2024-06-06T04:36:24.641055+0000 (oldest deadline 2024-06-06T04:36:49.341088+0000) 2024-06-06T04:37:29.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:29 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:29.542+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-06-06T04:36:29.341541+0000 front 2024-06-06T04:36:29.341507+0000 (oldest deadline 2024-06-06T04:36:55.241577+0000) 2024-06-06T04:37:29.860 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:29 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:29.542+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:39.342383+0000 front 2024-06-06T04:36:39.342409+0000 (oldest deadline 2024-06-06T04:37:03.442480+0000) 2024-06-06T04:37:29.861 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:29 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:29.542+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:48.445135+0000 front 2024-06-06T04:36:48.445205+0000 (oldest deadline 2024-06-06T04:37:13.144896+0000) 2024-06-06T04:37:29.861 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:29 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10[60733]: 2024-06-06T04:37:29.542+0000 7f244f6e0700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:53.145361+0000 front 2024-06-06T04:36:53.145413+0000 (oldest deadline 2024-06-06T04:37:19.045267+0000) 2024-06-06T04:37:30.254 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:30 smithi086 podman[70196]: 2024-06-06 04:37:30.000447367 +0000 UTC m=+5.581712920 container died 171a037fe253209d0f83941873a9ecaafb182d1be40a7e4848047d41b590b8c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T04:37:30.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:29 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:29.933+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:30.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:29 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:29.933+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:30.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:29 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:29.933+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:30.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:29 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:29.933+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:30.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:29 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:29.933+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:30.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:29 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:29.933+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:30.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:29 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:29.933+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-06-06T04:37:04.762784+0000 front 2024-06-06T04:37:04.762816+0000 (oldest deadline 2024-06-06T04:37:27.062676+0000) 2024-06-06T04:37:30.609 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:30 smithi086 podman[70196]: 2024-06-06 04:37:30.378929415 +0000 UTC m=+5.960194998 container cleanup 171a037fe253209d0f83941873a9ecaafb182d1be40a7e4848047d41b590b8c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T04:37:30.609 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:30 smithi086 bash[70196]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10 2024-06-06T04:37:30.980 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:30 smithi086 podman[70213]: 2024-06-06 04:37:30.629956989 +0000 UTC m=+0.625408982 container remove 171a037fe253209d0f83941873a9ecaafb182d1be40a7e4848047d41b590b8c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-06T04:37:31.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:30 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:30.887+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:31.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:30 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:30.887+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:31.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:30 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:30.887+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:31.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:30 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:30.887+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:31.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:30 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:30.887+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:31.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:30 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:30.887+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:31.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:30 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:30.887+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-06-06T04:37:04.762784+0000 front 2024-06-06T04:37:04.762816+0000 (oldest deadline 2024-06-06T04:37:27.062676+0000) 2024-06-06T04:37:31.267 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:30 smithi086 podman[70303]: 2024-06-06 04:37:30.980899898 +0000 UTC m=+0.100082390 container create 9f2661997dd316cab3bc519d54866217aa6295f96107aafaceff69b84b8084cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-06T04:37:31.268 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:30 smithi086 podman[70303]: 2024-06-06 04:37:30.898019355 +0000 UTC m=+0.017201875 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:37:31.268 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:31 smithi086 podman[70303]: 2024-06-06 04:37:31.183069868 +0000 UTC m=+0.302252361 container init 9f2661997dd316cab3bc519d54866217aa6295f96107aafaceff69b84b8084cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-06T04:37:31.268 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:31 smithi086 podman[70303]: 2024-06-06 04:37:31.186273774 +0000 UTC m=+0.305456266 container start 9f2661997dd316cab3bc519d54866217aa6295f96107aafaceff69b84b8084cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-06T04:37:31.609 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:31 smithi086 podman[70303]: 2024-06-06 04:37:31.267573017 +0000 UTC m=+0.386755510 container attach 9f2661997dd316cab3bc519d54866217aa6295f96107aafaceff69b84b8084cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=) 2024-06-06T04:37:31.609 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:31 smithi086 podman[70303]: 2024-06-06 04:37:31.478437374 +0000 UTC m=+0.597619869 container died 9f2661997dd316cab3bc519d54866217aa6295f96107aafaceff69b84b8084cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-06-06T04:37:32.077 INFO:journalctl@ceph.osd.10.smithi086.stdout:Jun 06 04:37:32 smithi086 podman[70329]: 2024-06-06 04:37:32.061055855 +0000 UTC m=+0.573803514 container remove 9f2661997dd316cab3bc519d54866217aa6295f96107aafaceff69b84b8084cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-10-deactivate, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-06T04:37:32.082 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.10.service' 2024-06-06T04:37:32.121 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:37:32.121 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-06-06T04:37:32.122 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-06-06T04:37:32.122 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.11 2024-06-06T04:37:32.149 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:31 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:31.882+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:32.149 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:31 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:31.882+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:32.149 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:31 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:31.882+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:32.150 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:31 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:31.882+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:32.150 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:31 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:31.882+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:32.150 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:31 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:31.882+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:32.150 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:31 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:31.882+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-06-06T04:37:04.762784+0000 front 2024-06-06T04:37:04.762816+0000 (oldest deadline 2024-06-06T04:37:27.062676+0000) 2024-06-06T04:37:32.413 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:32 smithi154 systemd[1]: Stopping Ceph osd.11 for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:37:32.680 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:32 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:32.413+0000 7fa917f4f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:37:32.680 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:32 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:32.413+0000 7fa917f4f700 -1 osd.11 104 *** Got signal Terminated *** 2024-06-06T04:37:32.680 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:32 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:32.413+0000 7fa917f4f700 -1 osd.11 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T04:37:33.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:32 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:32.914+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:33.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:32 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:32.914+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:33.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:32 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:32.914+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:33.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:32 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:32.914+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:33.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:32 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:32.914+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:33.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:32 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:32.914+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:33.181 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:32 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:32.914+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-06-06T04:37:04.762784+0000 front 2024-06-06T04:37:04.762816+0000 (oldest deadline 2024-06-06T04:37:27.062676+0000) 2024-06-06T04:37:34.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:33 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:33.959+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:34.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:33 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:33.959+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:34.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:33 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:33.959+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:34.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:33 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:33.959+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:34.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:33 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:33.959+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:34.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:33 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:33.959+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:34.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:33 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:33.959+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-06-06T04:37:04.762784+0000 front 2024-06-06T04:37:04.762816+0000 (oldest deadline 2024-06-06T04:37:27.062676+0000) 2024-06-06T04:37:35.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:34 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:34.953+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:35.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:34 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:34.953+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:35.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:34 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:34.953+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:35.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:34 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:34.953+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:35.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:34 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:34.953+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:35.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:34 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:34.953+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:35.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:34 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:34.953+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-06-06T04:37:04.762784+0000 front 2024-06-06T04:37:04.762816+0000 (oldest deadline 2024-06-06T04:37:27.062676+0000) 2024-06-06T04:37:36.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:35 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:35.955+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:36.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:35 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:35.955+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:36.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:35 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:35.955+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:36.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:35 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:35.955+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:36.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:35 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:35.955+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:36.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:35 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:35.955+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:36.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:35 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:35.955+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-06-06T04:37:04.762784+0000 front 2024-06-06T04:37:04.762816+0000 (oldest deadline 2024-06-06T04:37:27.062676+0000) 2024-06-06T04:37:36.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:35 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:35.955+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6820 osd.8 since back 2024-06-06T04:37:12.963430+0000 front 2024-06-06T04:37:12.963461+0000 (oldest deadline 2024-06-06T04:37:35.863474+0000) 2024-06-06T04:37:37.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:36 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:36.964+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-06-06T04:36:08.051455+0000 front 2024-06-06T04:36:08.051628+0000 (oldest deadline 2024-06-06T04:36:33.351389+0000) 2024-06-06T04:37:37.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:36 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:36.964+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-06-06T04:36:16.252834+0000 front 2024-06-06T04:36:16.252742+0000 (oldest deadline 2024-06-06T04:36:39.752775+0000) 2024-06-06T04:37:37.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:36 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:36.964+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-06-06T04:36:25.053885+0000 front 2024-06-06T04:36:25.053682+0000 (oldest deadline 2024-06-06T04:36:47.254380+0000) 2024-06-06T04:37:37.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:36 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:36.964+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-06-06T04:36:40.458199+0000 front 2024-06-06T04:36:40.458264+0000 (oldest deadline 2024-06-06T04:37:03.958144+0000) 2024-06-06T04:37:37.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:36 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:36.964+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-06-06T04:36:49.759561+0000 front 2024-06-06T04:36:49.759600+0000 (oldest deadline 2024-06-06T04:37:10.859708+0000) 2024-06-06T04:37:37.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:36 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:36.964+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-06-06T04:36:54.861305+0000 front 2024-06-06T04:36:54.861290+0000 (oldest deadline 2024-06-06T04:37:20.061800+0000) 2024-06-06T04:37:37.431 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:36 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:36.964+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.86:6820 osd.7 since back 2024-06-06T04:37:04.762784+0000 front 2024-06-06T04:37:04.762816+0000 (oldest deadline 2024-06-06T04:37:27.062676+0000) 2024-06-06T04:37:37.432 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:36 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11[61389]: 2024-06-06T04:37:36.964+0000 7fa91053b700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.154:6820 osd.8 since back 2024-06-06T04:37:12.963430+0000 front 2024-06-06T04:37:12.963461+0000 (oldest deadline 2024-06-06T04:37:35.863474+0000) 2024-06-06T04:37:38.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:37 smithi154 podman[71280]: 2024-06-06 04:37:37.916183993 +0000 UTC m=+5.600257921 container died c4e6efc6431e18f3b2d8a4538b94a2d9460a66918c7beb5e8ab00bd0a7fdb632 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-06T04:37:38.455 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:38 smithi154 podman[71280]: 2024-06-06 04:37:38.263133114 +0000 UTC m=+5.947207022 container cleanup c4e6efc6431e18f3b2d8a4538b94a2d9460a66918c7beb5e8ab00bd0a7fdb632 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-06T04:37:38.456 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:38 smithi154 bash[71280]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11 2024-06-06T04:37:38.886 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:38 smithi154 podman[71295]: 2024-06-06 04:37:38.456010438 +0000 UTC m=+0.530661024 container remove c4e6efc6431e18f3b2d8a4538b94a2d9460a66918c7beb5e8ab00bd0a7fdb632 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-06-06T04:37:39.137 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:38 smithi154 podman[71383]: 2024-06-06 04:37:38.789705393 +0000 UTC m=+0.017281457 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T04:37:39.137 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:38 smithi154 podman[71383]: 2024-06-06 04:37:38.898488083 +0000 UTC m=+0.126064136 container create e6f9229843d3bcb20a6d4adc2a2b6800c4b3cd578681c706a70d916ce2e1ec45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-06T04:37:39.137 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:39 smithi154 podman[71383]: 2024-06-06 04:37:39.134237599 +0000 UTC m=+0.361813663 container init e6f9229843d3bcb20a6d4adc2a2b6800c4b3cd578681c706a70d916ce2e1ec45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-06T04:37:39.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:39 smithi154 podman[71383]: 2024-06-06 04:37:39.137479341 +0000 UTC m=+0.365055402 container start e6f9229843d3bcb20a6d4adc2a2b6800c4b3cd578681c706a70d916ce2e1ec45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:37:39.430 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:39 smithi154 podman[71383]: 2024-06-06 04:37:39.234236544 +0000 UTC m=+0.461812617 container attach e6f9229843d3bcb20a6d4adc2a2b6800c4b3cd578681c706a70d916ce2e1ec45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T04:37:39.719 INFO:journalctl@ceph.osd.11.smithi154.stdout:Jun 06 04:37:39 smithi154 podman[71383]: 2024-06-06 04:37:39.431210908 +0000 UTC m=+0.658786969 container died e6f9229843d3bcb20a6d4adc2a2b6800c4b3cd578681c706a70d916ce2e1ec45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-06T04:37:39.994 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@osd.11.service' 2024-06-06T04:37:40.077 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:37:40.077 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-06-06T04:37:40.077 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-06-06T04:37:40.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.a 2024-06-06T04:37:40.379 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jun 06 04:37:40 smithi045 systemd[1]: Stopping Ceph mds.a for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:37:40.726 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jun 06 04:37:40 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-a[74643]: 2024-06-06T04:37:40.378+0000 7fe5a282c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:37:40.727 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jun 06 04:37:40 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-a[74643]: 2024-06-06T04:37:40.378+0000 7fe5a282c700 -1 mds.a *** got signal Terminated *** 2024-06-06T04:37:42.727 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jun 06 04:37:42 smithi045 podman[108472]: 2024-06-06 04:37:42.422131227 +0000 UTC m=+2.140156279 container died 4c72eb5e276405d355a2069cf0b8998adb9441642b3ff87020fb3fb8a16dc573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-06T04:37:43.156 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jun 06 04:37:42 smithi045 podman[108472]: 2024-06-06 04:37:42.811500666 +0000 UTC m=+2.529525715 container cleanup 4c72eb5e276405d355a2069cf0b8998adb9441642b3ff87020fb3fb8a16dc573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True) 2024-06-06T04:37:43.156 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jun 06 04:37:42 smithi045 bash[108472]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-a 2024-06-06T04:37:43.156 INFO:journalctl@ceph.mds.a.smithi045.stdout:Jun 06 04:37:43 smithi045 podman[108485]: 2024-06-06 04:37:43.029387025 +0000 UTC m=+0.602964651 container remove 4c72eb5e276405d355a2069cf0b8998adb9441642b3ff87020fb3fb8a16dc573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-06T04:37:43.176 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.a.service' 2024-06-06T04:37:43.211 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:37:43.212 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-06-06T04:37:43.212 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-06-06T04:37:43.212 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.d 2024-06-06T04:37:43.476 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jun 06 04:37:43 smithi045 systemd[1]: Stopping Ceph mds.d for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:37:43.977 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jun 06 04:37:43 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-d[73394]: 2024-06-06T04:37:43.565+0000 7f7ca63a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:37:43.977 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jun 06 04:37:43 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-d[73394]: 2024-06-06T04:37:43.565+0000 7f7ca63a4700 -1 mds.d *** got signal Terminated *** 2024-06-06T04:37:45.057 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jun 06 04:37:44 smithi045 podman[108594]: 2024-06-06 04:37:44.786605982 +0000 UTC m=+1.350419891 container died fc958d977ba81f455f2e27c1b44dd57b812506b939e5d612c96aa1e6b349ad22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-d, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-06-06T04:37:45.400 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jun 06 04:37:45 smithi045 podman[108594]: 2024-06-06 04:37:45.199223582 +0000 UTC m=+1.763037528 container cleanup fc958d977ba81f455f2e27c1b44dd57b812506b939e5d612c96aa1e6b349ad22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-d, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-06T04:37:45.400 INFO:journalctl@ceph.mds.d.smithi045.stdout:Jun 06 04:37:45 smithi045 bash[108594]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-d 2024-06-06T04:37:45.556 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.d.service' 2024-06-06T04:37:45.589 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:37:45.590 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-06-06T04:37:45.590 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-06-06T04:37:45.590 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.g 2024-06-06T04:37:45.903 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jun 06 04:37:45 smithi045 systemd[1]: Stopping Ceph mds.g for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:37:46.226 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jun 06 04:37:45 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-g[74223]: 2024-06-06T04:37:45.902+0000 7f2a9796b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:37:46.231 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jun 06 04:37:45 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-g[74223]: 2024-06-06T04:37:45.902+0000 7f2a9796b700 -1 mds.g *** got signal Terminated *** 2024-06-06T04:37:50.188 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jun 06 04:37:49 smithi045 podman[108713]: 2024-06-06 04:37:49.912502849 +0000 UTC m=+4.088954482 container died 011a566e7a776b3c2fbb06e67717e55484c4bc52c4bd6c769409c6c842942104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-g, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-06-06T04:37:50.477 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jun 06 04:37:50 smithi045 podman[108713]: 2024-06-06 04:37:50.352871791 +0000 UTC m=+4.529323470 container cleanup 011a566e7a776b3c2fbb06e67717e55484c4bc52c4bd6c769409c6c842942104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-g, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-06T04:37:50.477 INFO:journalctl@ceph.mds.g.smithi045.stdout:Jun 06 04:37:50 smithi045 bash[108713]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-g 2024-06-06T04:37:50.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.g.service' 2024-06-06T04:37:50.741 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:37:50.742 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-06-06T04:37:50.742 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-06-06T04:37:50.742 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.j 2024-06-06T04:37:51.065 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jun 06 04:37:50 smithi045 systemd[1]: Stopping Ceph mds.j for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:37:51.477 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jun 06 04:37:51 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-j[72977]: 2024-06-06T04:37:51.064+0000 7f050de42700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:37:51.477 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jun 06 04:37:51 smithi045 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-j[72977]: 2024-06-06T04:37:51.064+0000 7f050de42700 -1 mds.j *** got signal Terminated *** 2024-06-06T04:37:53.057 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jun 06 04:37:52 smithi045 podman[108834]: 2024-06-06 04:37:52.789155694 +0000 UTC m=+1.824412997 container died ef69f4b851cd7134968dc55e1fa7aa06f520fa43ef74ca7da74c44e3ed76fc05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-06T04:37:53.382 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jun 06 04:37:53 smithi045 podman[108834]: 2024-06-06 04:37:53.223829279 +0000 UTC m=+2.259086601 container cleanup ef69f4b851cd7134968dc55e1fa7aa06f520fa43ef74ca7da74c44e3ed76fc05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-j, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-06T04:37:53.408 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jun 06 04:37:53 smithi045 bash[108834]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-j 2024-06-06T04:37:53.614 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.j.service' 2024-06-06T04:37:53.636 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jun 06 04:37:53 smithi045 podman[108848]: 2024-06-06 04:37:53.383131693 +0000 UTC m=+0.590702141 container remove ef69f4b851cd7134968dc55e1fa7aa06f520fa43ef74ca7da74c44e3ed76fc05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=) 2024-06-06T04:37:53.636 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jun 06 04:37:53 smithi045 systemd[1]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.j.service: Deactivated successfully. 2024-06-06T04:37:53.637 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jun 06 04:37:53 smithi045 systemd[1]: Stopped Ceph mds.j for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:37:53.637 INFO:journalctl@ceph.mds.j.smithi045.stdout:Jun 06 04:37:53 smithi045 systemd[1]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.j.service: Consumed 1.008s CPU time. 2024-06-06T04:37:53.647 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:37:53.647 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-06-06T04:37:53.647 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-06-06T04:37:53.647 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.b 2024-06-06T04:37:53.954 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:37:53 smithi086 systemd[1]: Stopping Ceph mds.b for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:37:54.358 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:37:53 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-b[65935]: 2024-06-06T04:37:53.953+0000 7f40c2788700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:37:54.359 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:37:53 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-b[65935]: 2024-06-06T04:37:53.953+0000 7f40c2788700 -1 mds.b *** got signal Terminated *** 2024-06-06T04:37:57.697 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:37:57 smithi086 podman[70421]: 2024-06-06 04:37:57.435236854 +0000 UTC m=+3.585597331 container died fca99af9487b5b8fe96ba03cdde64797259c739ff737dcd15b1ca140d5c30175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-06-06T04:37:58.054 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:37:57 smithi086 podman[70421]: 2024-06-06 04:37:57.786798107 +0000 UTC m=+3.937158583 container cleanup fca99af9487b5b8fe96ba03cdde64797259c739ff737dcd15b1ca140d5c30175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-b, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-06-06T04:37:58.054 INFO:journalctl@ceph.mds.b.smithi086.stdout:Jun 06 04:37:57 smithi086 bash[70421]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-b 2024-06-06T04:37:58.217 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.b.service' 2024-06-06T04:37:58.252 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:37:58.252 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-06-06T04:37:58.253 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-06-06T04:37:58.253 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.e 2024-06-06T04:37:58.565 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:37:58 smithi086 systemd[1]: Stopping Ceph mds.e for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:37:58.859 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:37:58 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-e[65320]: 2024-06-06T04:37:58.565+0000 7f4f2da94700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:37:58.859 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:37:58 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-e[65320]: 2024-06-06T04:37:58.565+0000 7f4f2da94700 -1 mds.e *** got signal Terminated *** 2024-06-06T04:38:01.783 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:38:01 smithi086 podman[70542]: 2024-06-06 04:38:01.435555236 +0000 UTC m=+2.964716924 container died 3feb81881c0dff815d7ab28886432df5e29cb055023d9ea920506f497b871c27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-06T04:38:02.066 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:38:01 smithi086 podman[70542]: 2024-06-06 04:38:01.874052087 +0000 UTC m=+3.403213756 container cleanup 3feb81881c0dff815d7ab28886432df5e29cb055023d9ea920506f497b871c27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-e, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-06T04:38:02.067 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:38:01 smithi086 bash[70542]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-e 2024-06-06T04:38:02.314 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.e.service' 2024-06-06T04:38:02.339 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:38:02 smithi086 podman[70553]: 2024-06-06 04:38:02.066896131 +0000 UTC m=+0.631523229 container remove 3feb81881c0dff815d7ab28886432df5e29cb055023d9ea920506f497b871c27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-e, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-06-06T04:38:02.340 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:38:02 smithi086 systemd[1]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.e.service: Deactivated successfully. 2024-06-06T04:38:02.340 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:38:02 smithi086 systemd[1]: Stopped Ceph mds.e for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:38:02.340 INFO:journalctl@ceph.mds.e.smithi086.stdout:Jun 06 04:38:02 smithi086 systemd[1]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.e.service: Consumed 1.656s CPU time. 2024-06-06T04:38:02.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:38:02.353 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-06-06T04:38:02.353 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-06-06T04:38:02.353 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.h 2024-06-06T04:38:02.609 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:38:02 smithi086 systemd[1]: Stopping Ceph mds.h for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:38:03.109 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:38:02 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-h[65630]: 2024-06-06T04:38:02.702+0000 7fa7553f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:38:03.109 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:38:02 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-h[65630]: 2024-06-06T04:38:02.702+0000 7fa7553f5700 -1 mds.h *** got signal Terminated *** 2024-06-06T04:38:05.703 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:38:05 smithi086 podman[70674]: 2024-06-06 04:38:05.429148182 +0000 UTC m=+2.822995770 container died 8c8c48e2f25525a2518f284780f63b1bea8e517bc5ea25815aee1d26fa941a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=) 2024-06-06T04:38:06.078 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:38:05 smithi086 podman[70674]: 2024-06-06 04:38:05.802491741 +0000 UTC m=+3.196339327 container cleanup 8c8c48e2f25525a2518f284780f63b1bea8e517bc5ea25815aee1d26fa941a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-h, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:38:06.078 INFO:journalctl@ceph.mds.h.smithi086.stdout:Jun 06 04:38:05 smithi086 bash[70674]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-h 2024-06-06T04:38:06.231 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.h.service' 2024-06-06T04:38:06.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:38:06.265 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-06-06T04:38:06.265 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-06-06T04:38:06.266 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.k 2024-06-06T04:38:06.589 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jun 06 04:38:06 smithi086 systemd[1]: Stopping Ceph mds.k for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:38:06.859 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jun 06 04:38:06 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-k[65007]: 2024-06-06T04:38:06.589+0000 7fecbde68700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:38:06.859 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jun 06 04:38:06 smithi086 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-k[65007]: 2024-06-06T04:38:06.589+0000 7fecbde68700 -1 mds.k *** got signal Terminated *** 2024-06-06T04:38:08.665 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jun 06 04:38:08 smithi086 podman[70796]: 2024-06-06 04:38:08.393337459 +0000 UTC m=+1.911897822 container died 21cbfb7954d198c5ffd524c9c7aa944d4779695648704ee7124bb4afa94c9f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-06T04:38:08.990 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jun 06 04:38:08 smithi086 podman[70796]: 2024-06-06 04:38:08.747935729 +0000 UTC m=+2.266496092 container cleanup 21cbfb7954d198c5ffd524c9c7aa944d4779695648704ee7124bb4afa94c9f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-06T04:38:08.990 INFO:journalctl@ceph.mds.k.smithi086.stdout:Jun 06 04:38:08 smithi086 bash[70796]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-k 2024-06-06T04:38:09.114 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.k.service' 2024-06-06T04:38:09.146 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:38:09.147 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-06-06T04:38:09.147 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-06-06T04:38:09.147 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.c 2024-06-06T04:38:09.430 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:38:09 smithi154 systemd[1]: Stopping Ceph mds.c for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:38:09.930 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:38:09 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-c[66674]: 2024-06-06T04:38:09.536+0000 7f2310c0f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:38:09.930 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:38:09 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-c[66674]: 2024-06-06T04:38:09.536+0000 7f2310c0f700 -1 mds.c *** got signal Terminated *** 2024-06-06T04:38:13.895 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:38:13 smithi154 podman[71491]: 2024-06-06 04:38:13.453567515 +0000 UTC m=+4.105028453 container died 4bf74394285bd57e88cacb0b868a62e7b360003f2f29166afa8c178df689616e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-c, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-06-06T04:38:14.180 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:38:13 smithi154 podman[71491]: 2024-06-06 04:38:13.977626612 +0000 UTC m=+4.629087549 container cleanup 4bf74394285bd57e88cacb0b868a62e7b360003f2f29166afa8c178df689616e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-c, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T04:38:14.180 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:38:13 smithi154 bash[71491]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-c 2024-06-06T04:38:14.465 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.c.service' 2024-06-06T04:38:14.488 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:38:14 smithi154 podman[71514]: 2024-06-06 04:38:14.214248165 +0000 UTC m=+0.756904912 container remove 4bf74394285bd57e88cacb0b868a62e7b360003f2f29166afa8c178df689616e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-06T04:38:14.488 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:38:14 smithi154 systemd[1]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.c.service: Deactivated successfully. 2024-06-06T04:38:14.488 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:38:14 smithi154 systemd[1]: Stopped Ceph mds.c for c4f6b938-23bb-11ef-bc9b-c7b262605968. 2024-06-06T04:38:14.488 INFO:journalctl@ceph.mds.c.smithi154.stdout:Jun 06 04:38:14 smithi154 systemd[1]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.651s CPU time. 2024-06-06T04:38:14.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:38:14.502 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-06-06T04:38:14.502 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-06-06T04:38:14.502 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.f 2024-06-06T04:38:14.815 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:38:14 smithi154 systemd[1]: Stopping Ceph mds.f for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:38:15.180 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:38:14 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-f[66373]: 2024-06-06T04:38:14.814+0000 7fa58c484700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:38:15.180 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:38:14 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-f[66373]: 2024-06-06T04:38:14.814+0000 7fa58c484700 -1 mds.f *** got signal Terminated *** 2024-06-06T04:38:17.930 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:38:17 smithi154 podman[71623]: 2024-06-06 04:38:17.441358277 +0000 UTC m=+2.720659106 container died b1c620c0d3beb514b3b12e72d77e1a4c0e5a272d134ac48cebb41c58a23ffedf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-06T04:38:18.265 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:38:18 smithi154 podman[71623]: 2024-06-06 04:38:18.022659396 +0000 UTC m=+3.301960227 container cleanup b1c620c0d3beb514b3b12e72d77e1a4c0e5a272d134ac48cebb41c58a23ffedf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T04:38:18.265 INFO:journalctl@ceph.mds.f.smithi154.stdout:Jun 06 04:38:18 smithi154 bash[71623]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-f 2024-06-06T04:38:18.413 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.f.service' 2024-06-06T04:38:18.447 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:38:18.447 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-06-06T04:38:18.447 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-06-06T04:38:18.448 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.i 2024-06-06T04:38:18.930 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:38:18 smithi154 systemd[1]: Stopping Ceph mds.i for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:38:18.931 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:38:18 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-i[66974]: 2024-06-06T04:38:18.742+0000 7fe9e8e6d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:38:18.931 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:38:18 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-i[66974]: 2024-06-06T04:38:18.742+0000 7fe9e8e6d700 -1 mds.i *** got signal Terminated *** 2024-06-06T04:38:21.751 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:38:21 smithi154 podman[71743]: 2024-06-06 04:38:21.443981997 +0000 UTC m=+2.780319278 container died 456a88d250d145856dac083bceccd96cd029848a5321e57853bbf160cc9b55e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-i, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-06T04:38:21.751 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:38:21 smithi154 podman[71743]: 2024-06-06 04:38:21.751183056 +0000 UTC m=+3.087520331 container cleanup 456a88d250d145856dac083bceccd96cd029848a5321e57853bbf160cc9b55e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-i, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-06T04:38:22.117 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:38:21 smithi154 bash[71743]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-i 2024-06-06T04:38:22.117 INFO:journalctl@ceph.mds.i.smithi154.stdout:Jun 06 04:38:21 smithi154 podman[71755]: 2024-06-06 04:38:21.985556191 +0000 UTC m=+0.540188160 container remove 456a88d250d145856dac083bceccd96cd029848a5321e57853bbf160cc9b55e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-i, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=) 2024-06-06T04:38:22.135 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.i.service' 2024-06-06T04:38:22.167 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:38:22.167 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-06-06T04:38:22.168 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-06-06T04:38:22.168 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.l 2024-06-06T04:38:22.430 INFO:journalctl@ceph.mds.l.smithi154.stdout:Jun 06 04:38:22 smithi154 systemd[1]: Stopping Ceph mds.l for c4f6b938-23bb-11ef-bc9b-c7b262605968... 2024-06-06T04:38:22.930 INFO:journalctl@ceph.mds.l.smithi154.stdout:Jun 06 04:38:22 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-l[66070]: 2024-06-06T04:38:22.487+0000 7ff322e84700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T04:38:22.931 INFO:journalctl@ceph.mds.l.smithi154.stdout:Jun 06 04:38:22 smithi154 ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-l[66070]: 2024-06-06T04:38:22.487+0000 7ff322e84700 -1 mds.l *** got signal Terminated *** 2024-06-06T04:38:25.663 INFO:journalctl@ceph.mds.l.smithi154.stdout:Jun 06 04:38:25 smithi154 podman[71865]: 2024-06-06 04:38:25.377278763 +0000 UTC m=+2.988442655 container died 3ca7f1261d921251343152bd000696d58fdb3062548cf4a7baeb41431c2c92ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T04:38:25.930 INFO:journalctl@ceph.mds.l.smithi154.stdout:Jun 06 04:38:25 smithi154 podman[71865]: 2024-06-06 04:38:25.662888672 +0000 UTC m=+3.274052563 container cleanup 3ca7f1261d921251343152bd000696d58fdb3062548cf4a7baeb41431c2c92ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528) 2024-06-06T04:38:25.930 INFO:journalctl@ceph.mds.l.smithi154.stdout:Jun 06 04:38:25 smithi154 bash[71865]: ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-l 2024-06-06T04:38:25.930 INFO:journalctl@ceph.mds.l.smithi154.stdout:Jun 06 04:38:25 smithi154 podman[71879]: 2024-06-06 04:38:25.863822825 +0000 UTC m=+0.485983217 container remove 3ca7f1261d921251343152bd000696d58fdb3062548cf4a7baeb41431c2c92ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968-mds-l, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T04:38:26.001 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4f6b938-23bb-11ef-bc9b-c7b262605968@mds.l.service' 2024-06-06T04:38:26.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T04:38:26.033 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-06-06T04:38:26.033 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-06T04:38:26.234 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:38:28.791 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-06T04:38:28.981 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:38:31.353 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-06T04:38:31.552 INFO:teuthology.orchestra.run.smithi154.stdout:Deleting cluster with fsid: c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:38:33.861 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T04:38:33.896 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T04:38:33.924 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T04:38:33.951 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-06T04:38:33.953 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743045/remote/smithi045/crash 2024-06-06T04:38:33.954 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/crash -- . 2024-06-06T04:38:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:tar: /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-06T04:38:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:tar: Error is not recoverable: exiting now 2024-06-06T04:38:33.997 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743045/remote/smithi086/crash 2024-06-06T04:38:33.998 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/crash -- . 2024-06-06T04:38:34.039 INFO:teuthology.orchestra.run.smithi086.stderr:tar: /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-06T04:38:34.039 INFO:teuthology.orchestra.run.smithi086.stderr:tar: Error is not recoverable: exiting now 2024-06-06T04:38:34.041 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743045/remote/smithi154/crash 2024-06-06T04:38:34.042 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/crash -- . 2024-06-06T04:38:34.074 INFO:teuthology.orchestra.run.smithi154.stderr:tar: /var/lib/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-06T04:38:34.074 INFO:teuthology.orchestra.run.smithi154.stderr:tar: Error is not recoverable: exiting now 2024-06-06T04:38:34.075 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-06T04:38:34.075 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-06T04:38:34.113 INFO:tasks.cephadm:Compressing logs... 2024-06-06T04:38:34.113 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-06T04:38:34.156 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-06T04:38:34.158 DEBUG:teuthology.orchestra.run.smithi154:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-06T04:38:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.log 2024-06-06T04:38:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:: No such file or directory 2024-06-06T04:38:34.182 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-06T04:38:34.182 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-06-06T04:38:34.182 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-06-06T04:38:34.182 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-06T04:38:34.193 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.b.log: -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-06T04:38:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-06-06T04:38:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-06-06T04:38:34.194 INFO:teuthology.orchestra.run.smithi086.stderr: 86.8% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-06T04:38:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-06-06T04:38:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.h.log: 90.2% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-06T04:38:34.194 INFO:teuthology.orchestra.run.smithi086.stderr: 92.5% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-06-06T04:38:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-06T04:38:34.195 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-06-06T04:38:34.195 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.cephadm.log: 82.8% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-06T04:38:34.195 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-06T04:38:34.199 INFO:teuthology.orchestra.run.smithi045.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-06T04:38:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-06-06T04:38:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.log 2024-06-06T04:38:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-06T04:38:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-06-06T04:38:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-06-06T04:38:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-06T04:38:34.200 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-06T04:38:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-06-06T04:38:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.0.log: 86.8%gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-06-06T04:38:34.201 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-06T04:38:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mgr.x.log: 90.1% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-06T04:38:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-06-06T04:38:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-06T04:38:34.201 INFO:teuthology.orchestra.run.smithi045.stderr: 94.6% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-06-06T04:38:34.201 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.6.log: 94.7% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-06-06T04:38:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-06-06T04:38:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-06T04:38:34.202 INFO:teuthology.orchestra.run.smithi045.stderr: 83.0% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-06T04:38:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-06-06T04:38:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-06-06T04:38:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-06-06T04:38:34.202 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-volume.log: /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.d.log: 94.7% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-06-06T04:38:34.202 INFO:teuthology.orchestra.run.smithi154.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.log 2024-06-06T04:38:34.203 INFO:teuthology.orchestra.run.smithi154.stderr:: No such file or directory 2024-06-06T04:38:34.203 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-06T04:38:34.203 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-06-06T04:38:34.203 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-06T04:38:34.203 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-06T04:38:34.203 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-06-06T04:38:34.203 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.audit.log: /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-06-06T04:38:34.204 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-06-06T04:38:34.204 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-06-06T04:38:34.204 INFO:teuthology.orchestra.run.smithi154.stderr: 86.8% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-06T04:38:34.204 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-06T04:38:34.204 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mgr.z.log: /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.i.log: 90.3% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-06T04:38:34.204 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-06-06T04:38:34.204 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.cephadm.log: 82.7% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-06T04:38:34.204 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-06-06T04:38:34.205 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.11.log: 92.6% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-06-06T04:38:34.205 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-06T04:38:34.205 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-06-06T04:38:34.207 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-06-06T04:38:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-06T04:38:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-06-06T04:38:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.9.log: 91.1% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-06T04:38:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-06-06T04:38:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/cephadm.log: 94.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-06T04:38:34.222 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-06-06T04:38:34.223 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-06T04:38:34.226 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-06T04:38:34.227 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.k.log: 94.5% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-06-06T04:38:34.228 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-06T04:38:34.229 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.l.log: 94.8% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-06-06T04:38:34.232 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-06T04:38:34.242 INFO:teuthology.orchestra.run.smithi154.stderr: 93.6% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-06-06T04:38:34.244 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-06T04:38:34.246 INFO:teuthology.orchestra.run.smithi154.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-06T04:38:34.254 INFO:teuthology.orchestra.run.smithi086.stderr: 93.7% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-06-06T04:38:34.259 INFO:teuthology.orchestra.run.smithi154.stderr: 93.2% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-06-06T04:38:34.320 INFO:teuthology.orchestra.run.smithi154.stderr: 93.7% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-06-06T04:38:34.432 INFO:teuthology.orchestra.run.smithi045.stderr: 89.8% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-06-06T04:38:34.496 INFO:teuthology.orchestra.run.smithi154.stderr: 91.8% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-06-06T04:38:34.543 INFO:teuthology.orchestra.run.smithi086.stderr: 91.8% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-06-06T04:38:34.921 INFO:teuthology.orchestra.run.smithi045.stderr: 90.5% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-06-06T04:38:35.801 INFO:teuthology.orchestra.run.smithi086.stderr: 95.4% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-06-06T04:38:36.732 INFO:teuthology.orchestra.run.smithi154.stderr: 92.0% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-06-06T04:38:37.125 INFO:teuthology.orchestra.run.smithi086.stderr: 91.9% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-06-06T04:38:37.363 INFO:teuthology.orchestra.run.smithi154.stderr: 92.3% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-06-06T04:38:37.500 INFO:teuthology.orchestra.run.smithi154.stderr: 91.8% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-06T04:38:37.688 INFO:teuthology.orchestra.run.smithi086.stderr: 92.3% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-06-06T04:38:37.767 INFO:teuthology.orchestra.run.smithi086.stderr: 92.0% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-06T04:38:37.779 INFO:teuthology.orchestra.run.smithi045.stderr: 92.0% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-06-06T04:38:38.337 INFO:teuthology.orchestra.run.smithi045.stderr: 92.1% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-06T04:38:38.687 INFO:teuthology.orchestra.run.smithi154.stderr: 92.1% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-06T04:38:38.689 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-06-06T04:38:38.690 INFO:teuthology.orchestra.run.smithi154.stderr:real 0m4.517s 2024-06-06T04:38:38.690 INFO:teuthology.orchestra.run.smithi154.stderr:user 0m13.666s 2024-06-06T04:38:38.690 INFO:teuthology.orchestra.run.smithi154.stderr:sys 0m0.461s 2024-06-06T04:38:38.886 INFO:teuthology.orchestra.run.smithi045.stderr: 92.2% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-06T04:38:39.197 INFO:teuthology.orchestra.run.smithi086.stderr: 91.9% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-06T04:38:39.274 INFO:teuthology.orchestra.run.smithi045.stderr: 92.1% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-06-06T04:38:40.775 INFO:teuthology.orchestra.run.smithi045.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-06-06T04:38:40.777 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-06-06T04:38:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:real 0m6.606s 2024-06-06T04:38:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:user 0m24.389s 2024-06-06T04:38:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m0.733s 2024-06-06T04:38:49.476 INFO:teuthology.orchestra.run.smithi086.stderr: 93.5% -- replaced with /var/log/ceph/c4f6b938-23bb-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-06-06T04:38:49.478 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-06-06T04:38:49.478 INFO:teuthology.orchestra.run.smithi086.stderr:real 0m15.308s 2024-06-06T04:38:49.478 INFO:teuthology.orchestra.run.smithi086.stderr:user 0m31.286s 2024-06-06T04:38:49.479 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m0.994s 2024-06-06T04:38:49.479 INFO:tasks.cephadm:Archiving logs... 2024-06-06T04:38:49.479 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743045/remote/smithi045/log 2024-06-06T04:38:49.481 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-06T04:38:52.863 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743045/remote/smithi086/log 2024-06-06T04:38:52.864 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-06T04:38:57.280 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743045/remote/smithi154/log 2024-06-06T04:38:57.280 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-06T04:39:00.147 INFO:tasks.cephadm:Removing cluster... 2024-06-06T04:39:00.148 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 --force 2024-06-06T04:39:00.352 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:39:00.638 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 --force 2024-06-06T04:39:00.832 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:39:01.119 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c4f6b938-23bb-11ef-bc9b-c7b262605968 --force 2024-06-06T04:39:01.312 INFO:teuthology.orchestra.run.smithi154.stdout:Deleting cluster with fsid: c4f6b938-23bb-11ef-bc9b-c7b262605968 2024-06-06T04:39:01.579 INFO:tasks.cephadm:Removing cephadm ... 2024-06-06T04:39:01.579 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-06T04:39:01.596 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-06T04:39:01.612 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-06T04:39:01.630 INFO:tasks.cephadm:Teardown complete 2024-06-06T04:39:01.631 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-06T04:39:01.654 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-06T04:39:01.655 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-06T04:39:01.658 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-06T04:39:01.661 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-06T04:39:01.779 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-06T04:39:01.791 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-06T04:39:01.792 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T04:39:01.794 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T04:39:01.797 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T04:39:01.806 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-06-06T04:39:01.809 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-06-06T04:39:01.810 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T04:39:01.810 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-06-06T04:39:01.810 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv01.front.sepia.ceph.com 3 6 377 57 +558us[ +555us] +/- 46ms 2024-06-06T04:39:01.810 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 -469us[ -472us] +/- 57ms 2024-06-06T04:39:01.811 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 -392us[ -395us] +/- 91ms 2024-06-06T04:39:01.811 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T04:39:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-06-06T04:39:01.813 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T04:39:01.813 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-06-06T04:39:01.813 INFO:teuthology.orchestra.run.smithi086.stdout:^* hv01.front.sepia.ceph.com 3 7 377 58 +577us[ +554us] +/- 46ms 2024-06-06T04:39:01.813 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 57 -477us[ -477us] +/- 57ms 2024-06-06T04:39:01.813 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 -404us[ -404us] +/- 91ms 2024-06-06T04:39:01.814 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T04:39:01.815 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T04:39:01.815 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-06-06T04:39:01.815 INFO:teuthology.orchestra.run.smithi154.stdout:^* hv01.front.sepia.ceph.com 3 6 377 61 +567us[ +566us] +/- 46ms 2024-06-06T04:39:01.815 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 53 -465us[ -465us] +/- 57ms 2024-06-06T04:39:01.815 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 -396us[ -396us] +/- 91ms 2024-06-06T04:39:01.815 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T04:39:01.816 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-06T04:39:01.826 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-06T04:39:01.827 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-06T04:39:01.838 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-06T04:39:01.935 DEBUG:teuthology.orchestra.run.smithi086:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-06T04:39:02.001 DEBUG:teuthology.orchestra.run.smithi154:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-06T04:39:02.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T04:39:02.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T04:39:02.098 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T04:39:02.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T04:39:02.126 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T04:39:02.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T04:39:02.155 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-06T04:39:02.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-06T04:39:02.177 INFO:teuthology.task.internal:Duration was 3262.653732 seconds 2024-06-06T04:39:02.177 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-06T04:39:02.205 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-06T04:39:02.205 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-06T04:39:02.207 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-06T04:39:02.210 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-06T04:39:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T04:39:02.247 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T04:39:02.249 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T04:39:02.648 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-06T04:39:02.649 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-06-06T04:39:02.649 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-06T04:39:02.670 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-06-06T04:39:02.670 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-06T04:39:02.711 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2024-06-06T04:39:02.712 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-06T04:39:02.733 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-06T04:39:02.734 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-06T04:39:02.736 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-06T04:39:02.755 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-06T04:39:02.800 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-06T04:39:02.801 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-06T04:39:02.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-06T04:39:02.821 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-06T04:39:03.486 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-06T04:39:03.497 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-06T04:39:03.497 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-06T04:39:03.523 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-06T04:39:03.550 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-06T04:39:03.577 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-06T04:39:03.589 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-06T04:39:03.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-06T04:39:03.593 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-06T04:39:03.613 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-06-06T04:39:03.614 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-06-06T04:39:03.642 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2024-06-06T04:39:03.673 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-06T04:39:03.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T04:39:03.688 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-06T04:39:03.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T04:39:03.721 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-06T04:39:03.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T04:39:03.736 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-06T04:39:03.748 INFO:teuthology.task.internal:Transferring archived files... 2024-06-06T04:39:03.749 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743045/remote/smithi045 2024-06-06T04:39:03.749 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-06T04:39:03.802 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743045/remote/smithi086 2024-06-06T04:39:03.802 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-06T04:39:03.838 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743045/remote/smithi154 2024-06-06T04:39:03.838 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-06T04:39:03.874 INFO:teuthology.task.internal:Removing archive directory... 2024-06-06T04:39:03.874 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-06T04:39:03.876 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-06T04:39:03.878 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-06T04:39:03.924 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-06T04:39:03.939 INFO:teuthology.task.internal:Not uploading archives. 2024-06-06T04:39:03.939 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-06T04:39:03.950 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-06T04:39:03.950 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-06T04:39:03.953 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-06T04:39:03.955 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-06T04:39:03.966 INFO:teuthology.orchestra.run.smithi045.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 6 04:39 /home/ubuntu/cephtest 2024-06-06T04:39:03.968 INFO:teuthology.orchestra.run.smithi086.stdout: 265316 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 6 04:39 /home/ubuntu/cephtest 2024-06-06T04:39:03.981 INFO:teuthology.orchestra.run.smithi154.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 6 04:39 /home/ubuntu/cephtest 2024-06-06T04:39:03.983 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-06T04:39:03.995 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-06T04:39:04.022 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/fsx}} duration: 3262.6537322998047 failure_reason: 'Command failed (workunit test suites/fsx.sh) on smithi045 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d746ca5346f34eef7e826a1eb71e59e02b682ae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=5bbd1e28c2ad42fab247b8ab1a790b45 status: fail success: false 2024-06-06T04:39:04.023 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-06T04:39:04.108 INFO:teuthology.run:FAIL